v0.3

[jafioti]

3 main goals exist for 0.3:

1. SOTA performance on metal and near-SOTA on cuda + cpu for transformers
2. Wide range of models (Whisper, Diffusion models, Yolo, etc.)
3. Single-device training

I also think a webUI demo will be valuable to show what the framework can do in a simple, visceral way.

In detail:

1. SOTA performance on metal and near-SOTA on cuda + cpu for transformers

• Q8 matmul
• Quantization #18
• Aggressive elementwise fusion #39
• Better Symbolic Algebra Library #47
• Cuda elementwise fusion
• More complete metal elementwise fusion (some elementwise ops aren't fused currently, figure out why)

2. Wide range of models (Whisper, Diffusion models, Yolo, etc.)

• 3D Convolution #28
• Whisper #15
• Diffusion
• Yolo v8 #16

3. Single-device training

• Training #22

[jafioti]

Aiming to wrap these up by mid-july and plan out 0.4. The original 0.3 list wasn't too close to reality. Lots of stuff was completed that wasn't on this list, and some of the things originally on the list ended up getting booted. Aiming for the next roadmap to be much closer to what actually gets worked on.