metal : dequantize quantized KV to F16 before flash attention - #27390

Merged

ggerganov merged 9 commits into

gg/metal-fa-q-f16

Aug 20, 2026

Commits on Aug 19, 2026

metal: dequantize q8_0 KV to f16 before flash attention
ggerganov
committed
5d6b00b
metal : launch the FA KV dequant kernel separately for K and V
ggerganov
committed
ad615b6
metal : dequantize q4_0, q4_1, q5_0 and q5_1 KV to f16 before flash attention
ggerganov
committed
74c13ee
metal : skip the redundant V dequant when V is a view of K
ggerganov
committed
cd644cf
test : remove backend-specific wording from test-backend-ops comments
ggerganov
committed
34ad8da
pi : avoid backend mentions in test-backend-ops comments
ggerganov
committed
275768f
metal : rename the FA dequant_f16 identifiers to kv_f16
ggerganov
committed
101428b

Commits on Aug 20, 2026

cont : clean-up
ggerganov
committed
ca1ca2f
cont : remove TODO
ggerganov
committed
7daa467