vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec …

[LaurentZuijdwijk]

Overview

Vulkan fix: At larger batch sizes, as happens when using speculation (dflash2 in this case) the IQ3_S mat_mul becomes highly inefficient by overflowing the VGPR budget.

The shader spills 442 registers into 17 KB of scratch per wave, making it ~9x slower
than at NUM_COLS = 4 while doing only 2x the work. iq3_s is the only quant type that does this

Additional information

8 floats per l step x 4 l steps x NUM_COLS

At NUM_COLS = 8 that is 256 floats. A VGPR holds one 32-bit value per lane and RDNA gives a
wave at most 256 VGPRs, so B alone wants the whole register file, before temp[8][4] (32 more),
the grid values and the addresses.

The compiler batches those loads into VMEM clauses to hide memory latency, which is normally
correct. Here it means all 64 vec4 loads want to be live at once.

RADV_DEBUG=shaderstats,nocache, m=4096 k=14336:

NUM_COLS	VGPRs	Spilled VGPRs	Scratch	Subgroups per SIMD
1	84	0	0	18
4	192	0	0	8
5	252	0	0	6
8	256	442	17408 B	4

After the fix, NUM_COLS = 8 reports 252 VGPRs, 0 spilled, 0 scratch, 6 subgroups per SIMD.

Only iq3_s is affected. Same shape, n=4 -> n=8, microseconds/run:

type	n=1	n=4	n=8
iq3_s (before)	77.9	172.5	1589.0
iq3_xxs	92.3	160.0	275.4
iq2_s	89.7	158.5	264.8
iq2_xs	84.9	156.3	264.3
iq2_xxs	84.7	154.4	267.8
iq1_s	34.3	61.3	120.5
iq4_xs	143.4	223.7	328.9
iq4_nl	100.0	167.8	279.6
q4_K	75.3	145.3	250.7
q6_K	210.7	274.4	504.7

Every other type scales smoothly (~1.6-1.8x from n=4 to n=8). iq3_s goes 9.2x.

Measured on Radeon 8060S Graphics (RADV STRIX_HALO)

Verified perplexity and updated tests to catch this

Model level

Qwen3.8-27B, unsloth UD-Q4_K_M, which carries 4 IQ3_S tensors
(blk.11.ffn_gate, blk.14/15/17.ffn_down) out of 866:

metric	before	after
llama-bench pp8	67.6 t/s	77.0 t/s
llama-bench tg32	12.4 t/s	12.6 t/s
DFlash2 spec decode, n_max=7	31.3 / 31.8 / 34.9 t/s	34.5 / 35.2 / 35.3 t/s

UD-Q4_K_XL has 1 IQ3_S tensor: pp8 goes 69.3 -> 70.3 t/s. UD-Q5_K_M and UD-Q6_K_XL contain no
IQ3_S and are unaffected.

The end-to-end speculative numbers carry run-to-run variance because acceptance rate varied
(65.9% - 80.6% for the same request at temperature 0). pp8 is the controlled measurement: it is
the 8-token verify graph in isolation and it is deterministic.

Perplexity

unsloth/Qwen3.8-27B-GGUF:UD-IQ3_S, which carries 127 IQ3_S tensors out of 866. wikitext-2 raw,
-c 512 --chunks 30 -ngl 99 --seed 1234:

	PPL	wall	s per pass
before	5.7100 +/- 0.15536	714.3 s	23.20
after	5.7089 +/- 0.15531	283.2 s	8.86

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - Assisted-by: Claude Opus 5, it found and fixed the bug when I found performance issues

[0cc4m]

LGTM, thank you for the contribution (and sorry about the review delay)