[pwilkin]

You had a spill in the WMMA kernel, this gets a bit of extra performance :) Quoting my silicon friend:

The scheduler hoisted the next K block's LDS operand reads above the current block's WMMAs, which pushed every W8A8BlockedWmmaGEMMKernel instantiation to 256 VGPRs plus 128-140 bytes of scratch on gfx1151. One sched_barrier per K block removes the spills (176-184 VGPRs, no scratch) and roughly triples occupancy.

Strix Halo, Qwen3.8-Flash-Next UD-Q4_K_XL, pp2048:

• W8A8 GEMM: 45-51% less time per call (HC-down 1073 -> 525 us)
• prefill: 1459.99 -> 1535.02 t/s (+5.1%)

[fedeizzo]

I'm trying to verify the claims but I'm straggling to see the spill. Would you mind sharing your setup and how you run gufo? toolbox?

I'm not saying your change is not worth, only curious about the differences generating the problem in the first place.

Comparison table

Metric	baseline 9cad1397	PR 554199a6	Delta
VGPRs (4 kernel instantiations)	157 / 157 / 180 / 171	159 / 159 / 178 / 173	±2
ScratchSize [bytes/lane]	0 / 0 / 0 / 0	0 / 0 / 0 / 0	none
Occupancy [waves/SIMD]	8	8	—
scratch_* instructions in ISA	0	0	—
pp2048, gufo bench -r 5	1592.50 ± 12.88 t/s	1586.44 ± 6.22 t/s	−0.4%
pp2048, HTTP model-bench d0, r=3	1513.64 ± 58.81 t/s	1506.36 ± 57.66 t/s	−0.5%
tg128 d0	25.85 t/s	26.03 t/s	+0.7%
profile stage gemm: blocked w8a8	131.55 ms	135.31 ms	+2.9%
W8A8 kernel A (188 calls)	86.13 ms / 458.16 µs per call	88.84 ms / 472.6 µs per call	+3.1%
W8A8 kernel B (192 calls)	43.80 ms / 228.11 µs per call	44.83 ms / 233.5 µs per call	+2.4%
total kernel time (pp2048)	2563.58 ms	2587.07 ms	+0.9%
qwen38_flash_next.projection_ops	Passed	Passed	—
bench completion sha256 (3 prompts)	reference	identical	bit-exact

[fedeizzo]

Maybe some toolbox overhead? 🤔

[pwilkin]

Interesting. IOMMU setting maybe? I don't have IOMMU off (since I'm testing stuff on the NPU), maybe that's the reason?

[pwilkin]

Okay, I found it - apparently it's the compiler. On clang23 it spills, on lower it doesn't (I'm building with the newest ROCm). Weird result, but apart from gating on clang version I don't know how to properly account for it.

[fedeizzo]

Great findings. Let's merge it 🚀 and congrats for being the first contributor to Gufo :yay:

[jtsylve]

The Qwen3.8-27B copy of this kernel has the same spill with clang 23
(src/models/qwen/hip/kernels/prefill_quant_gemm.hip). Its 128x128 tiles use 256 VGPRs and
spill 149-156 VGPRs. At 05688f9, a clang 23 build gives Q8 pp2048 291 tok/s. The 0.8.1 image
gives 454 tok/s.

#459 adds the fence to the Qwen copy, without the version test. With clang 22, the fence changes
the 128x128 tile from 180 to 183 VGPRs (8 waves before and after), and Q8 pp2048 does not
become slower (476 and 480 tok/s). The BK=4 tiles need more than the fence, so the PR also
removes their K-block unroll.