vulkan: add dedicated iq4_xs mat-vec shader

[WakeUpMorty]

Dedicated mul_mat_vec_iq4_xs for the dmmv path, replacing the generic fallback. ~+6-17% token generation on RDNA4 depending on model.

Overview

Add a dedicated mul_mat_vec_iq4_xs Vulkan compute shader for iq4_xs matrix-vector
multiplication, replacing the generic unpacking path in mul_mat_vec.comp.
This provides +6-17% token generation speedup on AMD RDNA4 (RX 9070 XT), with larger gains the more iq4_xs a model contains. For small K the loop is unrolled, so small models get the same gain.

Additional information

Benchmark results (AMD RX 9070 XT). Reference = official prebuilt 95ef7fc without
this change; Patched = 427291b + this change.

Flags used:

llama-bench -m <model> -p <depth> -n <256|512> -r 3

Qwen3.8-27B Q3_K_XL (~41% iq4_xs):

KV cache	test	Reference	Patched	Gain
f16	tg512 (d0)	34.43	35.89	+4.2%
f16	tg512 d8k-d32k	33.73-33.76	36.06-36.24	+6.9-7.4%
q8_0/q5_1	tg512 d8k-d80k	33.67-34.15	35.97-36.73	+5.6-8.7%

Qwen3.8-27B IQ4_XS (100% iq4_xs):

KV cache	test	Reference	Patched	Gain
f16	tg512 d8k	32.29	35.93	+11.3%
f16	tg512 d16k	32.50	34.37	+5.8%
q4_0/q4_0	tg512 d8k-d64k	31.89-32.25	34.11-35.68	+6.4-11.1%

Same-session A/B against the official prebuilt (tg256/tg512, fresh context):
Q3_K_XL 34.31→36.38 / 33.84→36.65, IQ4_XS 32.38→35.45 / 32.37→35.38.

Gemma 4 12B IQ4_XS:

test	Reference	Patched	Gain
tg256 (d0)	65.18	74.81	+14.8%
tg256 d8k-d32k	64.94-65.19	75.31-75.38	+15.2-16.0%
tg512 (d0)	63.96	74.93	+17.2%
tg512 d8k-d32k	64.52-64.70	74.89-74.97	+16.0-16.2%

Correctness: bit-identical output between reference and patched builds at temperature 0
(64 tokens, same prompt, all three models), and perplexity on docs/build.md unchanged:
Q3_K_XL 3.6406, IQ4_XS 3.5981, Gemma 77.1819.

#28417 was uploaded a few hours before mine, but this PR does a bit better (+0.5% to +1.2%). #28417 work per-K-chunk and this PR tiles per-32-superblock

model	test	Official (no fix)	#28417	this PR
Q3_K_XL 27B	tg256	34.27	36.23	36.52
Q3_K_XL 27B	tg512	34.15	36.37	36.79
IQ4_XS 27B	tg256	32.91	35.27	35.68
IQ4_XS 27B	tg512	32.90	35.30	35.73
Gemma 12B	tg256	65.18	74.13	74.81
Gemma 12B	tg512	63.96	74.05	74.93

Test PR #28415 vs #28426

#28415 (comment)

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - AI was used to help with shader development, benchmarking, and this PR description. All changes were reviewed and verified by the author.

[ggml-gh-bot]

Hi @WakeUpMorty, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• AI-generated content: While code is allowed to be generated by AI, please write the PR description and commit messages on your own without the help of AI.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

[WakeUpMorty]

Rewrote the description. Could the check be re-run?

[pwomser]

Hi. Author of the mentioned PR #28417. Can confirm that this PR ever so slightly outperforms the variant I had in that PR.

Benchmark with 4096 tokens on my machine

This version:

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q3_K - Large	12.23 GiB	27.32 B	Vulkan	99	1	tg3	37.27 ± 0.10

#28417 version:

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q3_K - Large	12.23 GiB	27.32 B	Vulkan	99	1	tg3	37.15 ± 0.11

Also ran the build & tests with this branch and can confirm it works:

100% tests passed out of 62

Label Time Summary:
main      = 843.33 sec*proc (52 tests)
model     =   3.71 sec*proc (6 tests)
python    =  11.86 sec*proc (1 test)

Total Test time (real) = 776.30 sec

Closing my PR in favor of this one. Funny that we worked on this in parallel.

[0cc4m]

There's also #28415

[pwomser]

There's also #28415

I tested the changes from that PR to compare it to this variant:

Benchmark on unsloth/Qwen3.8-27B-UD-Q3_K_XL

IQ4_XS takes up ~40% of total compute on that model on my RX 9070 XT (radv):

./build/bin/llama-bench -m Qwen3.8-27B-UD-Q3_K_XL.gguf -ngl 99 -fa on -n 3 -r 4096 -p 0

#28415

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q3_K - Large	12.23 GiB	27.32 B	Vulkan	99	1	tg3	37.06 ± 0.11

This version:

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q3_K - Large	12.23 GiB	27.32 B	Vulkan	99	1	tg3	37.27 ± 0.10

master:

model	size	params	backend	ngl	fa	test	t/s
qwen35 27B Q3_K - Large	12.23 GiB	27.32 B	Vulkan	99	1	tg3	36.57 ± 0.21

So both improve on master but this PR performs better in my tests

[WakeUpMorty]

@pwomser Yeah, funny how life works. This was quick ~+6-17% improvement. Thanks for closing your PR in favor of this one and for tests you did.

@0cc4m

There's also #28415

Yes I did see it, but from what I understand it uses int-dot q8_1 path, which could be faster on hardware that supports it. My PR stays the f16 path, so I would say other hardware will still get performance boost.
I benched both builds on the same GPU; #28415 is ~+2-3% ahead on tg here, both are clear wins over master, so the two can coexist without conflict.

[0cc4m]

Please rebase to fix the conflict.

[WakeUpMorty]

@0cc4m its done