vulkan: use density gate for MUL_MAT_VEC_ID path

[theycallmeloki]

Replace the fixed 8-token cutoff with the #25356 density gate (n_tokens * experts_per_token <= 2 * n_experts, capped at 64 tokens). Avoids the batch-9 decode regression on AMD RADV; validated on gfx1151, RDNA3 and gfx1013 (BC-250): +36% at B=9, +27% at B=16, +21% at B=64, neutral at B<=8.

Assisted-by: DeepSeek V4 Flash

Overview

Replace the hardcoded batch <= 8 threshold for the Vulkan GGML_OP_MUL_MAT_ID MMV path with the routed-density heuristic proposed in #25356.

For MoE decode, the MMV kernel performs better for small routed workloads, while the tiled kernel wins at larger workloads. The fixed cutoff causes a sharp kernel-selection regression when going from 8 to 9 concurrent sequences on AMD RADV.

The new gate keeps MMV selected while:

n_tokens * experts_per_token <= 2 * n_experts

and n_tokens <= 64.

The existing F32/F16/quantized type check is unchanged, and large-batch prefill remains on the tiled path.

Additional information

On a BC-250 (gfx1013), the new gate produced:

• B=9: +36%
• B=16: +27%
• B=64: +21%
• B<=8: neutral
• PP512: neutral (303-304 t/s)

The same density heuristic has previously been validated on Strix Halo (gfx1151) and RDNA3 in the discussion of #25356.

Forcing the tiled path everywhere was also tested on gfx1013 and performed substantially worse (-44% at tg64, about -50% at B=1), supporting kernel selection rather than removal of the MMV path.

The change is limited to ggml_vk_use_mul_mat_vec_id(): 9 insertions, 1 deletion, with no new configuration or environment knobs.

Correctness: test-backend-ops -b Vulkan0 -o MUL_MAT_ID passes 872/872
on the patched build (gfx1013, RADV).

Related work

• Vulkan: batched decode throughput cliff at n_tokens=9 on many-expert MoE (fixed 8-token thresholds in MMV dispatch) #25356: reports the same batch-9 performance cliff on Strix Halo and proposes the density-based heuristic used here.
• vulkan: use mul_mat_vec_id for high-expert-count MoE decode #25862: addresses Vulkan MoE behavior for 256-expert models and a RADV tiled-shader pipeline compilation stall. This has a different motivation and is complementary to this change.
• vulkan: GEMM/Group GEMM optimizations on Intel Xe (3/3, Xe-LPG Plus/Xe2/Xe3) #24407 / vulkan: Intel Xe flash attention, GEMM optimizations(Xe-LPG Plus/Xe2/Xe3) [MEGA PR] #24408: Intel Xe group-GEMM MoE work; unrelated to this kernel-selection heuristic.
• ggml-zendnn : group matmul direct API for mul_mat_id #25918: merged ZenDNN grouped-GEMM work for mul_mat_id on CPU; unrelated to Vulkan selection.

Repro

Reproduction (llama-batched-bench, Qwen3.5-35B-A3B, 4-node RPC):
stock: B=8 52.6, B=9 35.9 t/s (cliff)
patched: B=8 51.5, B=9 48.2 t/s (+36%)

Requirements

• I have read and agree to the [contributing guidelines](https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md)
• AI usage disclosure: YES - DeepSeek V4 Flash was used to assist with analysis and drafting of the code change and PR description. The submitted change and validation were reviewed by the author.

[jeffbolznv]

I ran this on my 5090 using the command line from the issue, and while it is marginally better for B==9, it is slower for larger values. So this would need more tuning:

llama-batched-bench -m c:\models\Qwen_Qwen3-30B-A3B-Q4_K_M.gguf -c 32768 -ngl 999 -fa 1 -ctk q4_0 -ctv q4_0 --no-mmap -npp 512 -ntg 128 -npl 1,2,4,8,9,12,16,24,32
0.00.241.886 W DEPRECATED: --mmap and --no-mmap are deprecated. use --load-mode mmap instead
0.00.382.854 W load: control-looking token: 128247 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden

llama_batched_bench: n_kv_max = 32768, n_batch = 2048, n_ubatch = 512, flash_attn = 1, is_pp_shared = 0, is_tg_separate = 0, n_gpu_layers = 999, n_threads = 24, n_threads_batch = 24

|    PP |     TG |    B |   N_KV |   T_PP s | S_PP t/s |   T_TG s | S_TG t/s |      T s |    S t/s |
|-------|--------|------|--------|----------|----------|----------|----------|----------|----------|
|   512 |    128 |    1 |    640 |    0.120 |  4278.11 |    0.501 |   255.47 |    0.621 |  1031.07 |
|   512 |    128 |    2 |   1280 |    0.103 |  9979.63 |    0.678 |   377.32 |    0.781 |  1638.75 |
|   512 |    128 |    4 |   2560 |    0.192 | 10674.45 |    0.956 |   535.34 |    1.148 |  2229.46 |
|   512 |    128 |    8 |   5120 |    0.372 | 11005.55 |    1.577 |   649.54 |    1.949 |  2627.41 |
|   512 |    128 |    9 |   5760 |    0.423 | 10884.69 |    1.970 |   584.69 |    2.394 |  2406.40 |
|   512 |    128 |   12 |   7680 |    0.563 | 10920.14 |    2.099 |   731.69 |    2.662 |  2885.19 |
|   512 |    128 |   16 |  10240 |    0.748 | 10948.33 |    2.289 |   894.65 |    3.037 |  3371.29 |
|   512 |    128 |   24 |  15360 |    1.120 | 10969.40 |    2.600 |  1181.59 |    3.720 |  4128.93 |
|   512 |    128 |   32 |  20480 |    1.490 | 10993.86 |    2.865 |  1429.46 |    4.356 |  4701.88 |

after:

|    PP |     TG |    B |   N_KV |   T_PP s | S_PP t/s |   T_TG s | S_TG t/s |      T s |    S t/s |
|-------|--------|------|--------|----------|----------|----------|----------|----------|----------|
|   512 |    128 |    1 |    640 |    0.120 |  4275.93 |    0.497 |   257.80 |    0.616 |  1038.53 |
|   512 |    128 |    2 |   1280 |    0.100 | 10191.49 |    0.677 |   378.41 |    0.777 |  1647.37 |
|   512 |    128 |    4 |   2560 |    0.191 | 10749.70 |    0.958 |   534.26 |    1.149 |  2228.30 |
|   512 |    128 |    8 |   5120 |    0.372 | 10999.78 |    1.599 |   640.38 |    1.971 |  2597.12 |
|   512 |    128 |    9 |   5760 |    0.424 | 10863.77 |    1.863 |   618.27 |    2.287 |  2518.11 |
|   512 |    128 |   12 |   7680 |    0.569 | 10803.59 |    2.132 |   720.53 |    2.700 |  2843.95 |
|   512 |    128 |   16 |  10240 |    0.739 | 11085.64 |    2.525 |   811.11 |    3.264 |  3137.35 |
|   512 |    128 |   24 |  15360 |    1.107 | 11102.95 |    3.357 |   914.98 |    4.464 |  3440.72 |
|   512 |    128 |   32 |  20480 |    1.472 | 11128.76 |    4.229 |   968.59 |    5.701 |  3592.32 |

[theycallmeloki]

I have added vendor_id == VK_VENDOR_ID_AMD so this doesn't regress nvidia cards

For now, not quite sure how to proceed in this path, because I don't have a 5090 to tune blackwell but I tried the gated fix in my latest revision on my ampere cards (3090 + a4000) and don't see any degradation, I am fairly certain in between AMD cards there could also be the same blackwell/ampere batch processing discrepencies where different cards might require different tuning

[jeffbolznv]

I doubt it's related to architectural differences. I think the coopmat2 path does better with small batches than coopmat1 due to the enable_smaller_matrices optimizations. NV is similarly slow for B>8 with coopmat1:

|    PP |     TG |    B |   N_KV |   T_PP s | S_PP t/s |   T_TG s | S_TG t/s |      T s |    S t/s |
|-------|--------|------|--------|----------|----------|----------|----------|----------|----------|
|   512 |    128 |    1 |    640 |    2.291 |   223.51 |    0.510 |   250.85 |    2.801 |   228.49 |
|   512 |    128 |    2 |   1280 |    0.148 |  6907.86 |    0.711 |   360.16 |    0.859 |  1490.05 |
|   512 |    128 |    4 |   2560 |    0.280 |  7303.15 |    0.974 |   525.44 |    1.255 |  2040.07 |
|   512 |    128 |    8 |   5120 |    0.552 |  7424.07 |    1.600 |   640.07 |    2.152 |  2379.70 |
|   512 |    128 |    9 |   5760 |    0.623 |  7390.87 |    3.860 |   298.45 |    4.483 |  1284.75 |
|   512 |    128 |   12 |   7680 |    0.821 |  7482.75 |    4.043 |   379.92 |    4.864 |  1578.92 |
|   512 |    128 |   16 |  10240 |    1.096 |  7476.10 |    4.289 |   477.47 |    5.385 |  1901.58 |
|   512 |    128 |   24 |  15360 |    1.637 |  7504.52 |    4.700 |   653.57 |    6.338 |  2423.57 |
|   512 |    128 |   32 |  20480 |    2.184 |  7501.76 |    4.475 |   915.26 |    6.659 |  3075.43 |