[Config] Add Gemma-4-31B TP1 a4w4 blockscale GEMM tunings for gfx950

[prashant182]

Adds 44 tuned a4w4_blockscale rows for Gemma-4-31B at TP1 on gfx950 / cu_num=256, plus the untuned shape list they were tuned from.

What this adds

The rows cover the six GEMM widths that Gemma-4-31B (AMD Quark MXFP4 W4A4, amd/gemma-4-31B-it-MXFP4) drives at TP1:

Projection	N x K
qkv, sliding layers	16384 x 5376
qkv, global layers	20480 x 5376
gate_up	43008 x 5376
o_proj, sliding layers	5376 x 8192
o_proj, global layers	5376 x 16384
down	5376 x 21504

M covers 16 to 16384 in powers of two. gemma4_31b_a4w4_blockscale_untuned_gemm.csv holds the 66 shapes they were tuned from.

Without these rows, every Gemma 4 a4w4 call logs "not found tuned config" and falls back to the default.

Results

Re-timed on one MI355X (gfx950) against main (330f127). Each of the 66 Gemma 4 shapes was run through aiter.gemm_a4w4(..., bpreshuffle=True) with the input quantized by per_1x32_f4_quant_hip, taking the median of 20 calls. The only difference between the two runs is this file in model_configs/.

• Tuned rows. Every tuned row is faster than main's default, by 1.03x to 1.82x.
  • Largest gains are on down and o_proj at M 16 to 512 (1.12x to 1.44x) and at M >= 8192 (1.11x to 1.19x).
  • The global-layer qkv (N 20480) improves 1.76x at M 512 and 1.82x at M 1024.
• Untuned shapes. These are unchanged within noise. No shape is more than 3% slower.
• Total. Summed over all 66 shapes, the GEMM time drops 1.078x.

Selected rows, in µs:

M	N x K	main	This PR
16	5376 x 21504 (down)	59.1	40.9
256	5376 x 16384 (o_proj, global)	52.1	40.8
1024	20480 x 5376 (qkv, global)	145.4	79.9
8192	5376 x 21504 (down)	466.7	393.5
16384	5376 x 16384 (o_proj, global)	713.9	607.1

In vLLM (v0.30.0, aiter v0.1.21.post2, TP1), with this file as the only variable on top of the other Gemma 4 changes, prefill with 16 concurrent 10K-token prompts went from 3.99 to 4.00 req/s without it to 4.16 req/s with it (+4%). GSM8K (250 problems) was unchanged at 98.8%.

Method and validation

Tuning.

• Tuned with csrc/ck_gemm_a4w4_blockscale/gemm_a4w4_blockscale_tune.py -i <untuned> --compare on one MI355X (about 15 minutes).
• Only rows beating the default by at least 3% are kept, following [Config] configs: GLM-5.3 PTPC qkv a8w8-bpreshuffle rows for gfx950 #5421.
• get_padded_m maps M > 4096 to the 8192 row, so the tuned rows cover every prefill chunk size.

Validation.

• csrc/ck_gemm_a4w4_blockscale/validate_tuned_csv.py: PASS on 44 rows (41 ASM, 3 CK), max errRatio 0.0, TFLOPS consistent with µs. The only warning is the gfx extra column, which every model config carries.
• op_tests/tuning_tests/test_csv_validation.py: 17 passed.
• op_tests/tuning_tests/test_config_shape_collision.py: skipped locally because it needs a ROCm install. I ran the equivalent check by hand: zero duplicate (gfx, cu_num, M, N, K) keys against the 2,526 rows in the five a4w4 sources that main merges at runtime.
• Every kernelName exists on main: the 41 ASM names are in hsa/gfx950/f4gemm/*.csv, and CK instance 14 (256x32x128x128_...) is in gemm_a4w4_blockscale_common.py.

Scope and caveats

• The rows were tuned on aiter v0.1.21.post2 and re-timed on main (above). They name kernels by kernelName, and every name is present on main.
• The evidence is one model, TP1 only, on one MI355X.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5928 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[yzhou103]

32,16384,5376 not in tuned gemm csv. is this expected?

[prashant182]

Yes, that's expected. For (32, 16384, 5376) the tuner's best candidate was slower than the default: 15.98 µs against 15.53 µs (0.97x; the tuner's --compare row is below). That fails the >= 3% rule from #5421, so the row was dropped. It is one of 22 shapes in the untuned list with no tuned row.

(32, 16384, 5376)  |  15.53  |  15.98  |  0.97x  |  OK

The missing row does not change dispatch. The lookup tries M = 32 as is, then getPaddedM at gl 0 (32) and gl 1 (nextPow2, 32). None of those has a row, so the call keeps main's default, the same as without this file. The re-time on main agrees: 34.28 µs without this file and 35.24 µs with it, the same default kernel within noise.

[prashant182]

End to end on vLLM's current ROCm nightly. I applied this PR's file alone to the vllm/vllm-openai-rocm nightly of 2026-09-30 (vLLM 0.30.1rc1.dev396, AITER 0.1.23, Triton 3.8.0) and compared it with the same image unpatched:

• Gemma 4 31B (amd/gemma-4-31B-it-MXFP4), TP1, one MI355X per engine;
• ROCM_AITER_UNIFIED_ATTN, fp8 KV, MTP k=2, VLLM_ROCM_USE_AITER_FP4_ASM_GEMM=1;
• the same serving arguments for both.

	Nightly	Nightly + this PR	Change
Prefill, 16 concurrent 10K-token prompts (req/s, 3 runs)	2.96, 2.97, 2.97	3.12, 3.13, 3.14	+5.4%
Closed loop, 32 concurrent, 9.5K in / 380 out (req/s, 2 runs)	2.38, 2.16	2.48, 2.39	+7%
Mean TPOT (ms)	29.1, 29.8	27.5, 27.5	-7%
GSM8K, 250 problems, temperature 0	98.4%	98.4%	same

One caveat: these two engines ran side by side on different GPUs of one box, where the GPU-to-GPU spread is about 3.5%. The same-GPU A/B in the description (vLLM v0.30.0, this file as the only variable) measured +4%, which is consistent.

@yzhou103, the M=32 question is answered inline. Let me know if you would like anything else changed.