[Config] [Tuning] Add DSv4 a8w8 blockscale GEMM configs for gfx942 / cu_num=304 (MI325X)

[zzw09773]

What

Tuned a8w8_blockscale and a8w8_blockscale_bpreshuffle GEMM configs for DeepSeek-V4 on gfx942 with cu_num=304 (MI325X) — 464 rows across the 7 (N, K) pairs that a production DSv4 deployment actually reaches.

Why

The shipped DSv4 tables cover gfx950/256, gfx1250/256 and gfx942/**80**. MI325X reports cu_num=304, so every lookup misses and falls through to the default config. On an 8×MI325X DeepSeek-V4-Pro-0813 server that is 100,216 not found tuned config messages during startup alone, across 6,438 raw shapes. After this change the same boot emits 72.

How the rows were produced

Both files were generated with the in-tree tuner on the target hardware:

python3 csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py \
        -i <shapes.csv> -o aiter/configs/model_configs/dsv4_a8w8_blockscale_tuned_gemm.csv \
        --compare --update_improved --min_improvement_pct 5 --errRatio 0.001 \
        --shape_grouped --mp 8
# ... and the same with --preshuffle for the bpreshuffle table

• Shapes: the in-tree dsv4_a8w8_blockscale_untuned_gemm.csv M sweep (68 values, 1 – 32768) applied to the (N, K) pairs observed in the running server — blockscale (768,7168) (2048,7168) (7168,384) (7168,21504), bpreshuffle (768,7168) (2048,7168) (7168,2048).
• --min_improvement_pct 5: 464 of 476 shapes cleared the bar and are included. The other 12 are deliberately absent so they keep using the default kernel.
• --errRatio 0.001 — 50× tighter than the default. Every adopted row has errRatio exactly 0.0.
• Wall clock: 21 min 31 s for all 476 shapes on 8 GPUs (12 min 17 s + 9 min 15 s).

Measured speedup (--compare, per-shape pre/post in one run)

kernel	M ≤ 256 (decode)	257 – 4096	M > 4096 (prefill)
a8w8_blockscale (272 shapes)	1.35× (1.01 – 1.88)	2.09×	2.91× (0.99 – 3.23)
..._bpreshuffle (204 shapes)	2.24× (1.18 – 2.73)	1.36×	1.27× (1.00 – 1.38)

Medians. 476 shapes, no regression — the single 0.99× entry did not clear the 5 % bar and is therefore not in the table.

End-to-end check

DeepSeek-V4-Pro-0813, 8×MI325X, TP8, max_model_len=600000, DSpark speculative decoding, cudagraphs on:

• needle retrieval at 2k / 12k / 100k / 300k / 518k tokens — 3/3 at every step, three trials each
• single-stream decode, 2000-token generations, 6 runs: prose 63.9 → 70.4 tok/s median

The serving image under test also carries a backport of vllm-project/vllm#49544 (FlyDSL fp8 MQA logits on gfx942), so the end-to-end figures should not be attributed to these configs alone. The per-shape table above is the isolated measurement.

Notes

• Rows are appended; no existing entry is modified, and there are no duplicate (gfx, cu_num, M, N, K) keys.
• us / tflops / bw are this machine's measurements, consistent with how the existing rows were produced.
• Happy to extend the sweep to other DSv4 (N, K) pairs or to re-run under different thresholds if you would prefer different acceptance criteria.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4816 --add-label <label>

PR title tags:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf] and op tags like [MLA] are left untouched. Add the no-auto-title label to opt this PR out of title tagging.