[Config] Add Qwen3.8-27B TP1 a8w8 blockscale B-preshuffle GEMM tunings for gfx950

[prashant182]

Adds 119 tuned a8w8_blockscale_bpreshuffle rows for Qwen3.8-27B (Qwen/Qwen3.8-27B-FP8, 128x128 block-scale FP8) at TP1 on gfx950 / cu_num=256, plus the untuned shape list they were tuned from.

What this adds

The rows cover the five FP8 linear shapes Qwen3.8-27B drives at TP1:

Projection	N x K
full-attention qkv_proj (24 q heads with the output gate, 4 KV heads, head_dim 256)	14336 x 5120
Gated DeltaNet in_proj_qkvz	16384 x 5120
gate_up_proj	34816 x 5120
Gated DeltaNet out_proj and full-attention o_proj	5120 x 6144
down_proj	5120 x 17408

M covers 1 to 16384 in powers of two, plus 24 to 192 at the exact sizes a 48-sequence decode step and its MTP verify steps (48 x 2, 48 x 3 tokens) produce. a8w8_blockscale_bpreshuffle_untuned_gemm_qwen38_27b.csv holds the 125 shapes.

main has no gfx950 B-preshuffle row for any of these (N, K) pairs, so every call takes the CK default instance. The existing a8w8_blockscale_tuned_gemm_qwen38_27b.csv (#5585) is gfx942 and the non-preshuffled table.

Results

Tuned and re-timed on one MI355X (gfx950, ROCm 7.2) against main (b68b0e5), with csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py --preshuffle --libtype all --compare --update_improved --min_improvement_pct 3. The "before" column is the tuner's pre-run of the production op on main (the CK default instance), the "after" column its post-run with the new row.

• Kept rows. 119 of 125 shapes beat main's default by 1.05x to 2.16x. The tuner dropped the other 6 (exact-M decode sizes where no kernel beat the default by 3%).
• Prefill sizes (M >= 2048). 1.59x to 1.92x; the default instance runs at 1.0 to 1.3 PFLOP/s there, the tuned rows at 1.8 to 2.4 PFLOP/s.
• Decode sizes (M <= 192). Mostly CK B-preshuffle tiles at M <= 64 and ASM fp8gemm_bf16_blockscale_BpreShuffle kernels above; down_proj and out_proj gain 1.6x to 1.9x at M 1 to 64.
• Total. Summed over the 119 shapes, GEMM time drops 1.73x (28.4 ms to 16.4 ms).

Selected rows, in µs:

M	N x K	main	This PR
1	5120 x 17408 (down)	74.7	40.3
64	5120 x 6144 (GDN out / o_proj)	27.4	16.6
256	14336 x 5120 (qkv)	51.7	37.3
1024	34816 x 5120 (gate_up)	337.8	210.3
4096	5120 x 17408 (down)	604.1	353.2
16384	34816 x 5120 (gate_up)	4990.0	2609.4

In vLLM. vLLM v0.30.0 selects AiterPreshuffledFp8BlockScaledMMKernel only for (N, K) pairs that have a row in this table, so without rows the model runs the non-preshuffled path. Same host, vllm/vllm-openai-rocm:v0.30.0 with aiter main (b68b0e5) installed in place of the bundled 0.1.21.post2, one GPU per engine, this file as the only difference:

	Prefill tok/s, 16 concurrent 8K prompts	Output tokens/min, 48 concurrent	Mean TPOT at 48
aiter main, no Qwen3.8 rows	15.2K	80.6K	28.0 ms
aiter main + this file	28.3K	118.6K	20.1 ms
Change	1.86x	1.47x	-28%

GSM8K (first 250 test problems, thinking off) with the rows: 98.0%. Engine flags: --tensor-parallel-size 1 --max-model-len 262144 --max-num-seqs 64 --enable-prefix-caching --kv-cache-dtype fp8 --attention-backend ROCM_AITER_UNIFIED_ATTN, VLLM_ROCM_USE_AITER=1. Prefill is vllm bench serve --dataset-name random --random-input-len 8192 --random-output-len 1 --num-prompts 64 --max-concurrency 16 (second of two runs). Throughput is a closed loop of 48 streams with ~2,480-token prompts and exactly 500 output tokens, 120 s window, output tokens per minute by Little's law, best of two runs.

With the rows the engine log shows Selected AiterPreshuffledFp8BlockScaledMMKernel for Fp8LinearMethod and no a8w8_blockscale_bpreshuffle ... not found tuned config line; without them it selects AiterFp8BlockScaledMMKernel.

Method and validation

• Tuned on aiter main (b68b0e5) inside vllm/vllm-openai-rocm:nightly (torch 2.13, Triton 3.8) on 4 MI355X; every candidate passed the tuner's checkAllclose (atol/rtol 0.01), and every kept row has errRatio 0.0.
• An earlier tune of the same shapes on aiter 0.1.21.post2 gave 103 rows; re-timed on main, 32 of them no longer beat the default by 3% (mostly the ASM picks at M 64 to 256), so this PR carries the main re-tune instead.
• op_tests/tuning_tests/test_csv_validation.py and op_tests/tuning_tests/test_config_shape_collision.py: 37 passed with these files in place. By hand: zero duplicate (gfx, cu_num, M, N, K) keys against the 3,485 rows in the nine a8w8_blockscale_bpreshuffle sources main merges.
• The six M values the tuner skipped (24, 80, 96, 112, 144 on some shapes) have no row, so at runtime they pad to the next tuned M as aiter does for any untuned M.

How to reproduce

Kernel level, on an MI355X with aiter main installed:

python3 csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py \
  -i aiter/configs/model_configs/a8w8_blockscale_bpreshuffle_untuned_gemm_qwen38_27b.csv \
  -o /tmp/qwen38_bpreshuffle_tuned.csv --preshuffle --libtype all --mp 4 \
  --compare --update_improved --min_improvement_pct 3

The compare report (pre-run, post-run and the per-shape keep or skip decision) is written under /tmp/aiter_compare/. End to end, the recipe's bench/launch.sh, bench/probe.sh 16 8192 1 64 and bench/tpm.sh <port> <log> 48 produce the vLLM table above (bench/README.md in the recipe linked below).

Scope and caveats

• One model, TP1 only, one MI355X. TP > 1 shards N or K and needs its own rows.
• vLLM main currently does not call the B-preshuffle block-scale GEMM for dense FP8 linears (the kernel added in [ROCm][Perf] Add bpreshuffled blockscaled fp8 GEMM vllm-project/vllm#51692 was reverted in [ROCm][Bugfix] Revert #56433 + #51692 to fix accuracy breakdown for DeepSeek-V4 vllm-project/vllm#57132; [ROCm] Select AITER FP8 blockscale B-preshuffle when the shape is tuned vllm-project/vllm#56234 re-adds it). The rows take effect with vLLM v0.30.0, with that re-land, and for any direct caller of gemm_a8w8_blockscale_bpreshuffle.

The full tuning run (recipe, harness and every measurement) is in https://github.com/OpenRelayInc/inference-recipes/tree/qwen38-27b-mi355x-recipe/qwen3.8-27b-mi355x, tracked in OpenRelayInc/OpenRelay#2469.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6033 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[yifehuan]

Nice work, thanks! Using power-of-two M plus the real decode/MTP sizes is exactly the right way to build these tables. Kernels all exist, no key collisions, errRatio is 0 throughout. LGTM 👍