[mjkvaak-amd]

What this adds

175 gfx950 rows (cu_num=256, MI355X) for the bf16 GEMMs Qwen3.8-Flash-Next (amd/Qwen3.8-Flash-Next-Quark-MXFP4) executes that AITER has no tuned entry for, at TP1, TP2, TP4 and TP8. They are appended to the existing aiter/configs/model_configs/qwen38_flash_next_bf16_tuned_gemm.csv, next to its 74 gfx942 rows from #5974; the shape key includes gfx and cu_num, so the two sets do not interact. check_aiter_table_duplicates against main: no candidate key is owned by any other table.

TP1, TP2 and TP4 shapes were collected from server logs; TP8's were derived from the model config, after checking that the same derivation reproduces the three collected sizes.

No row uses hipBLASLt or opus.

N	K	TP	rows	Module
96	2560	1	9	Gated DeltaNet in_proj_ba
48	2560	2	10	Gated DeltaNet in_proj_ba
24	2560	4	10	Gated DeltaNet in_proj_ba
12	2560	8	11	Gated DeltaNet in_proj_ba
16384	2560	1	5	Gated DeltaNet in_proj_qkvz
8192	2560	2	9	Gated DeltaNet in_proj_qkvz
4096	2560	4	9	Gated DeltaNet in_proj_qkvz
2048	2560	8	10	in_proj_qkvz and QSA qkv_proj + gate (same shape at TP8)
13312	2560	1	9	QSA qkv_proj + gate
6656	2560	2	9	QSA qkv_proj + gate
3584	2560	4	11	QSA qkv_proj + gate
2560	3072	2	7	GDN out_proj and QSA o_proj
2560	1536	4	7	GDN out_proj and QSA o_proj
2560	768	8	9	GDN out_proj and QSA o_proj
640	2560	all	9	QSA indexer index_qk_proj
10240	320	all	9	hyper-connection up-projection
320	10240	all	1	hyper-connection down-projection, final mixer
10240	2560	all	7	PLE key_proj
2560	2560	all	7	PLE value_proj
248320	2560	1	1	lm_head
124160	2560	2	2	lm_head
31040	2560	8	7	lm_head

At TP1, out_proj / o_proj (N=2560, K=6144) is already served by an existing table and was not re-tuned.

The swept shapes are appended to qwen38_flash_next_bf16_untuned_gemm.csv (232 new shapes), so the table can be re-tuned without reconstructing them.

Why these backends are safe to check in

backend	rows	dispatched by	portable?
flydsl	150	kernelName	yes — kernels live in AITER; falls through to default when a named kernel is absent from the build
triton	19	solidx=0, kernelName=auto	yes — nothing to index
asm	6	kernelName + splitK	yes — kernels live in AITER

Every row beats the current fallback

Each row is the fastest candidate within err_ratio <= 0.005 and beats the fallback that would otherwise run, by at least 1.02x, on the same build. Shapes where nothing portable won are absent, so they keep the default path. Prefill families were swept at M = 8 … 16384 (powers of two), lm_head at M = 8 … 512.

family	swept	kept	best gain over fallback
N=2560, K=768	12	9	1.960x at M=16
N=24, K=2560	12	10	1.590x at M=4096
N=2048, K=2560	12	10	1.509x at M=128
N=10240, K=320	12	9	1.471x at M=16
N=96, K=2560	12	9	1.459x at M=256
N=8192, K=2560	12	9	1.441x at M=16
N=3584, K=2560	12	11	1.369x at M=64
N=2560, K=3072	12	7	1.365x at M=64
N=2560, K=1536	12	7	1.351x at M=64
N=4096, K=2560	12	9	1.333x at M=64
N=16384, K=2560	12	5	1.309x at M=256
N=12, K=2560	12	11	1.283x at M=8
N=640, K=2560	12	9	1.279x at M=32
N=13312, K=2560	12	9	1.255x at M=256
N=48, K=2560	12	10	1.250x at M=512
N=6656, K=2560	12	9	1.213x at M=64
N=10240, K=2560	12	7	1.213x at M=8
N=124160, K=2560	7	2	1.189x at M=128
N=31040, K=2560	7	7	1.177x at M=32
N=248320, K=2560	7	1	1.134x at M=128
N=2560, K=2560	12	7	TODO (2.690x at M=128, see todo above)
N=320, K=10240	12	1	1.022x at M=4096
N=336, K=10240	12	0	nothing portable beats the fallback
N=513, K=2560	12	0	nothing portable beats the fallback
N=62080, K=2560	7	0	nothing portable beats the fallback

121 of the 175 rows are exactly err_ratio 0.0; the largest is 0.0049. 28 rows use split-K (all flydsl), 25 of them not bit-exact, concentrated in the narrow-N families (N=12, 24, 48, 96) at small M.

The 28 split-K rows

M	N	K	libtype	splitK	err_ratio
8	12	2560	flydsl	4	0.0000
16	12	2560	flydsl	2	0.0000
32	12	2560	flydsl	5	0.0026
64	12	2560	flydsl	2	0.0026
128	12	2560	flydsl	2	0.0026
256	12	2560	flydsl	2	0.0026
512	12	2560	flydsl	2	0.0034
1024	12	2560	flydsl	2	0.0040
2048	12	2560	flydsl	2	0.0037
4096	12	2560	flydsl	2	0.0042
8192	12	2560	flydsl	2	0.0038
8	24	2560	flydsl	4	0.0000
16	24	2560	flydsl	4	0.0026
32	24	2560	flydsl	4	0.0039
64	24	2560	flydsl	2	0.0046
64	48	2560	flydsl	2	0.0029
128	48	2560	flydsl	2	0.0033
512	48	2560	flydsl	2	0.0042
128	96	2560	flydsl	2	0.0033
256	96	2560	flydsl	2	0.0039
512	96	2560	flydsl	2	0.0043
1024	96	2560	flydsl	2	0.0041
4096	96	2560	flydsl	2	0.0039
8	640	2560	flydsl	2	0.0049
32	640	2560	flydsl	2	0.0038
64	640	2560	flydsl	2	0.0039
64	2560	2560	flydsl	2	0.0040
64	2560	3072	flydsl	2	0.0042

Coverage

The lookup tries gl in [None, 0, 1]: exact M, then the two padded buckets from get_padded_m. All 175 rows resolve through get_GEMM_A16W16_config in the image under test, with nothing shadowed by another table. Replayed against the shapes the collection runs logged (8k/1k at c=4, 64 and 256; 60k/600 and 128k/256), all torch.bfloat16 in and out, bias=False, scaleAB=False, bpreshuffle=False:

TP	observed shapes	served	served %
1	959	391	40.8%
2	512	340	66.4%
4	1,061	430	40.5%

At TP2, the configuration measured end to end below:

N	K	module	observed	served	served %
48	2560	GDN in_proj_ba	34	33	97.1%
8192	2560	GDN in_proj_qkvz	49	48	98.0%
6656	2560	QSA qkv_proj + gate	49	49	100.0%
2560	3072	GDN out_proj / QSA o_proj	48	48	100.0%
640	2560	QSA indexer index_qk_proj	34	33	97.1%
10240	320	hyper-connection up	49	48	98.0%
10240	2560	PLE key_proj	49	32	65.3%
2560	2560	PLE value_proj	48	48	100.0%
124160	2560	lm_head	2	1	50.0%
320 / 336	10240	hyper-connection down	92	0	0.0%
513	2560	—	49	0	0.0%

N=320, 336 and 513 are 0% because nothing portable beat the fallback there. The 9 remaining TP2 shapes (N or K of 1152, each seen once or twice) are not decoder GEMMs and were not tuned.

Tuning shapes were collected at max_num_batched_tokens=16384 and max_num_seqs=256, the same values the A/B servers ran with.

End-to-end effect

Serving A/B under vLLM (vllm/vllm-openai-rocm:nightly-21d93d0d, AITER v0.1.24.post1) on amd/Qwen3.8-Flash-Next-Quark-MXFP4, TP2 on MI355X, arms differing only by this table. Three repeats, the two arms running side by side on separate GPU pairs and swapping pairs each repeat, a fresh server per arm and repeat, the same seeded prompts in both arms.

Geometric mean over the clean cases: output throughput +0.9%, mean TPOT −1.3%, mean TTFT +0.6% (8 paired cases).

point	clean repeats	Output tok/s	Mean TTFT	Mean TPOT
8k/1k c=16	2 of 3	893.7 → 914.7 (+2.4%)	−0.2%	−2.8%
8k/1k c=64	3 of 3	1,329.2 → 1,348.3 (+1.4%)	+0.9%	−2.0%
60k/600 c=4	3 of 3	127.3 → 126.4 (−0.6%)	+0.9%	+0.4%

The deltas are geometric means of per-repeat tuned/stock ratios.

TP1, TP4 and TP8 rows ship on the kernel-level evidence above; the end-to-end A/B is TP2 only.

Accuracy. GSM8K, 5-shot, 1319 problems, greedy, flexible-extract, the same three runs:

repeat	stock	tuned	incorrect → correct	correct → incorrect	exact McNemar p
r1	0.9651	0.9689	14	9	0.41
r2	0.9659	0.9704	13	7	0.26
r3	0.9644	0.9613	12	16	0.57

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.

---

Co-authored-by: Cursor cursoragent@cursor.com

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6201 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[yifehuan]

LGTM