[mqhc2020]

Motivation

Qwen3.5-397B-A17B-MXFP4 on gfx1250 had no grouped MoE tuned config, so every
MoE call fell back to default tiles. MoE is ~30% (cc4) to ~60% (cc64) of decode
time at TP1.

Changes

Add aiter/configs/model_configs/qwen3_5_397b_tuned_grouped_fmoe.csv, auto-merged
with tuned_grouped_fmoe.csv. 36 rows for the FlyDSL grouped a8w4 TDM kernel
(model_dim 4096, 512 experts, topk 10, fp8 act, MXFP4 weight, 192 CU):

• TP1: inter_dim 1024, TP2: inter_dim 512, TP4: inter_dim 256
• Token buckets per TP: 1-256, 1024, 4096, 8192
  No code changes.

Test Plan

• Kernel sweep on one gfx1250 GPU per shape: tile_m/n/k, num_buffers, prefetch,
  then stage2 tiles. Each config checked against default output
  (rel_l2 < 0.005) and torch reference. K tiles larger than inter_dim are
  excluded (they read out of bounds).
• E2E: sglang TP1/TP2/TP4, fp8 KV, 8k in / 1k out, concurrency 4-64,
  default tiles vs tuned on the same build.
• Accuracy: lm_eval GSM8K 5-shot, 250 questions.

Results

Geomean over cc 4-64:

TP	MoE kernel	Throughput	TPOT	GSM8K (base -> tuned)
TP1	1.27x	+8.7%	-7.9%	0.976 -> 0.972
TP2	1.24x	+6.0%	-4.7%	0.968 -> 0.968
TP4	1.30x	+3.6%	-2.8%	0.972 -> 0.980

Tuned total throughput (tok/s) / median TPOT (ms), change vs default tiles:

cc	TP1	TP2	TP4
4	2280 (+9.1%) / 14.88 (-8.5%)	1960 (+6.6%) / 17.40 (-4.2%)	1865 (+5.4%) / 18.33 (-2.6%)
8	3533 (+10.4%) / 19.26 (-8.9%)	3341 (+7.7%) / 20.50 (-5.1%)	2937 (+3.1%) / 23.45 (-2.0%)
16	5036 (+8.7%) / 27.31 (-8.1%)	5171 (+5.2%) / 26.60 (-4.7%)	4715 (+3.4%) / 29.26 (-3.3%)
32	6639 (+8.1%) / 41.69 (-7.5%)	7042 (+5.7%) / 39.26 (-5.0%)	7217 (+3.1%) / 38.22 (-2.9%)
64	8360 (+7.1%) / 67.36 (-6.6%)	9552 (+4.5%) / 58.73 (-4.3%)	9302 (+3.1%) / 60.55 (-3.0%)

GSM8K differences are within +/- 0.010. No GPU faults in any run.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6090 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.