[Perf] Add gfx950 DSV4.1 Flash EP4 a8w4 FMoE tuning

[kevin-mii]

Motivation

Upstream the DSV4.1 Flash EP4 a8w4 FMoE rows carried by sgl-project/sglang#39186. Without this table the exact gfx950 shape falls back to heuristic FlyDSL kernels. Removing the downstream CSV reduced performance in the operator comparison below, so it is restored downstream while this PR is reviewed.

Technical Details

• Add 12 unique tuning rows under aiter/configs/model_configs/, plus the corresponding untuned shape input.
• Shape: gfx950 / 256 CUs, model_dim=5120, inter_dim=2304, 96 local experts, topk=5, Silu, bf16 input/output, FP8 activations and FP4 weights with per-1x32 quantization, interleaved gate/up weights, no stage-1 routing weight.
• Token buckets: 1, 2, 4, 8, 32, 64, 128, 256, 512, 4096, 8192, 16384.
• Add op_tests/op_benchmarks/bench_dsv41_fmoe.py for a reproducible default-versus-tuned graph comparison. It excludes the candidate CSV from the baseline even when running from this branch and preserves the other model tables.
• All 12 rows survive normal AITER_CONFIG_FMOE_FILE merging; no matching shape keys exist in the current shipped model/default tables.

The rows and their us / error columns are imported from the original downstream tuning data; those columns are historical, not the measurements below. This PR is based on main at 0138f88b2.

Test Plan

For the measurements below, apply the stage-1 synchronization fix in #5561 to this branch, then run:

PYTHONPATH=. HIP_VISIBLE_DEVICES=0 TRITON_HIP_USE_ASYNC_COPY=0 \
AITER_BF16_FP8_MOE_BOUND=0 AITER_FLYDSL_FORCE_REDUCE=1 \
python op_tests/op_benchmarks/bench_dsv41_fmoe.py \
  --csv aiter/configs/model_configs/a8w4_tuned_fmoe_dsv41_flash_gfx950.csv \
  --tokens 1 32 128 512 4096

The benchmark uses seeded synthetic FP4 weights and bf16 activations. Timing is the median of five GPU-event measurements, each covering 100 graph replays after warmup. Both configurations use the same synchronization fix and inputs. The Triton environment flag is a local toolchain workaround.

Test Result

MI350X VF (gfx950), ROCm 7.2.4, PyTorch 2.11.0+rocm7.2:

Tokens	Default (us)	Tuned (us)	Speedup
1	48.37	37.44	1.29x
32	290.40	267.83	1.08x
128	349.43	320.22	1.09x
512	427.11	404.57	1.06x
4096	1411.56	1248.79	1.13x

A separate one-token smoke rerun measured 47.17 -> 37.61 us (1.25x). Black, Ruff, CSV uniqueness and normal config merge checks pass.

Numerical limitation: the benchmark checks finiteness and reports differences; it is not an independent accuracy oracle. Tuned versus default maximum absolute differences were 0.0273-0.0410 on the synthetic inputs. An initial strict comparison at 32 tokens (atol=0.02, rtol=0.05) rejected 13/163840 elements. The two paths differ in intermediate quantization/reduction, so this requires review rather than asserting equivalence.

Not executed: a fresh exhaustive tuning sweep, full-model accuracy/throughput, the other seven buckets, or other architectures. Keep this PR in draft until numerical tolerances, all buckets, and the paired synchronization fix are reviewed. These operator timings are not an end-to-end model speedup.

A second comparison on the installed SGLang-pinned AITER 4ad998328 also favored the CSV: M=1: 43.05 -> 37.17 us (1.16x), M=32: 291.22 -> 262.54 us (1.11x), M=128: 343.24 -> 316.97 us (1.08x), M=512: 419.93 -> 399.23 us (1.05x), M=4096: 1359.34 -> 1239.05 us (1.10x). This installation does not include the synchronization patch; the same installation was used for both sides. These are a8w4 measurements with AITER_BF16_FP8_MOE_BOUND=0, not the default small-batch bf16 route.

Submission Checklist

• Reviewed AITER conventions and recent config PRs ([Config] configs: GLM-5.3-Flash a8w8 blockscale fused-MoE configs for gfx942 #5500, [Config] [AMD][DSV4] [tune][gfx950] wo_b/wq_b a8w8 blockscale bpreshuffle configs #5279).
• Signed-off commit; additive shape keys and no new dependencies.
• Validate all imported rows against an independent numerical reference.
• Re-run full-model accuracy and benchmark the remaining buckets.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5562 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[jhinpan]

@kevin-mii Measurements for this CSV on MI355X, plus three tiers it is missing.

Setup. dsv41-amd-main @ e2e824dc58 (after 86ab3ad1bc dropped this CSV), AITER acf8fdf9, DeepSeek-V4.1-Flash TP4/EP4. The only arm difference is this CSV placed in aiter/configs/model_configs/; no Docker patches. Workload: B distinct real-text 4K prompts, 1024 output tokens, temperature 0, fresh servers in ABBA order, 4 scored bursts per arm, GSM8K 61-63/64 on every server.

What changes. Without the CSV every tier logs [fused_moe] using 2stage default, i.e. t32x128x256 at every M. With it, the tuned FlyDSL tiles are picked per tier.

Decode tok/s change (all requests decoding), 95% CI:

B	High-Throughput (no spec)	DSpark (Low-Latency cell, graph cap 256, per-step verify width from kevin-mii/sglang#16)
8	+0.5% [+0.3, +0.7]	+3.8% [+2.4, +5.2]
64	+4.5% [+4.4, +4.6]	+4.0% [+3.5, +4.4]
128		+2.5% [+1.8, +3.3]
256	+4.6% [+4.3, +4.9]	-0.2% [-0.5, +0.3], see below

So at batch 1-8 without speculation it is within noise, matching your 86ab3ad1bc measurement. At larger batches, and with DSpark even at B=8 (verify M = 6 x B = 48), it is worth about 4%.

Missing tiers. DSpark verify M at larger batches falls into tiers 1024/2048, which the CSV does not cover, so those still use the default. I tuned them with csrc/ck_gemm_moe_2stages_codegen/gemm_moe_tune.py on this pin (all candidates 0% error):

• 1024: t64 341 us vs t128 439 us vs t32 504 us
• 2048: t128 482 us vs t64 570 us vs t32 742 us

gfx950,256,16,5120,2304,96,5,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,32,0,153.4512,flydsl_moe1_afp8_wfp4_bf16_t32x128x256_w4_gui_fp8,0.0%,84.8623,flydsl_moe2_layout_afp8_wfp4_bf16_t32x128x256_reduce_nt_sbm32,0.0%,238.3135,0,0,0,23.76,14256.99
gfx950,256,1024,5120,2304,96,5,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,64,0,218.872,flydsl_moe1_afp8_wfp4_bf16_t64x256x256_w3_gui_fp8,0.0%,122.0679,flydsl_moe2_layout_afp8_wfp4_bf16_t64x256x128_atomic_nt_sbm64,0.0%,340.9399,0,0,0,1062.91,10010.9
gfx950,256,2048,5120,2304,96,5,ActivationType.Silu,torch.bfloat16,torch.float8_e4m3fn,torch.float4_e2m1fn_x2,QuantType.per_1x32,1,0,128,0,288.319,flydsl_moe1_afp8_wfp4_bf16_t128x256x256_w4_bnt0_gui_fp8,0.0%,194.1545,flydsl_moe2_layout_afp8_wfp4_bf16_t128x256x128_reduce_nt_sbm128,0.0%,482.4735,0,0,0,1502.21,7106.8

With these rows added, DSpark at B=256 (verify M = 768 -> tier 1024) decodes +3.6% [+3.5, +3.7] faster than with the current CSV (end to end +1.5%). B=8/64/128 are unchanged (their tiers already exist). Could you add these rows to this PR?

The tier-16 row's absolute timing comes from the tuner's synthetic routing and is not comparable to the existing rows. The choice was only validated through the end-to-end runs above.