[Config] [Perf] Add gfx950 DSV4.1 Flash TP4 fused-shared-expert a8w4 FMoE tuning

[kevin-mii]

Motivation

SGLang serves DeepSeek-V4.1-Flash with the shared expert fused into the routed MoE (--enforce-shared-experts-fusion, appended as the last expert). At TP4 the MoE therefore runs:

• 385 experts, top-7 for the target model.
• 129 experts, top-4 for the DSpark draft.

Both have per-rank inter_dim 640, A8W4. dsv41_fp8_tuned_fmoe.csv has rows for the unfused 384/6 and 128/3 shapes (#6125), but none for these, so fused_moe falls back to the heuristic FlyDSL pair at every token bucket.

Technical Details

• Add 8 rows to aiter/configs/model_configs/dsv41_fp8_tuned_fmoe.csv, plus the matching shapes in dsv41_fp8_untuned_fmoe.csv. No new file.
• Shape: gfx950 / 256 CUs, model_dim 5120, inter_dim 640, Silu, bf16 in/out, FP8 activations, FP4 weights, per-1x32 scales, gate/up interleaved, no stage-1 routing weight.
  • 385/7 at tokens 1, 32, 64, 128 and 256.
  • 129/4 at tokens 1, 2 and 32.
• All picks are from the bf16-output _gui stage 1 + non-layout stage 2 family, the same family the heuristic uses. No _fp8 stage 1, moe2_layout, or xcd4 kernels.
• Buckets where no candidate beat the current heuristic were left out: 385/7 at 2, 4, 8, 16, 512 and 1024, and 129/4 at 4, 8, 16 and 64. Prefill buckets (2048+ for 385/7, 128+ for 129/4) were not tuned.
• The 8 keys are new. They merge through the default model_configs path without collisions (3,080 merged rows, 8 unique new keys).

Test Plan

• Production fused_moe with the real expert weights of one DeepSeek-V4.1-Flash layer:
  • target layers.3, draft mtp.0, rank 0 of TP4.
  • inter 576 padded to 640 (intermediate_pad=64); the shared expert requantized to MXFP4 as the last expert.
  • GateMode.INTERLEAVE, swiglu_limit=10.
  • Routing comes from the real router on activations scaled by the layer's norm.
• Each token count is checked with an empty config (heuristic) and with only these rows. Timing uses CUDA graphs of 10 calls, median of 30 replays, median of 3 alternating rounds, and the whole comparison is run twice.
• Numerics: 10 runs per config on NaN-poisoned allocator memory; NaN and run-to-run determinism are counted.

Test Result

MI350X VF (gfx950), aiter 6bf550b61, µs per fused_moe call; heuristic → tuned (run 1 / run 2):

shape	tokens	heuristic	tuned	change
385/7	1	31.1 / 30.7	29.9 / 29.8	−3.8% / −2.9%
385/7	32	134.8 / 134.4	132.5 / 131.8	−1.7% / −2.0%
385/7	64	204.0 / 202.2	197.6 / 193.7	−3.1% / −4.2%
385/7	128	256.9 / 257.6	249.2 / 246.1	−3.0% / −4.4%
385/7	256	303.7 / 297.3	297.9 / 291.8	−1.9% / −1.9%
129/4	1	29.3 / 29.6	28.3 / 28.2	−3.7% / −4.7%
129/4	2	30.8 / 31.0	28.1 / 28.7	−8.7% / −7.5%
129/4	32	83.6 / 84.1	77.6 / 76.9	−7.1% / −8.5%

• NaN: none with either config.
• Determinism: the heuristic's output changed between repeated identical calls in 10 of 10 runs at every point. The tuned rows were deterministic, except 129/4 @ 32, whose atomic stage 2 is nondeterministic like the heuristic's.

End to end (on the older aiter that SGLang currently pins, e7d2453). The heuristic there is slower, and these rows were bitwise-equal to it.

• Setup: DeepSeek-V4.1-Flash TP4 + DSpark decode, 3 interleaved rounds per arm.
• Per-step decode throughput at 8 running requests: +3.3% (every round above every round without the rows). At 1-4 running requests the effect was within run-to-run noise. This used an earlier 12-row set that also had 385/7 @ 2 and 129/4 @ 4, 8 and 64; those rows are dropped here because they are not faster on current main.

Note on the fused-quant family (not in this PR). On these padded shapes (inter 576 → 640), some _fp8 stage-1 / moe2_layout / xcd4 candidates produced whole NaN rows at random:

• e.g. xcd4 stage 1 at 20-64 tokens, 15/15 runs.
• With intermediate_pad=0 they produced none, which points at the padded columns not being written.

The tuner did not catch this because its random data is not padded.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355	Run the full Triton test suite on MI35X, not only the tests the change affects
ci:multi-gpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6433 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.