[Config] [Perf] Add DeepSeek V4.1 Flash TP4 gfx950 MoE tuning

[Fangzhou-Ai]

Summary

Add 16 gfx950 tuning rows for the DeepSeek V4.1 Flash TP4 backbone and draft MoE shapes. These shapes currently have no exact entries in the shipped tuning tables and use heuristic FlyDSL configurations.

Configuration

• H5120; TP-local logical intermediate 576, padded to 640.
• Backbone: 384 experts, top6; draft: 128 experts, top3; expert parallelism disabled.
• Token buckets 8/16/32/64/128/256/512/1024 for each geometry. Actual tested decode rows were backbone M6/12/24/48/96/192/384/768 and draft M5/10/20/40/80/160/320/640.
• BF16 output, FP8 activations, MXFP4 weights, per-1x32 scaling, interleaved gate/up, SiLU and no stage-one routing-weight multiplication.
• Existing unfused stage-one N128 kernels with M16/M32 tiles; stage-two N128/N256. Exact names and ksplit=-1 are in the CSV.

The model-specific table contains only the 16 new keys, with explicit gfx950. Existing rows are unchanged. Kernel timing/error columns are blank: the measurements below include the complete local shared/routed MoE and TP collective, so they are not valid stage-one/stage-two kernel timings.

Performance on the measured runtime

MI355X, TP4, Torch 2.12.0+rocm10.0.0, HIP 7.15.26333, Triton 3.8.0, AITER 0.1.21.post1 with the merged sorter fix from #5295. AITER_BF16_FP8_MOE_BOUND=0, fusion and routed-first scheduling disabled. This explicitly measures the A8W4 path; the CSV does not change activation-dtype selection policy.

Seeded synthetic weights/inputs, warm repeated routes, native shared-first MoE including gating, quantization, shared expert, routed expert and BF16 TP4 RCCL. Each of 31 alternating paired measurements takes the maximum rank latency over 16 graph replays. The other four GPUs were idle during confirmation.

Concurrency	Backbone baseline → tuned (µs)	Median paired reduction	Draft baseline → tuned (µs)	Median paired reduction
1	105.44 → 100.99	4.28%	77.21 → 75.61	2.22%
2	148.98 → 136.65	8.30%	89.36 → 86.54	3.20%
4	175.62 → 168.02	4.43%	116.18 → 113.09	2.56%
8	253.40 → 246.09	2.89%	145.83 → 140.55	3.50%
16	342.17 → 332.31	2.95%	176.08 → 170.67	3.11%
32	429.24 → 417.17	2.82%	195.88 → 193.00	1.42%
64	524.29 → 512.66	2.20%	265.47 → 258.08	2.77%
128	584.40 → 577.08	1.24%	316.79 → 309.12	2.42%

The percentage is the median paired gain, so it need not equal the ratio of the two median times. All 16 target shapes passed the predeclared independent native-CSV gate: ≥1% median paired improvement and ≥25/31 faster pairs. Observed wins were 30/31 or 31/31. Earlier isolated selection also used an identical-kernel control to reject capture/timing bias.

Validation and limits

• The original native CSV lookup resolved all 16 expected kernel pairs, without metadata overrides. Each geometry also passed M31/32/33/255/256/257 boundary checks.
• Eager and changed-input graph checks covered original, changed-input, concentrated-routing and restored-input cases. Fixed per-rank atol=rtol=.02, explicit FP64 local error budgets, propagated FP32 TP-sum budgets, input preservation and finite timed outputs passed. Strict FP32/BF16 aggregate parity diagnostics include mismatches and are retained separately; this is not a claim of bitwise or independent full-model accuracy.
• The benchmark retained one capture stream and diagnostic graphs/output buffers until synchronized cleanup after reproducing a graph-lifetime fault. No kernel arithmetic or scheduling change is included here.
• N64 stage-one variants were excluded because the investigated padded path left part of the intermediate unwritten.
• Full-model accuracy, every shape inside each lookup bucket, and GPU execution/serving throughput on this branch's current upstream runtime have not been validated. A short serving comparison on the measured private runtime is reported below.

Measured source identities include moe_kernels.py SHA256 c6898ef73fa21fd519593592c511b29ae41443e3e753ce6165bc866271c2bd73 and mixed_moe_gemm_2stage_common.py SHA256 bba3e08a6ac16c2106a8bde9b06a98aea6fb0c7764117b2630a3a091265e1853. Upstream base b0ced008d89f3fe8ad9fa9388d78b615fd53cb0b has newer FlyDSL/LDS and stage-two addressing code. All 10 selected kernel names still register with identical parameters, but that CPU check does not establish equivalent performance or correctness on the newer implementation.

Short serving follow-up (measured private runtime)

A matched TP4 controlled-decode check used 350,000 input tokens and 512 output tokens per request, two warmup batches and five measured batches at each concurrency, with synthetic speculative acceptance 3.51. The comparison uses the saved untuned baseline from a separate server lifetime. The tuned MoE CSV was selected; dense GEMM configuration, shared-first scheduling and unfused arithmetic were unchanged.

Concurrency	Output tokens/s/chip, baseline → tuned	Throughput change	Mean server TPOT change
1	63.457 → 62.826	-0.99%	+1.36%
4	180.454 → 180.968	+0.28%	-0.74%
16	349.840 → 349.711	-0.04%	-0.12%
32	428.890 → 424.609	-1.00%	-2.00%

These short measurements show no clear overall serving-throughput gain, despite the isolated MoE gains above. All 20 measured batches completed with matching prompts, zero preemptions and no harness validation issues. TPOT is the server's mean time per output token, not p90 ITL. Synthetic acceptance and request completion do not establish model quality. This does not replace the still-pending current-upstream GPU validation; the PR remains a draft.

Command: sudo -n /home/amd/vllm/.venv/bin/python dsv41_rocm_bench/moe_tuned_serving_20260917/run_short_verification.py --execute. Both client/analyzer completed successfully and owned server cleanup completed without forced termination. Detailed local artifacts: dsv41_rocm_bench/moe_tuned_serving_20260917/short_controlled_decode/{comparison.json,SHORT_COMPARISON.md}.

Commands run

On the measured runtime, the artifact-local run_native_confirmation_v5.py harness ran each TP4 lane with --timing, all eight actual shapes and six boundaries; both controllers exited zero and all four rank reports completed. qualify_native_csv.py checked all 16 rows against disjoint tuning/native reports and source hashes. A separate native metadata lookup checked the final combined CSV. These benchmark artifacts are outside this CSV-only patch.

On this upstream checkout:

git diff --cached --check
bash .githooks/install
.git/hooks/pre-commit
/home/amd/vllm/.venv/bin/python op_tests/tuning_tests/test_csv_validation.py TestCSVValidation.test_flydsl_stage2_sort_block_matches_fmoe_config

The CSV layout test passed. CPU checks also confirmed 16 unique new keys, no collisions with shipped tables, unchanged qualified row contents and identical registry parameters for all 10 selected names. No GPU was launched for upstream preparation.

Related work

Open PR #5562 targets DSV4.1 EP4 with H5120/I2304, 96 local experts and top5. This table targets TP4 with H5120/I640, E384/top6 and E128/top3, so its lookup keys are distinct. Open-PR searches for DSV4.1, 640 fmoe, and fmoe CSV found no proposal for these exact keys. #5315 addresses explicit architecture fields across existing tables; this new table already supplies gfx950.

AI assistance was used to prepare the tuning harness, analyze results, and draft this CSV change and description.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5609 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.