[MoE][Ada] Add Qwen4-Exp FP8 Triton configs for NVIDIA L20

[positive666]

Motivation

Qwen3.8-Flash-Next-FP8 uses the Qwen4ExpForConditionalGeneration MoE architecture, but the TP4/EP4 FP8 path on NVIDIA L20 currently falls back to the default Triton MoE configuration because these exact keys are missing:

E=128,N=640,device_name=NVIDIA_L20,dtype=fp8_w8a8,block_shape=[128, 128].json
E=128,N=640,device_name=NVIDIA_L20,dtype=fp8_w8a8,block_shape=[128, 128]_down.json

The topology is TP4/EP4 with 128 local experts, hidden size 2560, MoE intermediate size 640, top-k 10, and FP8 block shape [128, 128]. All 48 transformer layers contain MoE blocks.

Modifications

Add independently tuned L20 configurations for the gate-up and down GEMMs at M buckets 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, and 1024 under the Triton 3.7.1 config directory.

This is a data-only change. The generic config loader on main already resolves these keys, so it has no code dependency. Qwen3.8-Flash-Next model integration has landed in #37500 (replacing #36497).

September 14 CPU-only update

Rebased onto 7465e42b7a, including the required CI baseline. Both JSON files are unchanged from the previously tuned submission; the base branch still has no matching L20 configurations.

A dependency-isolated CPU check of the current loader definitions verified unique/valid JSON keys, all 22 gate-up/down bucket records, filename resolution, loading and nearest-bucket selection for 15 batch sizes, matching gate-up/down BLOCK_SIZE_M, version fallback, and deterministic-mode bypass. No GPU work was performed during this update. The accuracy and timing results below were collected before this rebase and have not been revalidated against the current kernels.

Accuracy Tests (historical)

Compared the default and tuned kernels with fixed inputs at every configured M bucket. All 11 buckets were bitwise identical:

M=1..1024: max_abs=0, mean_abs=0, max_rel=0, allclose=true

Speed Tests and Profiling

Hardware and shape:

• 4 x NVIDIA L20, using physical GPU4-7 only
• TP4 / EP4, FP8 W8A8, Triton 3.7.1
• E=128 local experts, N=640, K=2560, top-k=10, block shape [128, 128]
• Separate gate-up/down search over the tuner's 432-candidate space for each M bucket

Routing methodology:

• Tuning trace: two 1024-token chunks captured from a real 2,081-token Qwen prefill, across all 48 MoE layers (96 trace files).
• Independent holdout: three different prefill requests (systems/code, Chinese distributed inference, and multilingual math), totaling 9,133 tokens. This produced eight 1024-token chunks plus M=150, 229, and 562 partial chunks.
• All four TP/EP ranks produced identical routing tensors. Both trace sets covered all 512 global experts.
• Holdout data was not used to select configurations.

Three-repeat interleaved default/tuned A/B on the independent holdout routing:

M	Default (us)	Tuned (us)	Speedup
1	69.700	53.234	30.93%
2	108.425	91.753	18.17%
4	183.519	159.456	15.09%
8	285.688	252.938	12.95%
16	451.962	398.041	13.55%
32	584.123	522.123	11.87%
64	697.600	631.866	10.40%
128	798.548	727.110	9.82%
256	880.764	806.966	9.15%
512	966.895	898.869	7.57%
1024	1088.934	1044.866	4.22%

• 11/11 buckets improved; no regression.
• Mean speedup across all buckets: 13.07%.
• Mean speedup across M=4..1024: 10.51%.
• Maximum tuned latency span across the three holdout repeats: 1.57%.
• Using the exact observed holdout chunk distribution (M=1024 x8, plus M=150, 229, and 562), aggregate per-layer gate-up + down time improved from 11,380.187 us to 10,819.297 us: 5.18% speedup (4.93% kernel-time reduction).

These are MoE kernel measurements. The trace requests were prefill-only (max_new_tokens=0); this PR does not claim end-to-end serving throughput because sparse QSA decode on SM89 still requires classic FA2.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests. (N/A: data-only config files; loader code is unchanged.)
• Update documentation according to Write documentations. (N/A: no user-facing behavior or API change.)
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ❌ Run #34864866608
Latest PR Test (Extra): ❌ Run #34864866514
Latest PR Test (AMD ROCm 10): ❌ Run #34864866545