[Kernel] Add Qwen3.8 NVFP4 skinny GEMM for SM120

[BBuf]

Motivation

Qwen3.8-27B NVFP4 decode on high-SM-count SM120 GPUs spends a significant amount of time in FlashInfer's general block-scaled GEMM. Its decode and DSpark verification shapes have only 1-16 rows, so the general kernel pays unnecessary tile padding and scheduling overhead.

Modifications

• Add an NVRTC-compiled SM120 NVFP4 skinny GEMM specialized for the Qwen3.8 gate/up, down, and lm_head shapes.
• Dispatch the kernel behind SGLANG_ENABLE_QWEN38_FP4_SKINNY_GEMM (off by default), with strict shape, dtype, stride, device, SM-count, and architecture guards.
• Keep the down projection on FlashInfer for M>1 by default. Its Stream-K accumulation order can change speculative acceptance decisions even when the output passes the normal numerical tolerance. Down M=1 remains enabled.
• Add correctness, CUDA Graph replay, safe-dispatch, and microbenchmark coverage.

Accuracy Tests

Tested on an NVIDIA RTX PRO 6000 Blackwell Server Edition:

TestNvfp4SkinnySm120.test_cuda_graph_replay                              PASS
TestNvfp4SkinnySm120.test_matches_flashinfer                              PASS
TestNvfp4SkinnySm120.test_safe_roles_keep_down_verification_on_flashinfer PASS

The FlashInfer comparison covers all three shapes at M=1, 9, and 16.

DSpark E2E validation used three fixed seeds. The accept lengths are unchanged after applying the safe down-projection routing:

Seed	Baseline	This PR
1234	3.14	3.14
42	2.86	2.86
7	2.90	2.90

Speed Tests and Profiling

RTX PRO 6000 kernel microbenchmarks against flashinfer.mm_fp4(backend="auto"):

Shape	M	FlashInfer (us)	This PR (us)	Speedup
gate/up	1	104.774	98.914	1.0592x
gate/up	9	104.624	99.877	1.0475x
gate/up	16	104.502	100.168	1.0433x
down	1	69.755	56.534	1.2339x
down	9	71.936	57.105	1.2597x
down	16	71.692	57.643	1.2437x
lm_head	1	540.978	522.787	1.0348x
lm_head	9	548.902	526.659	1.0422x
lm_head	16	552.271	530.363	1.0413x

Down M=9/16 numbers show the raw kernel result; those two paths are disabled by the safe default because of the acceptance sensitivity described above.

Qwen3.8-27B end-to-end output throughput:

• Normal decode: 67.24/67.52/67.87 -> 68.27/68.68/68.53 tok/s (1.41% geometric-mean improvement).
• DSpark: 135.13/114.77/143.35 -> 137.05/116.36/145.15 tok/s (1.35% geometric-mean improvement), with identical accept lengths.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations. (Not needed for this experimental, default-off path.)
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): ❌ Run #32962658503
Latest PR Test (Extra): ❌ Run #32962658497
Latest PR Test (AMD ROCm 7.2): ❌ Run #32962658549

[BBuf]

/tag-and-rerun-ci extra

[BBuf]

/rerun-failed-ci

[BBuf]

Closing this draft in favor of the stronger KDA-Pilot implementation from BBuf/KDA-Pilot#195. I will re-port that optimized SM120 NVFP4 GEMM into a dedicated python/sglang/kernels/kda_kernels package with explicit Humanize2 / Kernel Design Agents provenance, then revalidate DSpark end-to-end throughput and acceptance length on RTX PRO 6000 before opening the replacement PR.