[Kernel][SM70] Fuse exact-shape GDN decode projections

[yangzhuxinyzx]

Purpose

Add an opt-in SM70 GDN decode fusion while preserving the existing fallback paths. The measured Qwen3.8 NVFP4 workload is performance/quality evidence only: runtime admission does not inspect a model name, checkpoint path, architecture identity, or repository-specific identity field.

Source audit and fixes

• The submitted QPN8 QKV/Z plus FP16 b/a kernel arithmetic matches the archived accepted operator and endpoint evidence.
• The original branch exposed the b/a split independently even though that configuration diverged at token 202. The audited source now requires the accepted one-pass RMSNorm pair; an unpaired enable request fails clearly instead of silently entering the rejected route.
• Both controls are generic and default-off: VLLM_SM70_GDN_QPN8_BA_SPLIT=1 and VLLM_SM70_GDN_RMSNORM_ONEPASS=1.
• Admission is contract-based: SM70 capability, non-speculative cache stride, TP/layout dimensions, CUDA FP16 activations, exact QPN8 code/scale and b/a weight shapes, contiguous same-device tensors, no projection biases, a valid dense workspace, and both source-built extension operators.
• Weight admission is rechecked after quantized weights load and on dispatch. Missing source-built operators fail at setup; incompatible weights/tensors keep the existing path.
• Direct C++ entry points now validate exact shapes, dtypes, contiguity, device agreement, and the large-M workspace contract.
• The DFlash2 fused output-norm route keeps priority. Unsupported shapes, disabled flags, projection dumps, and prefill/large-M cases retain the established implementations.

Frozen endpoint evidence

Four V100-SXM2-32GB GPUs, TP4, input 1024, output cap 256, official random sampling, E4M3 KV, Flash-V100, full CUDA graph, and no MTP. Pure decode excludes TTFT/prefill; each number is the stable median of three sequential requests.

Route	Pure TPOT	Steady decode	Output gate
Matched control	12.249652 ms	81.634970 tok/s	frozen 256-token stream
Accepted paired fusion	11.921648 ms	83.881019 tok/s	3/3 exact

This is +2.751% steady decode and -2.678% TPOT. All three candidate requests reproduce SHA256 8b37337f4c393711cb8550db6bae909b1e85de8df1cf5ba8c60d8c000749c0a2.

The latest-main default-wrapper recheck reports 81.554583 tok/s / 12.261727 ms for control and 83.633383 tok/s / 11.956949 ms for candidate (+2.549%). All four TP ranks report the C++ route and all three 256-token candidate streams exactly match control SHA256 ca77db3b032a1600a8567adea706108c1bd8c5472b3ace2318c41ab17c66c1f9.

The 48-layer RMSNorm operator is bitwise exact and saves 55.946 us/token. QKV is exact; b/a relative L2 is 6.368e-8 with maximum absolute error 2.384e-7.

Rejected experiments remain excluded: b/a split alone diverged at token 202; the Q/K RMSNorm + MRoPE + KV-write route diverged at token 1 and stopped after 208 tokens.

Validation

• Replayed onto main a6f4b8b70 with both commits DCO-signed.
• Current CUDA source compiled successfully as the standalone source-built operator with TORCH_CUDA_ARCH_LIST=7.0.
• tests/model_executor/test_qwen3_5_quantization.py: 13 passed on the final source, including paired-flag, missing-extension, and loaded-weight layout contracts.
• Changed-file ruff, format, typos, clang-format, markdownlint, mypy, SPDX, forbidden-import, configuration, and CUDA-API hooks: all passed.
• git diff --check: passed.
• No additional full-model E2E was run during audit; the existing matched GPU artifacts were verified and the audit changes only harden admission/failure contracts without changing accepted kernel arithmetic.

Evidence is recorded in docs/design/sm70_qwen38_nvfp4_decode.md and docs/design/sm70_v100_migration_control.md; raw local artifacts remain intentionally untracked.