Use supported FP8 attention output GEMM on SM120 instead of forcing it off

[ormandj]

Motivation

On DeepSeek-V4 SM120 the model override currently forces SGLANG_OPT_FP8_WO_A_GEMM off. #29927 is now merged; that override bypasses a supported path on capable builds. The global default is true, and the later DeepGEMM capability check already disables builds that cannot run the GEMM.

Modifications

• Gate the DeepSeek-V4 SM120 override on DEEPGEMM_SCALE_UE8M0.
• Leave capable builds at the global default while preserving an explicit false setting.
• Add unit coverage for capable and incapable builds.

Default behavior changes only for DeepSeek-V4 on capable SM120 builds.

Accuracy Tests

Rebased on main 2c05ed4e77. Author test logs and source bindings record the tested snapshots. The subsequent main refresh changes only unrelated ROCm/NPU files; each repaired patch is byte-identical across that refresh. Author CPU validation passed 91 tests and 29 subtests with one skip. The helper now lives in arg_groups/model_hook.py.

The following accuracy and speed results are retained author reports from the original source identified below, not new measurements of this rebase.

DeepSeek-V4-Flash, TP=2, 2x RTX PRO 6000, temperature 0, no MTP, and no DSpark. The source was current main ffd4705baa39 + #29927 41ee852a6eb6 + this PR 1dbf09f6e015, with no other SGLang PRs or runtime source overlays.

The paired 1,319-question GSM8K run scored 1,250 correct with the flag off and 1,246 correct with the default enabled. There were 14 candidate-only and 18 control-only successes; McNemar p = 0.5966 and the paired bootstrap 95% interval was -1.14 to +0.53 percentage points. All 2,638 responses stopped cleanly with zero invalid or failed responses.

Speed Tests and Profiling

The control set SGLANG_OPT_FP8_WO_A_GEMM=0; the candidate left the variable unset. The source, image, dependencies, arguments, and hardware were otherwise identical. Sustained decode used zero-token contexts, 30-second cells, and a maximum of 8,192 output tokens. Each value is the median of five runs.

Concurrency	Off (tok/s)	Default (tok/s)	Change
1	97.90	99.46	+1.60%
2	167.26	179.86	+7.53%
4	289.05	308.80	+6.83%
8	470.66	493.86	+4.93%
16	710.93	737.59	+3.75%
32	1004.43	1041.03	+3.64%

Exact cold prefill	Off (tok/s)	Default (tok/s)	Change
8K	7,911	8,409	+6.30%
64K	8,685	8,847	+1.87%
128K	8,047	8,160	+1.40%

The five-run coding median changed from 97.68 to 99.64 tok/s (+2.00%). KV-cache token capacity changed from 1,723,648 to 1,823,488 (+5.79%).

The focused unit test and pre-commit checks pass. In the candidate container, the runtime source HEAD/tree matched 62d35aaa5e82/d0961aec18dd, the environment variable was absent, the DeepGEMM capability reported true, and the effective default remained true.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations. (No user-facing API change.)
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Developed with AI assistance.

---

CI States

Latest PR Test (Base): ❌ Run #34058833439
Latest PR Test (Extra): ❌ Run #34058833395
Latest PR Test (AMD ROCm 7.2): ❌ Run #34058833636

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[ormandj]

@AliceChenyy @b8zhong @mmangkad — review requested for the SM120 DeepGEMM capability gate. This draft is pending #29927.