[Model][NVIDIA] Route DSA models to the SM100 implementation

[zhou9402]

What this does

Point the DSA architectures at the SM100-optimized implementation in
vllm/models/deepseek_v32/, which nothing on main currently references.

• GlmMoeDsaForCausalLM now resolves to vllm.models.deepseek_v32 instead of the
  generic deepseek_v2, and DeepseekV32MTPModel is registered so speculative
  decoding uses the matching MTP module.
• deepseek_v32/__init__.py gains a platform gate: SM100 gets the optimized
  module, everything else (ROCm, XPU, pre-SM100 CUDA, CPU) falls back to the
  generic deepseek_v2. Previously it imported the SM100 module on any CUDA
  device, which is what broke the H100 CI job on [Perf][GLM-5.2] Blackwell decode optimizations #48597.
• DeepSeek V3.2 is unaffected: DeepseekV32ForCausalLM still maps to
  deepseek_v2, and the MTP rewrite is gated on model_type == "glm_moe_dsa".

This is the enabler for the rest of the series — until it lands, the other PRs in
the split are runtime no-ops.

Part of the #48597 re-split — see that PR for the tracker and merge order.

Benchmark

8×B300 TP8, nvidia/GLM-5.2-NVFP4, MTP=5, VLLM_USE_V2_MODEL_RUNNER=1, fp8 KV
cache, 8192 input / 1024 output tokens, concurrency 1, 64 prompts.

VLLM_USE_V2_MODEL_RUNNER=1 vllm serve $MODEL --served-model-name glm-5.2 \
  -tp 8 --port 8300 \
  --kv-cache-dtype fp8_e4m3 --max-model-len 16384 \
  --max-num-seqs 256 --max-num-batched-tokens 16384 \
  --no-enable-prefix-caching --gpu-memory-utilization 0.85 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --kernel-config '{"ir_op_priority":{"rms_norm":["vllm_c","native"],"fused_add_rms_norm":["vllm_c","native"]},"enable_flashinfer_autotune":false}'

Performance vs main

main @ 48a077e4cf versus the same commit with all six branches of the split
applied, each built and measured back-to-back on one node, 32 prompts at
concurrency 1, --kv-cache-dtype fp8_e4m3 throughout.

weights	build	output tok/s	total tok/s	median TPOT	GSM8K
NVFP4	main	447.4	4026.5	1.93 ms	0.940
NVFP4	full series	541.9	4876.7	1.56 ms	0.948
FP8	main	365.8	3291.8	2.39 ms	0.956
FP8	full series	403.1	3627.7	2.15 ms	0.946

That is +21.1% output throughput on NVFP4 (TPOT -19.2%) and +10.2% on FP8
(TPOT -10.0%). The GSM8K spread is roughly one standard error at 500 questions
(SE ~= 0.01), so it shows no accuracy regression in either direction rather than a
real difference.

AI assistance (Claude) was used for the split and the benchmarks; every changed line has been reviewed.

[zhou9402]

Part of the #48597 re-split. See that PR for the tracker: merge order, measured numbers, and what was dropped.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[yewentao256]

Thanks for the work!

Are you sure non-torch compiled path is faster than torch compiled path for GLM 5.2?

Please attach with full e2e benchmark command line and metrics. From my understanding there is still gap between this two (old path faster)

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @zhou9402.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[zhou9402]

541.9

added in the PR description, please have a review again.

[yewentao256]

Could you share with full benchmark command line and full output logs? Some guys faked the benchmark results before and it caused a lot of trouble. So now we are conservative about AI generated table without original data.

Please also run three times for benchmark and we can avoid fluctuations.

Additionally, could you also benchmark fp8 case? Using "zai-org/GLM-5.2-FP8"

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @zhou9402.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[WoosukKwon]

@zhou9402: GitHub auto-closed this PR when its head was synchronized to current main as part of resolving the old conflicts. A closed, no-diff cross-fork PR immediately loses its maintainer-edit grant, so an upstream maintainer cannot add the prepared commits or reopen it through the API.

The validated update remains preserved as commit a16fc42ceea3e8472ae785dc5e5d07a8fef79161 on WoosukKwon/vllm:codex/pr-49790-publish. The contributor branch is currently at b05ae5dc008850a620dec6de66635dec2b5913fd (the matching vllm-project/vllm:main commit), and the prepared commit has that main commit as an ancestor. The PR author can therefore recover the PR with a normal fast-forward push and reopen it:

git fetch https://github.com/WoosukKwon/vllm.git codex/pr-49790-publish
git push origin FETCH_HEAD:agent/sm100-sparse-model-routing
gh pr reopen 49790 --repo vllm-project/vllm

The prepared tree is rebased onto current vllm-project/vllm:main (b05ae5dc008850a620dec6de66635dec2b5913fd) and adds:

• MRV2 as the default runner for GlmMoeDsaForCausalLM.
• Automatic breakable CUDA graphs on every platform for GLM-5.2 and its DeepseekV32MTPModel draft model.
• CompilationMode.NONE when that breakable path is selected, with an explicit opt-out through VLLM_USE_BREAKABLE_CUDAGRAPH=0.
• Matrix coverage for both nvidia/GLM-5.2-NVFP4 and zai-org/GLM-5.2-FP8, with and without MTP, plus direct platform-independent main/draft architecture checks.

Validation completed:

env -u VLLM_USE_BREAKABLE_CUDAGRAPH -u VLLM_USE_V2_MODEL_RUNNER \
  .venv/bin/python -m pytest tests/test_config.py \
  tests/models/test_deepseek_v32_routing.py \
  tests/models/test_deepseek_v32_kv_cache_form.py -q
213 passed, 14 warnings in 114.06s

pre-commit run --files <all 10 changed files>
All hooks passed (including ruff, formatting, mypy, SPDX, and config-default checks).

I also ran the requested full GSM8K evaluation on four NVIDIA GB200 GPUs using nvidia/GLM-5.2-NVFP4, TP=4, MTP=3, 5-shot prompts, temperature 0, seed 42, max output 256, and concurrency 100. Startup logs confirmed GlmMoeDsaForCausalLM, DeepseekV32MTPModel, MRV2, automatic breakable graphs, CompilationMode.NONE, and successful FULL + PIECEWISE graph capture. The local stack selected the MARLIN NVFP4 fallback rather than a native FP4 MoE kernel.

GSM8K questions:       1,319
Exact-match accuracy:  94.768% (1,250/1,319)
Invalid responses:     0
Request errors:        0
256-token cap hits:    3
Evaluation latency:    44.235 s
Questions/s:           29.818
Output tokens/s:       2,937.604
MTP mean accept length: 3.373
Draft-token acceptance: 79.104%

The repository's current standalone GSM8K CLI omits the now-required model field when the Rust frontend is active, so the run used its existing prompt builder and scorer through a small external request wrapper that supplied model=nvidia/GLM-5.2-NVFP4; no evaluation files were added to the PR.

I additionally measured batch-size-1 output throughput on the same four GB200 GPUs, following glm_bench.sh with SPEED-Bench throughput_16k/low_entropy: 8,192 input tokens, 1,024 output tokens, concurrency 1, temperature 0, one warmup request, and three measured requests. Both arms used TP=4, FP8 KV cache, a 16,384-token model limit, a 16,384-token batching limit, prefix caching disabled, and otherwise identical server settings. The MTP arm used three speculative tokens.

The environment uses FlashInfer 0.6.17 with matching flashinfer-cubin==0.6.17 and flashinfer-jit-cache==0.6.17+cu130. Startup logs confirmed the FLASHINFER_TRTLLM NVFP4 MoE backend in both arms. An earlier measurement that selected the slower VLLM_CUTLASS fallback because the optional FlashInfer packages were absent has been discarded.

                           No MTP       MTP=3
Successful requests           3             3
Output tokens              3,072         3,072
Output throughput         130.17         304.15 tok/s
Decode TPS (1 / TPOT)     135.09         338.05 tok/s
Mean TPOT                   7.40           2.96 ms
Mean TTFT                 293.90         340.44 ms
Mean E2E latency        7,866.43       3,366.65 ms
MTP draft acceptance          -           78.90%
MTP mean accept length         -            3.37

The benchmark's output-throughput metric includes TTFT. MTP=3 delivered a 2.337x speedup (+133.7%) by that metric and a 2.502x decode-only speedup (+150.2%) from mean TPOT. Startup logs also confirmed MRV2, automatic breakable FULL + PIECEWISE CUDA graphs, CompilationMode.NONE, and DeepseekV32MTPModel for the MTP arm.

AI assistance was used to prepare the rebase/default-path changes and evaluation. The human submitter must review every changed line and the evidence above before merge.