[Spec Decode] Enable adaptive verification on DSv4 + sm90

[LucasWilkinson]

Purpose

Enable adaptive verification for the DeepSeek V4 indexer on Hopper (SM90).

The native variable-length DeepGEMM paged-MQA path remains restricted to SM100. On SM90, this change instead uses the existing flattened decode path and builds the single-token rows from the device-side decode lengths. This is necessary because adaptive verification can trim each request to a different length while the CPU metadata contains only a uniform placeholder. The builder forces this device-length flattening for adaptive verification, advertises support for the device/CPU query-length mismatch, and enables CUDA graph capture for this path. When there is only one decode request, the CPU and device request boundaries necessarily agree, so that case reuses the cheaper uniform flattened metadata path.

This addresses the Hopper startup failure reported in #47808 (comment).

Duplicate-work check

I searched the open PRs for adaptive verification Hopper SM90, DeepseekV4IndexerBackend adaptive verification, varlen indexer Hopper, and SM90 DeepGEMM indexer; no open PR implements this fix. #51972 concerns a separate indexer bounds bug, while #47808 is the merged SM100 implementation where the SM90 failure was reported.

Test Plan

Targeted correctness and regression tests:

.venv/bin/python -m pytest tests/v1/attention/test_indexer_native_next_n.py tests/v1/spec_decode/test_adaptive_verification.py -v
.venv/bin/python -m pytest tests/v1/worker/test_gpu_batch_ordering.py -v
CUDA_VISIBLE_DEVICES=0 .venv/bin/python -m pytest tests/kernels/attention/test_deepgemm_attention.py -v
.venv/bin/pre-commit run --from-ref origin/main --to-ref HEAD

End-to-end validation used DeepSeek-V4-Flash-0731 on 4x H200 (SM90):

• TP2 / DP2 / expert parallel startup and serving with 7 DSpark speculative tokens, probabilistic drafting, and adaptive verification enabled.
• The repository GSM8K 5-shot evaluation with TP4 / expert parallel and the adaptive DSpark configuration from DeepSeek-V4-Flash-DSpark-confidence-TP4.yaml. The Blackwell-only --attention_config.indexer_kv_dtype=mxfp4 option was removed for Hopper; the rest of the speculative configuration was unchanged.
• Warmed throughput comparison against fixed verification on TP2 / DP2 / expert parallel, averaged over three runs.
• Matched fixed and adaptive torch-profiler traces at concurrency one.

Test Result

• Indexer and adaptive-verification tests: 15 passed.
• GPU batch-ordering tests: 10 passed.
• DeepGEMM attention kernel tests on H200: 5 passed.
• Pre-commit, including ruff and Python 3.10 mypy: passed.
• User-equivalent 4x H200 server: started and served successfully. The log confirms use_flattening=True supports_varlen=False.
• Hopper GSM8K: 94.996% accuracy, 0% invalid, 4.106 mean acceptance length (1,319 questions, 5-shot).

Warmed output throughput (tokens/s):

Concurrency	Fixed verification	Adaptive verification	Delta
1	256.5	240.2	-6.35%
16	1520.9	1608.9	+5.78%
64	2663.6	3032.5	+13.85%

The concurrency-one regression is reduced but remains a known limitation of this draft. Reusing uniform flattened metadata for a single decode request improved adaptive throughput from 236.9 to 240.2 tokens/s (+1.4%) and reduced mean ITL from 12.455 to 12.355 ms. This matches the metadata microbenchmark: device-ragged preparation took 115.08 us versus 24.14 us for uniform preparation, accounting for about 0.100 ms of the original 0.961 ms per-verification-step gap.

Matched torch-profiler traces show that fixed and adaptive verification both execute an 8-token target graph at concurrency one. Median target GPU execution was effectively identical (10.181 ms fixed versus 10.185 ms adaptive), but the median gap between target graph executions increased from 1.811 to 2.517 ms. The adaptive trace contains one blocking confidence-copy event synchronization per step in AdaptiveVerificationManager.record_confidences; its approximately 0.706 ms net critical-path effect, together with the 0.100 ms metadata cost, explains most of the original step-latency gap. CUDA graph capture also increased in the TP2 / DP2 comparison (121 s / 4.71 GiB adaptive versus 52 s / 2.61 GiB fixed). The existing SM100 path is unchanged and was not rebenchmarked here.

AI assistance

AI assistance from OpenAI Codex was used to investigate, implement, test, benchmark, and draft this PR. Before this draft is marked ready, the human submitter will review every changed line and must be able to defend the change end-to-end.

---

• The purpose and related report are documented.
• Exact test commands are provided.
• Correctness, model-evaluation, and performance results are provided.
• No documentation update is required for this backend-only change.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[LucasWilkinson]

/ci run

[github-actions]

✅ Triggered Buildkite CI #84477 for commit b505462ef2fa.

[mergify]

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

[LucasWilkinson]

/ci run

[github-actions]

✅ Triggered Buildkite CI #84745 for commit 8b52cd5d9677.