[II] fix(ds4): keep indexer scoring in breakable graphs

[voipmonitor]

Behavior

DeepSeek V4 breakable CUDA graph capture always records learned sparse-indexer scoring. The eager short-context optimization still selects every candidate without scoring when the compressed context fits within the top-k set.

Technical reason

Graph capture uses short dummy attention metadata, but the captured graph can replay with a longer cached prefix. Encoding the short-context branch into the graph makes long-prefix replay select the first candidate indices instead of ranking the learned indexer scores.

The shortcut is therefore gated by torch.cuda.is_current_stream_capturing(). This is an exact backport of vLLM commit 292187dd8ca1b1bfa195f25b2886262527269999 from upstream PR vllm-project#52492.

Compatibility

• Eager execution retains the short-context shortcut.
• CUDA graph capture records the scoring path for both short and long replay contexts.
• The additional scoring work occurs during graph recording; the replay hot path is unchanged.
• No model weights, sampling behavior, KV-cache format, or public configuration interface changes.

Validation

• pytest -q --confcutdir=tests/models/deepseek_v4 tests/models/deepseek_v4/test_indexer_graph_capture.py: 2 passed.
• ruff check and ruff format --check: passed.
• TP2/DCP1/K5/full-graph source-overlay qualification used a 33,204-token, 14-tool request at concurrency 8. The unpatched runtime produced one severe multilingual/token-fragment corruption in 8 responses; the patched runtime produced none in 40 responses across five runs. This workload is stochastic, so the result supports but does not independently prove causality.
• The 6,543-token payload from upstream issue [Bug]: DeepSeek-V4-Flash think-until-cap / empty turn under concurrent DSpark + breakable CUDA graphs vllm-project/vllm#52448 completed with tool calls in all 16 patched C16 requests. The same payload did not reproduce its upstream cap-hit signature on the II/B12X baseline in 256 requests, so it is not treated as a local A/B discriminator.

References

• Upstream fix: [Bugfix][DSv4] Keep indexer scoring in breakable graphs vllm-project/vllm#52492
• Upstream reproducer: [Bug]: DeepSeek-V4-Flash think-until-cap / empty turn under concurrent DSpark + breakable CUDA graphs vllm-project/vllm#52448
• Short-context shortcut introduction: [DSv4 Perf] Skip topk and router when not needed, 3.4% E2E TTFT improvement for Decode case vllm-project/vllm#49486
• Breakable indexer graph boundary: [Perf] Narrow DeepSeek V4 eager CUDA graph region vllm-project/vllm#51430

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b6c303a6-20b5-4797-b084-543204d24b13

📥 Commits

Reviewing files that changed from the base of the PR and between 6dc2f51 and 8850b61.

📒 Files selected for processing (2)

• tests/models/deepseek_v4/test_indexer_graph_capture.py
• vllm/models/deepseek_v4/attention.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}