[Bugfix] Guard DeepSeek V4 MRV1 piecewise CUDA graphs - #51768
Merged
Merged
Conversation
WoosukKwon
force-pushed
the
agent/fix-dsv4-indexer-cudagraph-metadata
branch
from
August 11, 2026 06:29
1410285 to
7349476
Compare
WoosukKwon
marked this pull request as ready for review
August 11, 2026 06:32
WoosukKwon
requested review from
ProExpertProg,
houseroad,
mgoin,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256 and
youkaichao
as code owners
August 11, 2026 06:32
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
WoosukKwon
force-pushed
the
agent/fix-dsv4-indexer-cudagraph-metadata
branch
from
August 11, 2026 06:37
7349476 to
9a35f17
Compare
WoosukKwon
marked this pull request as draft
August 11, 2026 06:38
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
WoosukKwon
force-pushed
the
agent/fix-dsv4-indexer-cudagraph-metadata
branch
from
August 11, 2026 06:44
9a35f17 to
28cfd2e
Compare
WoosukKwon
marked this pull request as ready for review
August 11, 2026 07:19
Collaborator
Author
|
/ci run |
|
✅ Triggered Buildkite CI #83314 for commit |
4 tasks
This was referenced Aug 13, 2026
Open
1 task
njhill
added a commit
to njhill/vllm
that referenced
this pull request
Aug 14, 2026
vllm-project#51430 narrowed the DeepSeek V4 eager cudagraph region, which corrupts MRV1 output, and vllm-project#51768 responded by defaulting the model to MRV2 and rejecting MRV1 + PIECEWISE. That default costs ROCm, where MRV1 is still the faster runner for this model. Choose the region from the runner instead: MRV1 wraps the whole attention body in `_prepare_and_attn_eager`, restoring the pre-vllm-project#51430 region it needs, while MRV2 keeps the narrow region and its shorter TTFT. The nested `_sparse_indexer_and_attn` break runs inline, since `add_eager` clears `_capturing` first. MRV1 + PIECEWISE is then correct on every platform, so drop the rejection. CUDA still defaults to MRV2; ROCm defaults to MRV1 until MRV2 is competitive there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Nick Hill <nickhill123@gmail.com>
4 tasks
Fangzhou-Ai
added a commit
to Fangzhou-Ai/vllm
that referenced
this pull request
Aug 16, 2026
vllm-project#51430/vllm-project#51768 moved DSV4 to MRV2 and a narrow eager region. That is a large decode TPOT regression on ROCm. Default ROCm back to MRV1, wrap the full attention body in the eager break for MRV1 only, and drop the MRV1+PIECEWISE rejection. CUDA keeps MRV2 and the narrow region. Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Cursor Grok 4.6 <cursoragent@cursor.com> Signed-off-by: fai <fangzhouai@gmail.com>
shen-shanshan
pushed a commit
to shen-shanshan/vllm
that referenced
this pull request
Aug 17, 2026
…roject#51768)" Restore Model Runner V1 after the prerequisite eager-region revert makes piecewise graph execution correct again and recovers ROCm decode throughput. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>
zyp2014
pushed a commit
to zyp2014/vllm
that referenced
this pull request
Aug 21, 2026
…1768) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: OpenAI Codex <codex@openai.com>
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Default
DeepseekV4ForCausalLMto Model Runner V2 and reject only the known-broken configuration:PIECEWISEorFULL_AND_PIECEWISECUDA graphsThe validation runs after CUDA-graph mode resolution. MRV1 therefore remains available with eager/
NONE,FULL, orFULL_DECODE_ONLY; other model architectures are unaffected.Why
#51430 exposed a correctness problem in the legacy V1 model runner's breakable PIECEWISE CUDA-graph path. The same attention implementation retains normal GSM8K accuracy and MTP acceptance with Model Runner V2.
Failing this specific combination avoids silent output corruption while preserving MRV1 for configurations that do not exercise the affected path.
The error directs users to either select MRV2 or disable PIECEWISE CUDA graphs.
Relationship to #51750
This is an alternative to #51750, not a duplicate. #51750 exactly reverts #51430 and restores the wider MRV1 eager region. This PR leaves the attention implementation unchanged, defaults DeepSeek V4 to the already-correct MRV2 path, and fails closed if a user explicitly reaches the broken MRV1 PIECEWISE path.
The required duplicate searches found no other open PR adding this configuration guard.
Model validation
GPU validation ran through Slurm on a GB200 node with TP=2, FP4 indexer cache, MTP with two speculative tokens, and CUDA graphs enabled.
635dd6aae635dd6aaeTests
The tests cover both rejected PIECEWISE modes and the allowed MRV2, eager/NONE, FULL, FULL_DECODE_ONLY, and non-DeepSeek-V4 cases.
AI assistance
AI assistance was used to investigate the runner-specific regression, implement this change, run validation, and prepare this PR.
Before marking ready for review, the human submitter must: