[Bugfix][ROCm] Fix DeepSeek V4 DSpark probabilistic startup - #51145
Merged
Merged
Conversation
Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
tuukkjs
requested review from
dllehr-amd,
hongxiayang,
tjtanaa and
zyongye
as code owners
August 5, 2026 09:37
Member
|
/ci run |
|
✅ Triggered Buildkite CI #82508 for commit |
Contributor
Author
|
@tjtanaa would you have time to review this one-line fix? It seems |
tjtanaa
approved these changes
Aug 11, 2026
zyp2014
pushed a commit
to zyp2014/vllm
that referenced
this pull request
Aug 21, 2026
…ject#51145) Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
DeepSeek V4 DSpark uses a full-vocabulary draft model, so draft token ids are already target token ids and no draft-to-target remapping is needed. The CUDA implementation exposes this by defining
draft_id_to_target_id = None, but the ROCm platform implementation was missing the same marker.The shared DSpark speculator reads
model.draft_id_to_target_idwhendraft_sample_method="probabilistic"to decide whether reduced-vocabulary draft logits need scattering into target-vocabulary columns. On ROCm, the missing attribute causes probabilistic DeepSeek V4 DSpark startup to fail with:This PR adds the full-vocabulary marker to the ROCm DeepSeek V4 DSpark class.
Duplicate-Work Check
No open PR seems to address this ROCm omission.
Test Plan
Validate on ROCm with DeepSeek V4 Flash DSpark using
draft_sample_method="probabilistic".Test Result
ROCm serving validation:
AI assitance was used for this PR.