Skip to content

[Bugfix][ROCm] Fix DeepSeek V4 DSpark probabilistic startup - #51145

Merged
tjtanaa merged 2 commits into
vllm-project:mainfrom
tuukkjs:fix/dspark-prob-mi355x
Aug 11, 2026
Merged

tjtanaa merged 2 commits into
vllm-project:mainfrom
tuukkjs:fix/dspark-prob-mi355x

Conversation

@tuukkjs

@tuukkjs tuukkjs commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Purpose

DeepSeek V4 DSpark uses a full-vocabulary draft model, so draft token ids are already target token ids and no draft-to-target remapping is needed. The CUDA implementation exposes this by defining draft_id_to_target_id = None, but the ROCm platform implementation was missing the same marker.

The shared DSpark speculator reads model.draft_id_to_target_id when draft_sample_method="probabilistic" to decide whether reduced-vocabulary draft logits need scattering into target-vocabulary columns. On ROCm, the missing attribute causes probabilistic DeepSeek V4 DSpark startup to fail with:

AttributeError: 'DSparkDeepseekV4ForCausalLM' object has no attribute 'draft_id_to_target_id'

This PR adds the full-vocabulary marker to the ROCm DeepSeek V4 DSpark class.

Duplicate-Work Check

No open PR seems to address this ROCm omission.

Test Plan

Validate on ROCm with DeepSeek V4 Flash DSpark using draft_sample_method="probabilistic".

Test Result

ROCm serving validation:

Model: deepseek-ai/DeepSeek-V4-Flash-0731
Speculative config: {"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic"}
Result: server started successfully and completed a random 8192-input / 512-output benchmark.
Acceptance rate: 43.19%
Acceptance length: 3.16
Per-position acceptance: 73.01%, 57.06%, 37.42%, 27.61%, 20.86%

AI assitance was used for this PR.

Signed-off-by: Tuukka Sarvi <tuukka.sarvi@amd.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added deepseek Related to DeepSeek models rocm Related to AMD ROCm labels Aug 5, 2026
@mergify mergify Bot added the bug Something isn't working label Aug 5, 2026
@github-project-automation github-project-automation Bot moved this to Todo in AMD Aug 5, 2026
@AndreasKaratzas

Copy link
Copy Markdown
Member

/ci run

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #82508 for commit 4166b8d698bc.

@tuukkjs

tuukkjs commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

@tjtanaa would you have time to review this one-line fix? It seems draft_sample_method="probabilistic" is not working atm on ROCm.

@tjtanaa
tjtanaa merged commit 12bea3e into vllm-project:main Aug 11, 2026
60 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in AMD Aug 11, 2026
zyp2014 pushed a commit to zyp2014/vllm that referenced this pull request Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working deepseek Related to DeepSeek models rocm Related to AMD ROCm

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants