Skip to content

[0.13.0][Bugfix] Fix setting of speculative_config.enforce_eager for dsv32 #5958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yiz-liu merged 1 commit into from
Jan 21, 2026
Merged

[0.13.0][Bugfix] Fix setting of speculative_config.enforce_eager for dsv32 #5958

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions vllm_ascend/platform.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,18 @@ def check_and_update_config(cls, vllm_config: VllmConfig) -> None:
"needs to be equal if use pcp or dcp > 1 in P/D disaggregate and kv pool scenario."
)

# NOTE: vllm sets `speculative_config.enforce_eager` as True if using
# deepseek_v32 with mtp. Since we support graph mode, we simply ignore
# it here. However, this fix will also implicitly ignore user setting of
# `speculative_config.enforce_eager`, we need to take care and remove it
# once vllm supports this feature.
speculative_config = vllm_config.speculative_config
if model_config and speculative_config and \
hasattr(model_config.hf_text_config, "model_type") and \
model_config.hf_text_config.model_type == "deepseek_v32" and \
speculative_config.enforce_eager:
speculative_config.enforce_eager = False
Comment on lines +368 to +372
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The condition to check for the deepseek_v32 model type is not fully robust. It could raise an AttributeError if model_config.hf_text_config is None, which would cause the application to crash. It's safer to use getattr to access nested attributes with a default value to prevent such crashes. I've refactored the logic to be more robust and readable.

Suggested change
if model_config and speculative_config and \
hasattr(model_config.hf_text_config, "model_type") and \
model_config.hf_text_config.model_type == "deepseek_v32" and \
speculative_config.enforce_eager:
speculative_config.enforce_eager = False
if model_config and speculative_config and speculative_config.enforce_eager:
hf_text_config = getattr(model_config, "hf_text_config", None)
if (hf_text_config and
getattr(hf_text_config, "model_type", None) == "deepseek_v32"):
speculative_config.enforce_eager = False


@classmethod
def import_kernels(cls) -> None:
# Directly importing vllm_ascend_C prevents ASCEND_RT_VISIBLE_DEVICES
Expand Down