Skip to content

[Bugfix][DBO] Run the profile run without DBO when DBO is enabled. - #52176

Open
SageMoore wants to merge 1 commit into
vllm-project:mainfrom
neuralmagic:sage/dbo-profile-run-sizing
Open

SageMoore wants to merge 1 commit into
vllm-project:mainfrom
neuralmagic:sage/dbo-profile-run-sizing

Conversation

@SageMoore

Copy link
Copy Markdown
Contributor

Purpose

Running the profile run with DBO effectively sets the max batch size to max_num_tokens // 2. If vllm bails out of DBO for any reason, it can end up in a situation where it's running with a batch size that is larger than max_num_tokens // 2. Deep EP v2 crashes in this case because an internal buffer wasn't sized correctly, but it's plausible that there is other undesirable behavior in the system when we encounter this scenario.

The fix is to run the profile run with DBO explicitly disabled.

Test Result

vllm serve deepseek-ai/DeepSeek-V2-Lite \
--data-parallel-size 2 \
--enable-expert-parallel \
--all2all-backend deepep_highthroughput \
--enable-dbo \
--dbo-decode-token-threshold 32 \
--dbo-prefill-token-threshold 32

lm_eval result

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.3798|±  |0.0134|
|     |       |strict-match    |     5|exact_match|↑  |0.3783|±  |0.0134|

Signed-off-by: Sage Moore <sage@neuralmagic.com>
@SageMoore
SageMoore requested a review from njhill as a code owner August 13, 2026 14:24

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the bug Something isn't working label Aug 13, 2026
@njhill njhill added the mrv1-only Issues/PRs which apply only to Model Runner V1 (not applicable to Model Runner V2) label Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working mrv1-only Issues/PRs which apply only to Model Runner V1 (not applicable to Model Runner V2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants