Skip to content

[Bugfix] Fix request batching regression on Ming-flash-omni ImageGen - #4837

Merged
Gaohan123 merged 3 commits into
vllm-project:mainfrom
yuanheng-zhao:fix/request-batching-ming-imagegen
Jul 3, 2026
Merged

Gaohan123 merged 3 commits into
vllm-project:mainfrom
yuanheng-zhao:fix/request-batching-ming-imagegen

Conversation

@yuanheng-zhao

@yuanheng-zhao yuanheng-zhao commented Jul 2, 2026 •

Copy link
Copy Markdown
Collaborator

PLEASE FILL IN THE PR DESCRIPTION HERE.

Purpose

Fix regression imported in #4079
Resolves #4839

After some bridge func runs, diffusion_prompt is a list[dict]. In vllm_omni/engine/orchestrator.py, Both of the two submit paths reject a list for diffusion stages, however, the unwrap diffusion_prompt = diffusion_prompt[0] get covered only when already_submitted is true.

This PR fixed the above regression, as well as a trivial DiffusionOutput issue in Ming Image-gen pipeline .

Test Plan

vllm serve Jonathan1909/Ming-flash-omni-2.0 --omni \
    --deploy-config vllm_omni/deploy/ming_flash_omni_image.yaml \
    --port 8091

Send a single request with

curl -s http://127.0.0.1:8091/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "Jonathan1909/Ming-flash-omni-2.0",
      "messages": [{"role": "user", "content": "Please draw a cute cat."}],
      "modalities": ["image"]
    }' | jq -r '.choices[0].message.content[0].image_url.url | split(",")[1]' | base64 -d > ming_imagegen.png

vLLM Version: 0.24.0

vLLM-Omni Commit: 5c51782

Test Result

ming_imagegen

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.
(anything written below this line will be removed by GitHub Actions)

Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@yuanheng-zhao

Copy link
Copy Markdown
Collaborator Author

PTAL @yJader , @ZhengWG , @princepride
cc @tzhouam

@yuanheng-zhao yuanheng-zhao changed the title [Bugfix][WIP] Fix request batching regression on Ming-flash-omni ImageGen [Bugfix] Fix request batching regression on Ming-flash-omni ImageGen Jul 2, 2026

@yJader yJader left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

)
return
if already_submitted and len(diffusion_prompt) == 1:
if len(diffusion_prompt) == 1:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

already_submitted is only used here to distinguish the orchestrator submission path (submit_update vs submit_initial). For diffusion, both paths eventually enter the same diffusion client API and then DiffusionEngine.step/step_streaming -> add_request, so DiffusionEngine does not attach different semantics to initial vs update submissions.

Since diffusion StagePool rejects list prompts in both submit_initial and submit_update, singleton-list normalization should not depend on already_submitted. Applying it consistently makes the diffusion payload contract clearer and avoids the initial path still passing a list-shaped payload.

@Gaohan123 Gaohan123 added this to the v0.24.0 milestone Jul 3, 2026

@Gaohan123 Gaohan123 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks

@Gaohan123 Gaohan123 added the ready label to trigger buildkite CI label Jul 3, 2026
@Gaohan123
Gaohan123 enabled auto-merge (squash) July 3, 2026 02:26
@Gaohan123
Gaohan123 merged commit 06329c6 into vllm-project:main Jul 3, 2026
7 of 9 checks passed
@yuanheng-zhao
yuanheng-zhao deleted the fix/request-batching-ming-imagegen branch July 3, 2026 03:56
SamitHuang pushed a commit to SamitHuang/vllm-omni that referenced this pull request Jul 7, 2026
wkutak pushed a commit to wkutak/vllm-omni-cosmos-genai-nim that referenced this pull request Jul 13, 2026
…llm-project#4837)

Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
khairulkabir1661 pushed a commit to khairulkabir1661/vllm-omni that referenced this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Single element diffusion bridge output rejected on initial submit (Diffusion list-prompt ValueError)

3 participants