fix(container): backport vllm PR #40932 to unblock Qwen3-VL disagg multimodal

[furionw]

Why

The vllm-runtime image pins vllm_ref: v0.20.1, which is missing upstream vllm-project/vllm#40932 ([Bugfix] Remove invalid deepstack boundary check for Qwen3-VL). That fix removes a strict size check originally added by vllm-project/vllm#40145 ([Opt] Optimize deepstack buffer handling for multimodal Qwen3 models). #40932 landed on main and shipped on the v0.20.1rc0 tag, but it was dropped between rc0 and the final v0.20.1 tag — so the 1.2.0rc0 image inherits the regression.

Symptom: the EngineCore crashes on the first inference of any disaggregated multimodal request to a Qwen3-VL model: ValueError: Requested more deepstack tokens than available in buffer: num_tokens=288 > self.deepstack_input_embeds_num_tokens=276 — the gap is cudagraph capture-bucket padding the scheduler can't avoid.

A clean vllm_ref bump isn't an option: v0.20.1rc0 predates v0.20.1 on the release branch (regresses unrelated commits), main carries unrelated changes, and there's no patched-v0.20.1 tag.

What Change

• Add per-vllm-ref container/deps/vllm/patches/<ref>/ directory
• Vendor [Bugfix] Remove invalid deepstack boundary check for Qwen3-VL vllm-project/vllm#40932 as 0001-pr40932-...patch under patches/v0.20.1/
• install_vllm.sh applies any matching patches after the PyPI wheel install; skipped when VLLM_VER >= 0.21.0 (assumes upstream merges by then)

Test Plan

• Applied same patch to a v0.20.1 image, ran disagg_multimodal_e_pd.sh --model Qwen/Qwen3-VL-2B-Instruct on 2 GPUs with cudagraph active (bucket 288 captured)
• 3/3 multimodal chat completions returned 200 with correct image descriptions; 0 Requested more deepstack / EngineCore encountered a fatal error lines in logs

Summary by CodeRabbit

• Bug Fixes

  • Fixed invalid boundary validation checks in Qwen3 model processing that could trigger unexpected errors during inference.

• Chores

  • Improved vLLM installation process with smart patch management that automatically applies version-specific patches for enhanced compatibility with different vLLM releases.

[coderabbitai]

Walkthrough

The PR introduces a mechanism to apply vendored vLLM patches during container build and includes a bugfix patch that removes invalid boundary checks in Qwen3-VL model code. The installation script conditionally applies patches based on vLLM version, skipping v0.21.0 and later, while a new patch file targets two Qwen3 model classes.

Changes

vLLM Patch Infrastructure and Qwen3-VL Fix

Layer / File(s)	Summary
Patch application infrastructure container/deps/vllm/install_vllm.sh	Shell script adds conditional post-install patching that computes base vLLM version, skips for v0.21.0+, locates the installed vllm site-packages directory, and applies all *.patch files from version-specific patch directories using patch -p1.
Qwen3-VL deepstack boundary check removal container/deps/vllm/patches/v0.20.1/0001-pr40932-remove-invalid-deepstack-boundary-check-for-qwen3-vl.patch	Patch removes invalid ValueError boundary checks that were raised when num_tokens exceeded buffered deepstack_input_embeds_num_tokens in Qwen3OmniMoeThinker and Qwen3VL models during both buffer fetch and clear/zero operations.

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: backporting vllm PR #40932 to fix Qwen3-VL disaggregated multimodal support in the container.
Description check	✅ Passed	The description provides detailed context (Why, What Change, Test Plan) but lacks explicit mapping to the template sections (Overview, Details, Where to start, Related Issues).
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

container/deps/vllm/install_vllm.sh (1)

264-264: ⚡ Quick win

Derive PATCH_DIR from the script location instead of hard-coding /tmp/deps.

This path assumption is brittle; if the image layout changes, patching will be skipped and the regression can reappear.

Proposed change

+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PATCH_DIR="/tmp/deps/vllm/patches/${VLLM_REF}"
+PATCH_DIR="${SCRIPT_DIR}/patches/${VLLM_REF}"

Based on learnings: "Flag hard-coded portability-reducing constants in shell/scripts across the repository (e.g., fixed memory sizes, fixed memory fractions, static ports, hard-coded temporary file names)."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@container/deps/vllm/install_vllm.sh` at line 264, The PATCH_DIR is hard-coded
to /tmp/deps which breaks portability; change how PATCH_DIR is computed in
install_vllm.sh to derive it from the script's location instead (use the script
directory as the base and join "patches/${VLLM_REF}" to it), e.g., compute the
script base with dirname/BASH_SOURCE (or an equivalent portable approach), then
set PATCH_DIR="${script_base}/patches/${VLLM_REF}" and ensure the directory is
created/validated before use so patching works regardless of image layout.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@container/deps/vllm/install_vllm.sh`:
- Around line 271-274: The patch command in the for loop that iterates over
"$PATCH_DIR"/*.patch can prompt interactively on mismatches and hang builds;
update the invocation of patch (the line using patch -p1 -d
"$VLLM_SITE_PACKAGES" < "$p") to run non-interactively by adding the --batch and
--forward flags so it suppresses prompts and ignores already-applied or reversed
patches.

---

Nitpick comments:
In `@container/deps/vllm/install_vllm.sh`:
- Line 264: The PATCH_DIR is hard-coded to /tmp/deps which breaks portability;
change how PATCH_DIR is computed in install_vllm.sh to derive it from the
script's location instead (use the script directory as the base and join
"patches/${VLLM_REF}" to it), e.g., compute the script base with
dirname/BASH_SOURCE (or an equivalent portable approach), then set
PATCH_DIR="${script_base}/patches/${VLLM_REF}" and ensure the directory is
created/validated before use so patching works regardless of image layout.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7c25ac7e-a80e-46bf-b559-f47d8e663aa7

📥 Commits

Reviewing files that changed from the base of the PR and between 22805f0 and 7c3f895.

📒 Files selected for processing (2)

• container/deps/vllm/install_vllm.sh
• container/deps/vllm/patches/v0.20.1/0001-pr40932-remove-invalid-deepstack-boundary-check-for-qwen3-vl.patch

[indrajit96]

Quick Question
"disaggregated multimodal request to a Qwen3-VL" failed
Did our CI catch this when vllm was upgraded?
If not we need a pre-merge also?

[furionw]

Quick Question "disaggregated multimodal request to a Qwen3-VL" failed Did our CI catch this when vllm was upgraded? If not we need a pre-merge also?

This errors shows up in cudagraph.

• our CI uses --single-gpu. our EPD tests disable cudagraph under single-gpu
• SA uses 2 GPUs (DYN-3008) and surfaces this issue

[rmccorm4]

Thanks Qi - How was this caught? pre-merge CI, post-merge CI, something else?

Is this something we need to fix/CP into release/1.2.0 branch to avoid lots of Qwen3-VL failures or QA bugs? If so, please raise awareness on it in release channel

[dmitry-tokarev-nv]

please apply copyright/license comment

[krishung5]

Thanks for the fix, Qi! If you haven't done this, could we add some ticket to remove this patch when vLLM ships 0.20.2 with #40932 backported?