[tzhouam]

Contributor intent

1. What outcome do you want to achieve?

Align vLLM-Omni with the final vLLM v0.32.0 release and retain multimodal inference compatibility.

2. What is the highest-leverage variable for that outcome?

Regressions unique to the alignment branch, measured against an upstream-main Buildkite baseline.

3. How will this work move that variable?

Update the runtime integration, release wheels, container defaults and documentation, then compare full paired Buildkite runs and debug failures locally.

4. Whose customer or user needs does this address, and what are those needs?

All vLLM-Omni users who need alignment with upstream vLLM.

Purpose

Align with vLLM v0.32.0, commit 5df63a7203cdb9328d824a27b682e77c7b9928be.

• Adapt the affected model-runner, parallel-state, layer and preprocessing interfaces. Preserve soundfile-first automatic audio decoding and explicit decoder selection.

• Update dependencies, Python support (>=3.11,<3.14), container references and installation guidance. Pin the CUDA CI image by digest and verify the installed wheel's build commit; retain cross-container release-equality checks.

• Preserve native pooling results and structured cache-miss errors while keeping tokenizer-free generation outputs intact. Reject malformed Qwen video scopes through the validation error path, including mixed flat/scoped options.

• Forward profiling delay and iteration limits through inline/subprocess diffusion and NPU workers while preserving default calls.

• Merge Omni main 71f0fa47a, resolving the CuMem free-callback conflict with [Bugfix][Diffusion] Fix inline CuMem shutdown and FA3 FP8 CPU numerics #9063 by keeping its _releasable_handle() path.

• Fix a latent Qwen-Image-2.1 decode-graph use-after-free that surfaced as a CUDA illegal memory access, and seed the Qwen3-TTS Base PCM quality cases (details below).

The comparison now contains 86 files. Header-only edits and independent presentation, configuration, test-fixture and diagnostics work were removed. The independent work is preserved as five commits on the split branch; that branch also repairs the Helm snippet indentation. The weakened ROCm assertions were discarded.

Validation

Local tested head: 14b6ea49aad225118543b848a0b94088a476808a, using Python 3.12 and the pinned vLLM 0.32.0 runtime with devices hidden.

• 553 distinct targeted CPU tests passed: output/preprocessing 362, profiling/diffusion dispatch 66, orchestrator error/cache handling 111, Docker/runtime guards 14. New pooling, video-scope and profiling regressions reproduced failures before their fixes.
• Scoped Ruff, formatting, SPDX, forbidden-import, CUDA-call and test-mark checks pass. All 65 changed Python files parse.
• Existing gate failures remain disclosed: 64 Markdown lint warnings reproduce on the comparison merge base; 30 test mypy errors reproduce on the original PR head. The full library mypy gate hits the same third-party librosa Python-3.12 syntax blocker on both snapshots. An isolated check of the nine modified library files introduces no errors (27 baseline, 26 current).
• Physical ROCm/XPU/NPU validation and a fresh full GPU comparison remain pending. NPU profiling tests validate forwarding with an external Ascend stub, not device execution.

Earlier GPU evidence

The full comparison below tested 737c90cb3fb7fee0666b09b2a7474361c1c7f2ab, not the current head:

• Alignment #3171: 119 passed, 3 failed, 1 broken.
• Main baseline #3166, a965251bf1a3db3ab6871307ec39c938c0634479: 120 passed, 2 failed, 1 broken.

Resolved alignment-only failures

Both were reproduced and fixed locally on L20X; current head 89195e718e1f6c4abbc63da5cdee1b69504c26e1.

Qwen3-TTS Base PCM HNR (0.60 dB < 1.0 dB): test flakiness, not a v0.32 regression. The case sends unseeded requests, so its audio follows the server's request-order random stream. Fifty unseeded requests on upstream main with vLLM 0.31 and fifty on this branch with vLLM 0.32 produced byte-identical PCM for all 50 responses (SHA-256), including the same 0.958 dB sample below the floor. The test now runs seeds 0 and 42 for both async modes with the unchanged 1.0 dB floor; all four cases pass (2.01–2.40 dB).

Qwen-Image-2.1 CUDA illegal memory access at concurrency 8: fixed. Decode-graph warm-up runs on a new pooled stream. PyTorch recycles 32 pooled streams, so every 32nd warm-up runs on the capture stream and allocates that stream's cuBLAS workspace outside the graph pool (observed at capture 33 in the real workload). The captured graph then references memory that eviction plus empty_cache() releases under surviving graphs. Workspaces are now cleared around capture, as torch cudagraph trees do. The new regression test fails 3/3 with cudaErrorIllegalAddress without the fix and passes 3/3 with it; test_decode_graph.py passes 15/15, and the original benchmark passes, including 160/160 requests at concurrency 8. The decode-graph code is unchanged from main, so main carries the same latent bug.

Remaining limits

A fresh H100 Buildkite comparison on the current head is still required; the fixes above were validated on L20X. These local checks do not establish a regression-free branch.

AI assistance: Codex, Claude and InferMatrixCopilot assisted with compatibility changes, documentation, diagnostics, validation and this description. Human review remains required.

[natureofnature]

【Conclusion: Static review found no blocking code issues in commit 14b6ea4.】
【Action recommended: Reuse applicable existing GPU results and complete the planned H100 comparison for the updated branch, checking seeded TTS quality and concurrent image generation without illegal memory access.】