[tzhouam]

Purpose

Seventh cherry-pick batch for release/v0.32.0 (follows #9206, #9251, #9295, #9334, #9352, #9394; batch 6 is #9397). Base: ab216c0b.

Scope: bugfix PRs merged to main that close v0.32.0 milestone / Outstanding-DI dashboard bugs, plus the milestone-tagged CI dependency #9393. Audited every bugfix/CI/doc PR merged to main since the branch cut against release history, merged batches and the open release PRs (#9397, #9377, #9335, #9381).

Included (git cherry-pick -x, main merge order)

• [Bugfix] Keep the offline Omni engine alive after a non-fatal request error #8450 [Bugfix] Keep the offline Omni engine alive after a non-fatal request error (fixes [Bug]: Offline Omni.generate() closes the engine after a non-fatal client error #8449, milestone)
• [CI/Build] Install fa3-fwd 0.0.3 with the dev extra #9393 [CI/Build] Install fa3-fwd 0.0.3 with the dev extra (milestone; pairs with [Bugfix] Optionally bind fa3-fwd dense FlashAttention on Hopper #9201 already on release)
• [Bugfix] Reject oversized stage budgets on integrated GPUs #9404 [Bugfix] Reject oversized stage budgets on integrated GPUs (fixes [Bug]: On unified-memory GPUs, request_memory_tolerant gives a stage all free host memory #8550)
• [Bugfix] Preserve the diffusion parallel knobs when engine args come from parsed CLI args #8041 [Bugfix] Preserve the diffusion parallel knobs when engine args come from parsed CLI args (fixes [Bug]: Diffusion parallel knobs are dropped when engine args are built from parsed CLI args #8037, [Bug]: OmniEngineArgs.from_cli_args silently drops OrchestratorArgs-owned diffusion engine args (library path) #8503)
• [Bugfix][Benchmark] Match official Omni-DuplexEval GT reference handling #9056 [Bugfix][Benchmark] Match official Omni-DuplexEval GT reference handling (fixes # [Bug][Benchmark][Duplex] GT references are injected without empty-string filtering or dedup, deviating from the official Omni-DuplexEval reference implementation #9054, milestone)
• [Bugfix] Align MiniCPM-o reference audio with Transformers while retaining vision SDPA #9386 [Bugfix] Align MiniCPM-o reference audio with Transformers while retaining vision SDPA (fixes [Bug]: MiniCPM-o 4.5 AV duplex ends with incomplete responses and post-commit drain timeout on a 180s stream #8637)
• [Bugfix] Stop vocoder stages from declaring SupportsPP without make_empty_intermediate_tensors #6860 [Bugfix] Stop vocoder stages from declaring SupportsPP without make_empty_intermediate_tensors (fixes [Bug]: Vocoder stages declare SupportsPP without make_empty_intermediate_tensors (PP>1 would hit the #6790 failure) #6859)
• [Bugfix][Benchmark] Enable audio_ttfp goodput SLOs on the current bench CLI #9322 [Bugfix][Benchmark] Enable audio_ttfp goodput SLOs on the current bench CLI (fixes [Bug]: audio_ttfp goodput SLO is rejected by the CLI and not applied by Omni benchmark metrics #5668)
• [Bugfix][Engine] Preserve non-parallel diffusion engine args across from_cli_args #8585 [Bugfix][Engine] Preserve non-parallel diffusion engine args across from_cli_args (fixes [Bug]: OmniEngineArgs.from_cli_args silently drops OrchestratorArgs-owned diffusion engine args (library path) #8503)
• [Bugfix][Frontend] fix: park generate() through partial wakes #8891 [Bugfix][Frontend] fix: park generate() through partial wakes (fixes [Bug]: AsyncOmni deadlocks when aborted requests are resubmitted — pending submits never complete under pause and sleep()'s admission barrier waits forever #8678, milestone)
• [Bugfix] Fix Qwen2.5-Omni talker end conditioning #9371 [Bugfix] Fix Qwen2.5-Omni talker end conditioning (dependency of [Bugfix] Qwen2.5-Omni: assemble talker spans from runner phase metadata #5019: its tests expect the EOS/pad end-conditioning rows)
• [Bugfix] Fix Qwen2.5-Omni speaker test fixture #9389 [Bugfix] Fix Qwen2.5-Omni speaker test fixture (companion test fix)
• [Bugfix] Qwen2.5-Omni: assemble talker spans from runner phase metadata #5019 [Bugfix] Qwen2.5-Omni: assemble talker spans from runner phase metadata (fixes [Bug]: Qwen2.5-Omni talker misroutes multi-token decode spans and corrupts per-position conditioning #5018, milestone)
• [Perf][Core] Materialize duplex Realtime response history at read boundaries #8691 [Perf][Core] Materialize duplex Realtime response history at read boundaries (fixes [Bug]: Duplex Realtime projection rebuilds cumulative response history on every delta (O(n^2) CPU per response) #8690, milestone)
• [Bugfix][Kernel] Fix MoT GEMM rejecting a one-sided text/vae bias #6490 [Bugfix][Kernel] Fix MoT GEMM rejecting a one-sided text/vae bias (fixes [Bug]: MoT fused GEMM rejects a one-sided bias, so MoT linear layers crash with their default bias flags #6488)
• [Bugfix][VoxCPM2] Reject outer full CUDA graph modes at startup #9458 [Bugfix][VoxCPM2] Reject outer full CUDA graph modes at startup (fixes [Bug]: VoxCPM2 EngineCore dies on first request when CUDA graphs are enabled #6003)
• [Bugfix][Diffusion] Prepare the MoE LoRA context #9443 [Bugfix][Diffusion] Prepare the MoE LoRA context (fixes [Bug][B200]: Release CI, HunyuanImage3-DIT · Accuracy Test, AssertionError: LoRA context must be set #9232, B200 release CI)
• [Bugfix] CosyVoice3: reject unsupported generation APIs and guard prefill embeddings #4738 [Bugfix] CosyVoice3: reject unsupported generation APIs and guard prefill embeddings (fixes [Bug]: CosyVoice3 talker text-only /v1/completions triggers CUDA device-side assert and kills EngineCore #4721, milestone)

Backport adaptations (conflicts were only with main-only context)

• Revision 2: the first CI run failed 6 cases in tests/model_executor/models/qwen2_5_omni/test_qwen2_5_omni_talker_preprocess.py ([Bugfix] Qwen2.5-Omni: assemble talker spans from runner phase metadata #5019) because [Bugfix] Qwen2.5-Omni: assemble talker spans from runner phase metadata #5019 was written on top of [Bugfix] Fix Qwen2.5-Omni talker end conditioning #9371, which was not on release. Added [Bugfix] Fix Qwen2.5-Omni talker end conditioning #9371 and [Bugfix] Fix Qwen2.5-Omni speaker test fixture #9389 before [Bugfix] Qwen2.5-Omni: assemble talker spans from runner phase metadata #5019.

• [Bugfix] Preserve the diffusion parallel knobs when engine args come from parsed CLI args #8041, [Bugfix][Engine] Preserve non-parallel diffusion engine args across from_cli_args #8585, [Bugfix] CosyVoice3: reject unsupported generation APIs and guard prefill embeddings #4738: tests/engine/test_arg_utils.py / tests/worker_v2/test_omni_model_state.py conflicted because the upstream hunks carried neighbouring tests from [OOT] [vllm-metal] Add external CustomOp registration [1/3] #8943 and [Refactor] Retire the generic deferred MTP path in MRV2 #9346, which are not on the release branch. Kept only each PR's own additions; dropped the foreign tests.

• [Bugfix] Align MiniCPM-o reference audio with Transformers while retaining vision SDPA #9386: dropped the docs/serving/realtime_duplex_api.md and tests/model_executor/models/fish_omni/test_fish_omni_config.py hunks. Both edit content introduced by [Docs] Clarify MiniCPM-o attention backend comparisons #9344 and a later fish-omni commit that are not on release; the MiniCPM-o fix itself (other 25 files) is unchanged.

Upstream authors, sign-offs and source SHAs are preserved.

Not included

• Already in open release PRs: [Bugfix] fix flashinfer backend and torch.compile(fullgraph=True) incompatible #4989 ([cherry-pick][release/v0.32.0] Backport FlashInfer fullgraph fix (batch 6) #9397), [Bugfix] Fix grouped chunk interaction latency #8810 ([Bugfix] Backport grouped chunk interaction latency fix to v0.32.0 #9377), [Bugfix][Diffusion] Reject unsupported pre-tokenized prompt ids at engine admission #8448 ([Bugfix][release/v0.32.0] Reject unsupported pre-tokenized diffusion prompt ids #9335), [Doc] MiniMax-H3: document the Ref2VA dialogue prompt contract #9331 ([Doc][release/v0.32.0] MiniMax-H3: separate reported observations from verified claims #9381).
• CI-only: [CI/Build] Restore SenseNova-U1 smoke and H100 accuracy coverage #8759 (SenseNova-U1 coverage, +1020), [CI/Build] Update MiniCPM-o A3 C8 TTFT baseline #9388 (A3 TTFT baseline), [Bugfix][CI/Build] Fix MiniCPM-o watchdog CI regressions #9374 (watchdog diagnostics), ROCm/AMD CI chain [CI/Build][ROCm] Select qualified ROCm tests in AMD CI #9320 [CI/Build] Fix AMD R2 collection and Qwen graph oracle #9340 [CI/Build] Filter AMD PR nightlies by source changes #9372 [Bugfix][CI/Build] Add MiniCPM-o Stage-2 timeout diagnostics #9370 [CI/Build] Keep AMD nightly dependency check independent of job count #9444 [CI/Build][ROCm] Update Qwen3-Omni AMD collection count #9448 [CI/Build][ROCm] Shorten READY R2-01 and AITER consumer jobs #9439 (conflict on .buildkite/amd/*), ROCm/NPU nightly additions.
• Other clean bugfixes not tied to milestone bugs, deferred for a separate batch: [Bugfix] Lock shared memory before opening segments #9285 [Bugfix] Place the MAGI-2 preview pipeline on the current accelerator #8655 [Bugfix] Validate diffusion speech sampling parameters #8685 [Bugfix] Validate served model names in diffusion speech #8882 [Bugfix][AURA] Keep sentence TTS turns alive across vision and segments #8829 [Bugfix] Reuse one Qwen3-Omni fault payload across concurrent requests #9407 [Bugfix][XPU] Support auto-round-lib >= 0.15 in SageAttention backend #9255 [Bugfix][ROCm] Make Qwen OOM recovery pressure deterministic #9425 [Bugfix] Release diffusion ZMQ resources on loop initialization errors #8419 [CI/Build] Re-enable turn-based server VAD realtime test #9300.
• Conflicting, not milestone: [Bugfix] Read Cosmos3 Edge PP ownership from the live modules #9304 [Bugfix] Expect Qwen3-Omni Thinker prefix cache only on CUDA #9312 [Bugfix] Reject invalid Omni-DuplexEval judge scores #9390 [CI/Build] Avoid busy polling in stage memory lifecycle tests #9406 [Bugfix][Core] Schedule native MRv2 async chunks atomically #9060 [Bugfix] Avoid retaining tensors in warning_once exception logging #9445 [Bugfix] Fix MiniMax-H3 ComfyUI workflow schemas #9337; [Bugfix][Hardware][AMD] Initialize Mammoth attention test weights #9341 already adapted into [cherry-pick][release/v0.32.0] Backport 14 milestone fixes and regressions (batch 4) #9334.
• Perf/feature: [1/N][Perf][Core] Stop the omni prefix cache from collapsing under concurrent hits #9162 [2/N][Perf][Core] Cut the per-step cost of the omni prefix cache #9193 [BugFix][TTS] Support long-form OmniVoice generation #6409 [Bugfix][Wan2.2] Cache-DiT: support low-noise-only (single transformer_2) runs and discover cache targets #8800 (needs [Bugfix][Diffusion] Make Cache-DiT effective when serving Wan2.2 T2V/I2V/S2V without num_inference_steps #7335) [Kernel] Split the fused GroupNorm+SiLU reduction across CTAs #6662.

Test Plan

• ruff check / ruff format --check on all 66 changed Python files: clean.
• Local pytest is limited to vLLM 0.30 on this machine; tests/worker_v2/test_omni_model_state.py non-cosyvoice3 cases pass (63), the remaining failures are ImportError: MultiModalDummyOptions from the vLLM version gap, not from these picks.
• PR CI on this branch.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-27T07:43:32.132697Z We couldn’t complete this review. Comment @codex review to try again.	5af7827	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[amy-why-3459]

Review (core logic; 82-file cherry-pick batch)

Seventh release/v0.32.0 backport batch. Scoped this pass to the production engine / entrypoint / duplex / diffusion-LoRA / model-guard paths plus the documented conflict adaptations — not a line-by-line re-audit of every test asset.

Covered

• Engine queue / pause (#8891 / #8450): frontend puts go through the orchestrator loop (orchestrator_loop + _put_request_queue*) instead of blocking on sync_q.put; never-admitted enqueue failures reset the frontend MM cache, while half-enqueued groups leave the cache alone so orphan revoke can finish. Streaming updates share the same cancellable path.
• Artifact dirs: client-supplied REQUEST_ARTIFACT_DIRS_KEY is stripped before the transform; cleanup uses server-tracked dirs only — matches the path-injection hardening intent.
• Duplex history (#8691): materialize-on-read + truncation cursors / item_truncation_sources look coherent; pending audio-text marks stay O(delta) until materialization; native SignalTurn truncate now records the same cursor as the control path.
• Diffusion LoRA / MoE (#9443): _ensure_moe_lora_context at init, republish on activate / deactivate / rollback, and mapping derived from _active_adapter_id address the silent no-LoRA MoE path.
• MoT GEMM (#6490): one-sided bias acceptance is consistent with the reported rejection bug.
• Engine-args / stage projection (#8041 / #8585 / #9404): diffusion parallel knobs preserved across from_cli_args; LLM stages still exclude diffusion-only load knobs (incl. enable_broadcast_weight_load); oversized stage-budget rejection on integrated GPUs is present.
• Model guards: CosyVoice3 unsupported-API / prefill embed guard (#4738), VoxCPM2 outer full-cudagraph reject (#9458), Qwen2.5-Omni talker span assembly from runner phase metadata (#5019), MiniCPM-o CUDA Thinker TRITON_ATTN pin for reference-audio parity (#9386).
• Bench / eval: _gt_references matches Omni-DuplexEval empty/dedupe semantics (#9056); goodput validates audio_ttfp (not the old audio_ttft typo) and treats 0.0 as unmeasured for non-duplex backends (#9322).
• Backport adaptations in the PR body (drop foreign tests from #8943/#9346; drop #9386 docs/fish hunks not on release) look correctly scoped.

Non-blocking notes

1. Large surface (engine + duplex + LoRA + several model guards). Release CI on this branch remains the real integration gate for cross-pick interactions that unit tests cannot catch in isolation.
2. #8891 parking through partial wakes depends on abort/cancel releasing admission slots as described — worth watching pause/resume + duplex load in release CI for any residual wedge symptoms.
3. MoE LoRA publish-after-commit is easy to regress if a future change mutates _active_adapter_id ordering; the added tests look like the right lock.

No blockers spotted in the core paths reviewed here.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/ar_runtime.md, docs/design/module/entrypoints.md, docs/design/module/diffusion/index.md.

Module owners: @david6666666 @Isotr0py @princepride

Routing: @david6666666 via module named in the PR description; @Isotr0py via module named in the PR description; @princepride via module named in the PR description

@tzhouam, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit 56aa11c085e4 produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

[amy-why-3459]

Delta review (56aa11c0, Revision 2)

Tip moved after the first pass (5af78273): Revision 2 inserts #9371 and #9389 ahead of #5019 so the talker-preprocess tests that expect EOS/pad end-conditioning rows actually match the release tree. Re-checked the new dependency picks plus a spot check that prior core coverage still holds on the rewritten tip.

Covered this pass

• #9371 talker end conditioning: end-of-span EOS/pad rows are assembled for the talker path; new test_qwen2_5_omni_talker_conditioning.py pins the contract the #5019 preprocess cases assume.
• #9389 speaker fixture: speaker-test fixture fix is a small companion so the Qwen2.5-Omni speaker tests stay green on release; no production behavior change beyond the fixture.
• #5019 ordering: with #9371/#9389 ahead of it, the talker span-from-phase-metadata pick should no longer trip the six preprocess failures called out in the Revision 2 note.
• Spot-checked that engine queue/pause (#8891), duplex materialize-on-read (#8691), MoE LoRA context (#9443), MoT one-sided bias (#6490), CosyVoice3 / VoxCPM2 guards, and the documented backport adaptations remain present on the new tip.

Non-blocking

1. Large surface still; release CI on this branch remains the real integration gate for cross-pick interactions.
2. Same note as before on #8891 parking through partial wakes — watch pause/resume + duplex load in release CI.

No blockers spotted in the Revision 2 delta.