[hacZ17]

Contributor intent

1. What outcome do you want to achieve?

Make the AMD CI Qwen3-Omni job accept the production deploy's platform contract: Thinker prefix caching stays on for CUDA and off on ROCm. Do not change CUDA behavior.

2. What is the highest-leverage variable for that outcome?

test_structured_multistage_config_reaches_runtime requires thinker_vllm.cache_config.enable_prefix_caching is True on every platform.

3. How will this work move that variable?

Expect that runtime flag to equal current_omni_platform.is_cuda(). That is True on CUDA and False on ROCm, which is the value the ROCm overlay already applies.

4. Whose customer or user needs does this address, and what are those needs?

AMD CI and ROCm serving of vllm_omni/deploy/qwen3_omni_moe.yaml. That deploy keeps Thinker prefix caching off outside the CUDA V1 turn profile that #9203 validated.

Summary

Build 14512 job mi300_2: Omni Model Test Qwen3-Omni (01a169f6-90dd-4c99-8105-3f87f7b85dff) ran current main e12f297536d0a5ae6b6c503c26236f0b2fb66719. It is not a stale SHA. The checkpoint loaded, then this assertion failed:

FAILED tests/e2e/offline_inference/test_qwen3_omni.py::test_structured_multistage_config_reaches_runtime[omni_runner0] - assert False is True
E       assert False is True
E        +  where False = CacheConfig(... enable_prefix_caching=False ...).enable_prefix_caching

#9203 turns stage-0 enable_prefix_caching on for CUDA V1 turn serving and documents that the non-CUDA platform blocks keep it off. qwen3_omni_moe.yaml sets it true on stage 0 and false under platforms.rocm (also NPU, MUSA, and XPU). The ROCm overlay is the intended contract, so this change updates the runtime expectation instead of enabling the cache on ROCm. Talker and Code2Wav stay off. CUDA still expects True.

Why this is not a duplicate

• No open or merged PR changes this assertion. tests/config/test_config_factory.py::TestDeployConfigLoading::test_thinker_prefix_cache_defaults already expects CUDA True and ROCm/NPU/XPU/MUSA False, and that test passes on this base.
• [Misc] Clean up redundant Qwen3-Omni Code2Wav sampling defaults #8705 edits later Code2Wav sampling expectations in the same e2e function. It does not change enable_prefix_caching.
• Merged [Bugfix][ROCm] Compare Wan SDPA realignment with the math kernel #9303 (Wan SDPA) and [Bugfix] Read Cosmos3 Edge PP ownership from the live modules #9304 (Cosmos3 PP) do not touch this path. [Bugfix] Read Cosmos3 Edge PP ownership from the live modules #9304 is the current main commit this build ran.

Test Plan

vLLM Version: local unit run imported vLLM 0.31.0. The AMD job log reported vLLM 0.32.0. This patch does not change model code.

vLLM-Omni Commit: 93a45d36487442472f8d0f7265f24f0bc55782d3 on base e12f297536d0a5ae6b6c503c26236f0b2fb66719.

Test Result

.venv/bin/python -m pytest \
  tests/config/test_config_factory.py::TestDeployConfigLoading::test_thinker_prefix_cache_defaults \
  -q --tb=short -p no:cacheprovider

Result: 17 passed in 1.14s.

.venv/bin/pre-commit run --files tests/e2e/offline_inference/test_qwen3_omni.py

Result: passed, including ruff, mypy for Python 3.10, SPDX, and test-mark checks. The commit hook sign-off check passed.

Not run: tests/e2e/offline_inference/test_qwen3_omni.py (needs GPUs and the Qwen3-Omni weights). No model-output or accuracy claim. AMD CI was not rerun.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-23T19:10:02.892871Z We couldn’t complete this review. Comment @codex review to try again.	93a45d3	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to be related to model: Wan.

Model owners: @bjf-frz @NickCao @yenuo26

Routing: @bjf-frz via model owner; @NickCao via CODEOWNERS; @yenuo26 via CODEOWNERS

@hacZ17, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[hsliuustc0106]

@hacZ17 — the updated expectation matches the deployment configuration; the remaining evidence gap is the changed runtime test itself.

• When a compatible environment is available, attach the named test_structured_multistage_config_reaches_runtime result for the failing baseline e12f297536d0a5ae6b6c503c26236f0b2fb66719 and head 93a45d36487442472f8d0f7265f24f0bc55782d3, with versions and command. Acceptance: the ROCm assertion changes from the reported failure to pass while the CUDA-true expectation is preserved. If local hardware is unavailable, retain that explicit exception for maintainer validation rather than treating the 17 config-only tests as execution of this e2e path.

Source-only review; no demonstrated code defect and no hardware tests executed by me.

[andyluo7]

@hacZ17 — current-head maintainer update for f0b560353cbafc5bbd21b733d86ae3c8cc28c8c0.

The functional fix remains the same: ROCm keeps the Thinker prefix cache off. The follow-up commit refines the E2E assertion to compare the materialized vLLM runtime value with thinker.cache_config.enable_prefix_caching, the already-resolved stage value. This makes the test verify startup projection directly and leaves CUDA/ROCm/NPU/MUSA/XPU policy in the deploy YAML plus test_thinker_prefix_cache_defaults.

Current-head validation:

• git diff --check passed.
• pre-commit run --files tests/e2e/offline_inference/test_qwen3_omni.py passed, including Ruff, formatting, mypy, CI-mark, SPDX, and forbidden-import checks.
• The local Python environment does not have vllm, so I did not claim a local execution of the config or GPU E2E test on this head. The original 17-case config result was collected on 93a45d36487442472f8d0f7265f24f0bc55782d3; this follow-up changes only the E2E assertion and comment.
• No production code or deploy YAML changed. There are no unresolved review threads.

I added the ready label to trigger exact-head Buildkite CI. Approval should wait for the AMD Qwen3-Omni model job to exercise test_structured_multistage_config_reaches_runtime on this head.

[andyluo7]

Approved at exact head 52e003fc28dd099c7a3d3337178c56ad5eaac937.

The test now validates that the resolved Thinker cache setting reaches the materialized vLLM runtime config, while platform policy remains covered by the deploy/config tests. No production code or deploy YAML changes.

Exact-head evidence:

• AMD build #14522 passed all 51 jobs. mi300_2: Omni Model Test Qwen3-Omni collected test_structured_multistage_config_reaches_runtime and finished with 9 passed, 7 skipped, 14 deselected.
• CUDA build #17960 passed all 31 jobs. Omni · Qwen3-Omni Test collected the same test and finished with 16 passed, 23 deselected.
• GitHub build, pre-commit, CodeQL, DCO, and docs checks passed.
• No unresolved review threads.