[CI][ROCm] Align AMD image with vLLM 0.29

[andyluo7]

Summary

• update the default ROCm Dockerfile base image from vLLM v0.28.0 to v0.29.0
• keep the optional source-build ref aligned with the same release
• fail the image build early if the vLLM APIs required by current vLLM-Omni are missing
• add focused regression tests for the ROCm defaults and the AMD image-build command
• document the vLLM 0.29 ROCm source-build default, version contract, and optional BASE_IMAGE override

Why

vLLM-Omni moved to the vLLM 0.29 API in #7230, including compute_layout_strides, but docker/Dockerfile.rocm still installs vLLM 0.28. Current AMD merge builds therefore build successfully and then fail at runtime before model inference:

• AMD #11660, exact [CI] Isolate layerwise offload memory measurements #6938 merge commit d8d40625
• AMD #11672, current-main commit ffcaaa94

Both report vLLM-Omni 0.29 with vLLM 0.28 and fail importing compute_layout_strides from vllm.v1.kv_cache_interface. This affects the AMD merge suite generally; Stable Audio is only one visible quarantined leaf.

Validation

Current exact head: 15d3c106799d3a90db64df8bdf8a51f961094dec

• focused test from the repository working directory with --strict-markers: 5 passed
• focused test from /tmp with --strict-markers: 5 passed
• changed-file pre-commit hooks, including Markdown lint: passed
• git diff --check: passed
• Docker Hub publishes vllm/vllm-openai-rocm:v0.29.0 for linux/amd64
• vLLM tag v0.29.0 contains the required compute_layout_strides and KVCacheLayout APIs

AMD #11682 validated parent commit f8af7e2d146cfa78629c36bd9ffd463f4f518b39 before the final documentation/comment-only clarification:

1. the image used vllm/vllm-openai-rocm:v0.29.0 and printed Validated vLLM 0.29.0;
2. tools/install_torchcodec_rocm.sh built and installed TorchCodec v0.10.0;
3. the Docker layer containing from torchcodec.decoders import VideoDecoder completed;
4. the Z-Image diffusion worker initialized, launched its orchestrator, and began loading model weights.

The final exact-head AMD rerun remains required before merge because the review clarification changed the PR head after #11682 started.

Scope

This changes the default base used by AMD CI and by users building docker/Dockerfile.rocm without an override, plus static regression coverage and source-build documentation for that contract.

The BASE_IMAGE pin governs Docker builds. Non-Docker installs must install the matching platform-specific vLLM release explicitly; this PR does not add a global vLLM package requirement.

It intentionally does not change the CUDA/XPU/NPU image pins in this PR. Their supported release contracts and remaining 0.28 defaults are tracked separately in #7410 rather than being bumped without backend-specific image and runtime validation.

It also does not change Stable Audio thresholds, offloader behavior, or blocking grades. Stable Audio follow-up remains #7396. The separate vllm/vllm-omni-rocm published-image and prebuilt-installation documentation gap remains #7405.

@yenuo26 @hsliuustc0106, please review the AMD image update and final documentation clarification at exact head 15d3c106799d3a90db64df8bdf8a51f961094dec.

[andyluo7]

Stable Audio follow-up and quarantine disposition are tracked in #7396. The 512 MB threshold and NonBlocking grade remain unchanged; valid measurements start only after this image fix produces a vLLM 0.29 AMD runtime.

[andyluo7]

Author self-review for exact head 997a4915641f76598b9fda4cbcfa9fe022b45ef4:

• Scope is limited to docker/Dockerfile.rocm and one static Buildkite regression test; CUDA, runtime code, thresholds, and job grades are unchanged.
• The ROCm base/default source ref now matches CUDA CI's vLLM v0.29.0 target.
• The image build now imports compute_layout_strides before expensive downstream setup, converting the current repeated runtime failure into an early image-build failure if the API is absent.
• The public vllm/vllm-openai-rocm:v0.29.0 linux/amd64 tag exists, and upstream vLLM v0.29.0 contains the required API.
• Focused regression: 2 passed; changed-file pre-commit, mypy, SPDX, compileall, and git diff --check: passed.

Local Docker execution is unavailable on this Mac. The remaining acceptance gate is an exact-head AMD image build followed by a test job confirming vLLM 0.29 and successful diffusion import/startup.

[andyluo7]

@yenuo26 @hsliuustc0106, no AMD Buildkite context has registered yet for exact head 997a4915641f76598b9fda4cbcfa9fe022b45ef4.

Please start one AMD merge validation for this exact head, preferably through the authorized manual route rather than a broad ready label pulse. The immediate acceptance evidence is:

1. the AMD image build prints Validated vLLM 0.29.0 and completes;
2. the runtime reports vLLM-Omni 0.29 with vLLM 0.29, with no major/minor mismatch warning;
3. a diffusion job imports compute_layout_strides and reaches model startup;
4. the quarantined Stable Audio log is retained for [ROCm][CI] Validate Stable Audio layerwise-offload memory after vLLM 0.29 image alignment #7396, but its memory result is evaluated separately from this image-fix gate.

[vllm-omni-review-bot]

This PR was classified as CI work.

CI owner: @yenuo26 @NickCao

Routing: @yenuo26 via semantic router, CI owner, CODEOWNERS; @NickCao via CODEOWNERS

@andyluo7, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[andyluo7]

@vllm-omni-review-bot please review the ROCm image-version alignment and fail-fast compatibility check on the current exact head.

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Strict under experiment vllm-omni-strict-5050-20260829.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit 9f2656b4c445 produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	3 finding(s) below
Security	no finding reported
Docs / comments	5 finding(s) below
Behavior / compatibility	no finding reported
Correctness	no finding reported

Validated:

• [resolved] AMD exact-head validation ask already owned in PR discussion; not re-filed. Residual: static tests ≠ image-build/runtime gate.
• [resolved] Stable Audio/#7396 disposition already tracked; thresholds/grades unchanged here.
• [claim-verified] Docker Hub tag vllm/vllm-openai-rocm:v0.29.0 exists (linux/amd64, active).
• [claim-verified] Scope claim: only Dockerfile.rocm + tests/buildkite/test_rocm_dockerfile.py changed; CUDA CI Dockerfile.ci untouched at v0.29.0.
• [claim-verified] test_rocm_base_tracks_cuda_vllm_release asserts tag equality vs Dockerfile.ci VLLM_BASE_TAG (CI alignment), not Dockerfile.cuda.
• [validated] Producer of required API: vllm_omni/diffusion/diffusion_kv/layout.py:23 imports compute_layout_strides — explains AMD runtime ImportError on 0.28 bases.

Kept four threads: tighten the fail-fast test to the RUN canary (test integrity), resolve Dockerfile paths from __file__ like sibling Buildkite tests, ask whether non-AMD 0.28 pins are intentional deferral after the ROCm-only 0.29 bump (blast-radius), and rename CUDA_DOCKERFILE to CI taxonomy as a nit. Duplicate pairs 0/3 and 2/5 were collapsed onto the surviving indices.

Verdict: COMMENT

Findings

• **[P2] This diff adds test_rocm_image_fails_fast_on_missing_vllm_api and inserts RU…** — tests/buildkite/test_rocm_dockerfile.pyThis diff addstest_rocm_image_fails_fast_on_missing_vllm_apiand insertsRUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; ..."atdocker/Dockerfile.rocm:46, but the new test only asserts that the import substring appears anywhere in the file (tests/buildkite/test_rocm_dockerfile.py:35). A comment or echo of the same text would keep the named fail-fast gate green while the executing RUN layer could disappear. Pin the assertion to the concrete RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides;` snippet so the test matches the image-build canary this PR claims to guard.

Evidence: tests/buildkite/test_rocm_dockerfile.py:35 assert "from vllm.v1.kv_cache_interface import compute_layout_strides" in dockerfile — bare substring only; docker/Dockerfile.rocm:46 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; print(f'Validated vLLM {vllm.__version__}')" — the executing fail-fast layer the test name claims to guard.

Suggestion: assert (
'RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides;'
in dockerfile
)

• **[P2] ROCm no longer stuck on 0.28 after #7230: docker/Dockerfile.rocm:4 ARG BASE_IM…** — Dockerfile.rocm:4ROCm no longer stuck on 0.28 after #7230: docker/Dockerfile.rocm:4ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0` (also VLLM_VERSION_OR_COMMIT_HASH=v0.29.0). Residual: unchanged sibling pins still on v0.28.0 — docker/Dockerfile.cuda:1, docker/Dockerfile.xpu:21, docker/Dockerfile.npu:2. AMD runtime CI greenness not verified here.

Evidence: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0 — ROCm pin is v0.29.0. Unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0.

• [P2] Residual only: API canary at docker/Dockerfile.rocm:46 runs before TorchCodec a… — ``
  Residual only: API canary at docker/Dockerfile.rocm:46 runs before TorchCodec and COPY/uv omni install; no post-install re-check of vllm. Acceptable because requirements/*.txt do not declare a vllm dependency that would replace the validated package.

Evidence: docker/Dockerfile.rocm:46 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; print(f'Validated vLLM {vllm.__version__}')" — canary; docker/Dockerfile.rocm:48 # Build TorchCodec after any optional nightly vLLM reinstall... then COPY/install TorchCodec; docker/Dockerfile.rocm:70 COPY . ${COMMON_WORKDIR}/vllm-omni and :93–94 uv pip install ... ".[dev]" — omni install after canary with no later Validated vLLM check; unchanged by this diff, present in the PR-time tree: requirements/*.txt have no vllm package pin (grep ^vllm|vllm==|vllm>=|vllm~ → no matches).

• [P2] Fail-fast at docker/Dockerfile.rocm:46 imports compute_layout_strides before om… — Dockerfile.rocm:46
  Fail-fast at docker/Dockerfile.rocm:46 imports compute_layout_strides before omni install; no re-validation after uv pip install ".[dev]" (lines 92–99). Residual only: vllm is not in omni requirements, so install is unlikely to replace it—minor.

Evidence: docker/Dockerfile.rocm:46 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; print(f'Validated vLLM {vllm.__version__}')" — only fail-fast import of compute_layout_strides. Later, unchanged ordering in the same file: docker/Dockerfile.rocm:92–94 RUN cd ${COMMON_WORKDIR}/vllm-omni && \ / uv pip install ... ".[dev]" ... installs omni with no subsequent compute_layout_strides check. Unchanged by this diff, present in the PR-time tree: pyproject.toml:11 dynamic = ["version", "dependencies"] and requirements/common.txt has no vllm package pin (only a comment referencing a vllm_omni path).

• [P2] Residual (out of this PR’s ROCm↔CUDA-CI scope): unchanged docker/Dockerfile.cud… — ``
  Residual (out of this PR’s ROCm↔CUDA-CI scope): unchanged docker/Dockerfile.cuda still pins ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0 while Dockerfile.ci `VLLM_BASE_TAG` and Dockerfile.rocm defaults are v0.29.0.

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0 — contrast docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0 and docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0

• **[P2] This diff lifts only docker/Dockerfile.rocm to vllm/vllm-openai-rocm:v0.29.0…** — docker/Dockerfile.xpuThis diff lifts onlydocker/Dockerfile.rocmtovllm/vllm-openai-rocm:v0.29.0/VLLM_VERSION_OR_COMMIT_HASH=v0.29.0so AMD CI can importcompute_layout_strides. Same-class pins remain on v0.28.0 in docker/Dockerfile.cuda:1, docker/Dockerfile.xpu:21, docker/Dockerfile.npu/Dockerfile.npu.ci:2, and .buildkite/intel/pipeline-intel.yml:13, while unchanged vllm_omni/diffusion/diffusion_kv/layout.py:23` already imports that 0.29 API. Is the lag intentional unpublished-artifact deferral for non-AMD platforms, or should a follow-up track those pins so non-AMD image builds do not keep the old runtime ImportError?

Evidence: diff-changed: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; docker/Dockerfile.rocm:16 ARG VLLM_VERSION_OR_COMMIT_HASH=v0.29.0. unchanged by this diff, present in the PR-time tree: docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.npu.ci:2 ARG VLLM_ASCEND_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0; .buildkite/intel/pipeline-intel.yml:13 VLLM_VERSION: "v0.28.0"; vllm_omni/diffusion/diffusion_kv/layout.py:23 from vllm.v1.kv_cache_interface import KVCacheSpec, KVCacheTensor, compute_layout_strides

[andyluo7]

Updated self-review for exact head 1e2ce96:

Review feedback addressed:

• Dockerfile paths now resolve from Path(file).resolve().parents[2], so the test is independent of the process working directory.
• CUDA_DOCKERFILE / tracks_cuda naming is now CI_DOCKERFILE / tracks_ci, matching docker/Dockerfile.ci as the source of VLLM_BASE_TAG.
• The fail-fast regression now requires a line beginning with the concrete executing RUN python3 -c import canary, rather than accepting the import text anywhere in the Dockerfile.

Scope disposition:

• The unchanged CUDA, XPU, and NPU v0.28 pins are intentionally deferred. This PR is limited to repairing the demonstrated ROCm image/runtime mismatch and does not claim cross-platform release alignment.
• Thresholds and job grades remain unchanged. Stable Audio quarantine and measurement follow-up remain tracked in [ROCm][CI] Validate Stable Audio layerwise-offload memory after vLLM 0.29 image alignment #7396.

Validation:

• focused static test from repository cwd: 2 passed
• focused static test from /tmp: 2 passed
• all changed-file pre-commit hooks: passed
• git diff --check: passed

The focused pytest invocations disabled the repository conftest/addopts because this Mac environment lacks torch and pytest-xdist; no local Docker or AMD runtime result is claimed.

@vllm-omni-review-bot please re-review this exact head.

@yenuo26 @NickCao please start one AMD merge validation for exact head 1e2ce96. Required evidence remains:

1. AMD image build prints Validated vLLM 0.29.0 and completes.
2. Runtime reports vLLM-Omni 0.29 with vLLM 0.29 and no major/minor mismatch warning.
3. A diffusion job imports compute_layout_strides and reaches model startup.
4. Any quarantined Stable Audio log is retained for [ROCm][CI] Validate Stable Audio layerwise-offload memory after vLLM 0.29 image alignment #7396 and evaluated separately from this image-fix gate.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	7 finding(s) below
Security	no finding reported
Docs / comments	1 finding(s) below
Behavior / compatibility	no finding reported
Correctness	no finding reported

Validated:

• [resolved] cwd-independent Path(file).resolve().parents[2] — quote tests/buildkite/test_rocm_dockerfile.py:10; residual: still no docker build execution in-test.
• [resolved] CUDA/XPU/NPU still on v0.28 (Dockerfile.cuda:1, Dockerfile.xpu:21, Dockerfile.npu:2) — author deferred; residual: other platforms can still drift vs Dockerfile.ci.
• [resolved] canary startswith(RUN python3 -c ...) — quote lines 35–37; residual: string presence ≠ image build/runtime.
• [resolved] tracks_cuda→tracks_ci naming: CI_DOCKERFILE→Dockerfile.ci — residual: Dockerfile.ci is CUDA CI image, not platform-neutral.
• [resolved] Canary startswith pin: test requires RUN line prefix — residual: proves Dockerfile text, not image-build behavior.
• [resolved] AMD exact-head runtime gate owned in discussion — residual: static tests ≠ image/runtime validation.

Primary mergeable findings center on the new ROCm↔CI alignment contract and fail-fast canary: keep tightening the BASE_IMAGE assert beyond tag-only equality (i=0; dups i=2/i=5), and extend the Dockerfile RUN canary to also import KVCacheLayout as used by layout.py (i=3). Keep the misleading fail-fast test rename as a nit (i=1; dup i=4), and keep gating the source-ref assert on USE_NIGHTLY_BUILD==0 as an optional nit (i=6). No blockers.

Verdict: COMMENT

Findings

• **[P2] test_rocm_base_tracks_ci_vllm_release only checks the tag: assert rocm_base.rs…** — tests/buildkite/test_rocm_dockerfile.pytest_rocm_base_tracks_ci_vllm_release only checks the tag:assert rocm_base.rsplit(":", 1)[-1] == ci_release(tests/buildkite/test_rocm_dockerfile.py:29). A wrong CUDA-family default likevllm/vllm-openai:v0.29.0would still pass because Dockerfile.ci’sVLLM_BASE_TAGis alsov0.29.0while its image isvllm/vllm-openai(docker/Dockerfile.ci:6-7). Also assert the ROCm image name (allowing a registry prefix), e.g.image_ref.endswith("vllm-openai-rocm")` plus tag equality, matching the default this PR sets at docker/Dockerfile.rocm:4 and that AMD builds without BASE_IMAGE override (.buildkite/amd/test-template-amd-omni.j2:77).

Evidence: tests/buildkite/test_rocm_dockerfile.py:29 assert rocm_base.rsplit(":", 1)[-1] == ci_release — tag only, no image-name check; docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; docker/Dockerfile.ci:6-7 ARG VLLM_BASE_IMAGE=vllm/vllm-openai / ARG VLLM_BASE_TAG=v0.29.0 (same tag, different family); unchanged by this diff, present in the PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 docker build -f docker/Dockerfile.rocm ... --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 — no BASE_IMAGE override.

Suggestion: image_ref, _, image_tag = rocm_base.rpartition(":")
assert image_tag == ci_release
assert image_ref.endswith("vllm-openai-rocm"), image_ref
assert rocm_source_ref == ci_release

• **[P2] tests/buildkite/test_rocm_dockerfile.py:29 — test_rocm_base_tracks_ci_vllm_rel…** — `` tests/buildkite/test_rocm_dockerfile.py:29 — test_rocm_base_tracks_ci_vllm_releaseonly assertsrocm_base.rsplit(":", 1)[-1] == ci_release(androcm_source_ref == ci_release) against Dockerfile.ci VLLM_BASE_TAG; it does not couple to CI image family/VLLM_BASE_IMAGE`. Residual coverage gap only (naming: CI_DOCKERFILE / tracks_ci at L11/L24).

Evidence: tests/buildkite/test_rocm_dockerfile.py:25-30 ci_release = _docker_arg(CI_DOCKERFILE, "VLLM_BASE_TAG") / rocm_base = _docker_arg(ROCM_DOCKERFILE, "BASE_IMAGE") / assert rocm_base.rsplit(":", 1)[-1] == ci_release / assert rocm_source_ref == ci_release — tag-only vs CI; unchanged by this diff, present in PR-time tree: docker/Dockerfile.ci:6-7 ARG VLLM_BASE_IMAGE=vllm/vllm-openai / ARG VLLM_BASE_TAG=v0.29.0 vs docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0 (different image family, shared tag).

• [P2] tests/buildkite/test_rocm_dockerfile.py:35–37 only checks that a Dockerfile lin… — ``
  tests/buildkite/test_rocm_dockerfile.py:35–37 only checks that a Dockerfile line startswith the canary RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides;. That is static string presence, not an image build or runtime execution of the fail-fast import (the real RUN lives in docker/Dockerfile.rocm:46).

Evidence: tests/buildkite/test_rocm_dockerfile.py:35 canary = 'RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides;'
tests/buildkite/test_rocm_dockerfile.py:37 assert any(line.startswith(canary) for line in dockerfile.splitlines()) — CPU test reads Dockerfile text only; does not build or run the image. Unchanged residual vs docker/Dockerfile.rocm:46 which does contain the real RUN python3 -c ... layer.

• [P2] Residual drift: only ROCm was bumped to v0.29.0 — Dockerfile.cuda:1
  Residual drift: only ROCm was bumped to v0.29.0; unchanged by this diff, CUDA/XPU/NPU still pin v0.28.0 while docker/Dockerfile.ci:7 has VLLM_BASE_TAG=v0.29.0 (docker/Dockerfile.cuda:1, docker/Dockerfile.xpu:21, docker/Dockerfile.npu:2).

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0; contrast docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0 (this PR only updates docker/Dockerfile.rocm to v0.29.0).

[andyluo7]

Follow-up self-review for exact head 5dc9929:

• The base-image contract now verifies both the vllm-openai-rocm image family and the CI release tag, while allowing a registry prefix.
• The active BASE_IMAGE check and opt-in nightly VLLM_VERSION_OR_COMMIT_HASH check are separate tests, so the default-runtime semantics are explicit.
• The Docker image-build canary now imports both compute_layout_strides and KVCacheLayout, matching the compatibility-sensitive imports used by diffusion layout code.
• The CPU test is renamed to test_rocm_dockerfile_contains_vllm_api_canary. It intentionally proves Dockerfile content only; actual import execution remains the exact-head AMD image-build gate.

Validation:

• focused tests from repository cwd: 3 passed
• focused tests from /tmp: 3 passed
• all changed-file pre-commit hooks: passed
• git diff --check: passed

@vllm-omni-review-bot please re-review exact head 5dc9929.

@yenuo26 @NickCao, this supersedes the prior exact-head request. Please run AMD merge validation against 5dc9929 and retain the same image-version, API-import, diffusion-startup, and #7396 Stable Audio evidence.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	7 finding(s) below
Security	no finding reported
Docs / comments	1 finding(s) below
Behavior / compatibility	no finding reported
Correctness	no finding reported

Validated:

• [resolved] cwd-independent REPO_ROOT via Path(file).resolve().parents[2] — quote tests/buildkite/test_rocm_dockerfile.py:10; residual: still no docker build execution in-test
• [resolved] CUDA/XPU/NPU still on v0.28 (Dockerfile.cuda:1, Dockerfile.xpu:21, Dockerfile.npu:2) — author deferred; residual: Dockerfile.cuda is also the release/docs build path (.buildkite/release/release-pipeline.yml:66,86)
• [resolved] AMD exact-head validation ask already owned in PR discussion; residual: gate_report clean ≠ proof of AMD Validated vLLM 0.29.0 log for 5dc9929
• [resolved] Stable Audio/#7396 disposition already tracked; thresholds/grades unchanged here
• [resolved] cwd-independent Path(file).resolve().parents[2] (test L10); residual: still no docker build execution in-test.
• [resolved] canary startswith(RUN python3 -c ...) (test L49); residual: order vs nightly block not asserted; USE_NIGHTLY=1 mis-order would still pass the static test.

Keep two concrete follow-ups on the PR’s new ROCm alignment contract: lock USE_NIGHTLY_BUILD=0 in the new regression file (i=0, minor), and fix the Dockerfile.rocm rebase REMINDER so it no longer invites custom ARG pins without updating the new hash lock (i=2, minor; i=1 and i=4 are dups). Also keep a nit rename so the nightly-named source-ref test matches what it actually asserts (i=3). No blockers; primary version bump and API canary are not contested.

Verdict: COMMENT

Findings

• [P2] This diff adds test_rocm_nightly_source_ref_tracks_ci_vllm_release asserting VL… — docker/Dockerfile.rocm
  This diff adds test_rocm_nightly_source_ref_tracks_ci_vllm_release asserting VLLM_VERSION_OR_COMMIT_HASH == Dockerfile.ci VLLM_BASE_TAG on every core_model/cpu run (tests/buildkite/test_rocm_dockerfile.py:34-38). The unchanged rebase REMINDER in docker/Dockerfile.rocm:10-11 still tells maintainers they may change those two ARGs for custom commits, which now fails that CPU gate unless the test is edited in the same change. Update the REMINDER to state the enforced alignment and that intentional commit pins must update the test alongside the ARG.

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.rocm:10-11 # 1. If vLLM-omni maintainer is forced to use custom commits / # during rebasing, they can change the two variables; added by this diff: tests/buildkite/test_rocm_dockerfile.py:36-38 rocm_source_ref = _docker_arg(ROCM_DOCKERFILE, "VLLM_VERSION_OR_COMMIT_HASH") then assert rocm_source_ref == ci_release (ci_release from Dockerfile.ci VLLM_BASE_TAG).

Suggestion: # NOTE: REMINDER to vLLM-Omni rebase maintainer

Keep BASE_IMAGE's tag and VLLM_VERSION_OR_COMMIT_HASH equal to

docker/Dockerfile.ci's VLLM_BASE_TAG (enforced by

tests/buildkite/test_rocm_dockerfile.py). An intentional commit pin

must update that test in the same change.

Remember to set USE_NIGHTLY_BUILD to 0 when switching back to the

stable vLLM docker image.

• [P2] CUDA/XPU/NPU remain on v0.28 while CI/ROCm moved to v0.29 (Dockerfile.cuda:1, D… — Dockerfile.cuda:1
  CUDA/XPU/NPU remain on v0.28 while CI/ROCm moved to v0.29 (Dockerfile.cuda:1, Dockerfile.xpu:21, Dockerfile.npu:2). Residual risk: Dockerfile.cuda is the release/docs image path (.buildkite/release/release-pipeline.yml:66,86).

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0; contrast PR-time docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0 and docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; release consumer unchanged by this diff: .buildkite/release/release-pipeline.yml:66 -f docker/Dockerfile.cuda . and :86 -f docker/Dockerfile.cuda .

• [P2] Residual only: cpu test test_rocm_dockerfile_contains_vllm_api_canary proves… — ``
  Residual only: cpu test test_rocm_dockerfile_contains_vllm_api_canary proves canary text exists, not that an AMD image build for 5dc9929 printed `Validated vLLM 0.29.0`. gate_report clean ≠ that log.

Evidence: tests/buildkite/test_rocm_dockerfile.py:8 pytestmark = [pytest.mark.core_model, pytest.mark.cpu] — canary test is CPU static; tests/buildkite/test_rocm_dockerfile.py:49 assert any(line.startswith(canary) for line in dockerfile.splitlines()) — asserts Dockerfile text only; docker/Dockerfile.rocm:46 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')" — log only emitted during ROCm image construction, absent from PR-time tree as an AMD build artifact for 5dc9929.

• **[P2] tests/buildkite/test_rocm_dockerfile.py:49 only checks canary presence via any…** — `` tests/buildkite/test_rocm_dockerfile.py:49 only checks canary presence via any(line.startswith(canary)...)`; it does not assert the canary RUN comes after the USE_NIGHTLY_BUILD reinstall block (docker/Dockerfile.rocm:32–46). A canary placed before that block would still pass this static test while leaving USE_NIGHTLY=1 builds unchecked after reinstall.

Evidence: tests/buildkite/test_rocm_dockerfile.py:49 assert any(line.startswith(canary) for line in dockerfile.splitlines()) — presence-only, no order vs nightly. Unchanged ordering fact in PR-time tree: docker/Dockerfile.rocm:32 RUN if [ "${USE_NIGHTLY_BUILD}" = "1" ]; then \ then docker/Dockerfile.rocm:46 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')" — canary after nightly today, but test does not lock that order.

[andyluo7]

Final follow-up for exact head 177ac10:

• Added a regression locking USE_NIGHTLY_BUILD=0 so normal AMD builds continue using the published ROCm base image.
• The canary test now proves that the optional nightly reinstall block closes before the API canary RUN layer.
• Renamed the source-ref test to describe the value it actually checks.
• Updated the Dockerfile rebase reminder to document the enforced alignment and intentional custom-pin procedure.
• Updated the PR body to the current 4-test result.

The API canary remains a blocking amd-build step intentionally: the shared AMD image must support the repository diffusion path, and failing once during image construction prevents misleading partial greens and repeated downstream runtime failures. The CPU regression still claims only Dockerfile structure; the exact-head AMD build remains execution proof.

Validation:

• focused tests from repository cwd: 4 passed
• focused tests from /tmp: 4 passed
• all changed-file pre-commit hooks: passed
• compileall: passed
• git diff --check: passed

@vllm-omni-review-bot please re-review exact head 177ac10.

@yenuo26 @NickCao, this supersedes earlier SHA-specific requests. Please run AMD merge validation against exact head 177ac10 and retain the image-version, API-import, diffusion-startup, and #7396 Stable Audio evidence.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	6 finding(s) below
Security	no finding reported
Docs / comments	2 finding(s) below
Behavior / compatibility	no finding reported
Correctness	no finding reported

Validated:

• [resolved] CUDA/XPU/NPU still on v0.28 (Dockerfile.cuda:1, Dockerfile.xpu:21, Dockerfile.npu:2) — author deferred; residual: those platforms can still drift vs Dockerfile.ci.
• [resolved] cwd-independent REPO_ROOT Path(file).resolve().parents[2] — quote test_rocm_dockerfile.py:10; residual: no docker build in-test
• [resolved] canary startswith RUN python3 -c … — quote test:55-60; residual: string presence ≠ import execution
• [resolved] USE_NIGHTLY_BUILD=0 lock — quote test:40-41 + Dockerfile:14; residual: CI --build-arg override not locked
• [resolved] USE_NIGHTLY_BUILD=0 locked by test_rocm_defaults_to_prebuilt_base_image — residual: nightly=1 path still unexecuted in CI defaults.
• [resolved] cwd-independent REPO_ROOT via Path(file).resolve().parents[2] — residual: tests assert Dockerfile text only, not a real docker build.

Kept two minors on the primary v0.29 ROCm rebase: docs/users inherit the new BASE_IMAGE default without a CUDA-style pin (blast-radius), and TorchCodec’s pinned branch still needs amd-build confirmation after the base bump (adversary+behavior). Dropped speculative amd-template test locks and a rename-only nit; collapsed the duplicate TorchCodec asks into one.

Verdict: COMMENT

Findings

• [P2] Residual: CUDA/XPU/NPU Dockerfiles still pin v0.28.0 while Dockerfile.ci is on… — Dockerfile.cuda:1
  Residual: CUDA/XPU/NPU Dockerfiles still pin v0.28.0 while Dockerfile.ci is on v0.29.0 (Dockerfile.cuda:1 vllm/vllm-openai:v0.28.0, Dockerfile.xpu:21 VLLM_VERSION=v0.28.0, Dockerfile.npu:2 VLLM_ASCEND_VERSION=v0.28.0). This PR only aligns ROCm + tests/buildkite/test_rocm_dockerfile.py; those platforms can still drift vs CI.

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0; docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0 (CI pin this PR aligns ROCm to).

• **[P2] test_rocm_dockerfile_contains_vllm_api_canary only checks Dockerfile text: _li…** — `` test_rocm_dockerfile_contains_vllm_api_canary only checks Dockerfile text: _line_indexmatches a line startswith theRUN python3 -c "import vllm; …prefix and that it follows the nightlyfi`. That is string presence/order, not import execution (execution happens only if the image is built).

Evidence: tests/buildkite/test_rocm_dockerfile.py:24-27 _line_index uses line.startswith(prefix); tests/buildkite/test_rocm_dockerfile.py:55-60 canary = ('RUN python3 -c "import vllm; ' …); canary_index = _line_index(lines, canary) — no import/runtime of that snippet. Unchanged-by-claim but present in PR-time tree: docker/Dockerfile.rocm:45 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')" — build-time execution is outside this CPU test.

• **[P2] Add a consumer lock in tests/buildkite/test_rocm_dockerfile.py (mirroring te…** — .buildkite/amd/test-template-amd-omni.j2:77Add a consumer lock intests/buildkite/test_rocm_dockerfile.py(mirroringtests/buildkite/test_amd_pipeline.py's live pipeline-command assertions) so amd-build at .buildkite/amd/test-template-amd-omni.j2:77stays pinned to-f docker/Dockerfile.rocmwith no--build-arg BASE_IMAGE=` override.

Evidence: unchanged by this diff, present in the PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 - "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ." (key amd-build at :79). PR-added tests/buildkite/test_rocm_dockerfile.py only asserts Dockerfile ARG tags/canary (e.g. :30-37 test_rocm_base_tracks_ci_vllm_release / _docker_arg(..., "BASE_IMAGE")) and never reads the AMD template or amd-build command. Sibling contrast, unchanged by this diff: tests/buildkite/test_amd_pipeline.py:31-42 locks live Buildkite step commands (test_qwen3_tts_base_preserves_advanced_model_arguments).

• [P2] Update docs/getting_started/installation/gpu/rocm.inc.md:135 (and :154) sti… — docs/getting_started/installation/gpu/rocm.inc.md:135
  Update docs/getting_started/installation/gpu/rocm.inc.md:135 (and :154) still showing prebuilt vllm/vllm-omni-rocm:v0.28.0, or track that published-omni tag as an explicit follow-up outside this PR's stated CI-only scope.

Evidence: unchanged by this diff, present in the PR-time tree: docs/getting_started/installation/gpu/rocm.inc.md:135 vllm/vllm-omni-rocm:v0.28.0 \ — also docs/getting_started/installation/gpu/rocm.inc.md:154 vllm/vllm-omni-rocm:v0.28.0 (published omni image tag, not the CI vllm/vllm-openai-rocm base bumped in docker/Dockerfile.rocm).

• **[P2] This diff sets docker/Dockerfile.rocm ARG BASE_IMAGE=vllm/vllm-openai-rocm:v…** — docs/getting_started/installation/gpu/rocm.inc.mdThis diff setsdocker/Dockerfile.rocm ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0, so the bare user build in docs/getting_started/installation/gpu/rocm.inc.md (DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.rocm ...) inherits that default with no version callout, while docs/getting_started/installation/gpu/cuda.inc.mdalready shows an optional--build-arg BASE_IMAGE=...:v0.29.0`. Widen the PR scope claim beyond “ROCm CI image only” and mirror that explicit pin (or a one-line note that the default base is now v0.29.0).

Evidence: Changed by this diff: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0. Unchanged by this diff, present in the PR-time tree: docs/getting_started/installation/gpu/rocm.inc.md:75 DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.rocm -t vllm-omni-rocm .; docs/getting_started/installation/gpu/cuda.inc.md:131-136 If you want to specify the base vLLM version: / DOCKER_BUILDKIT=1 docker build \ / -f docker/Dockerfile.cuda \ / --build-arg BASE_IMAGE=vllm/vllm-openai:v0.29.0 \

Suggestion: DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.rocm -t vllm-omni-rocm .

Optional: pin the base explicitly (should match docker/Dockerfile.ci VLLM_BASE_TAG)

DOCKER_BUILDKIT=1 docker build
-f docker/Dockerfile.rocm
--build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0
-t vllm-omni-rocm .

[vllm-omni-review-bot]

[P2] This diff rebases docker/Dockerfile.rocm ARG BASE_IMAGE/`VLLM_VERSION_OR_CO…

This diff rebases docker/Dockerfile.rocm ARG BASE_IMAGE/VLLM_VERSION_OR_COMMIT_HASH to v0.29.0 and adds an import-only vLLM API canary before the unchanged TorchCodec layer that still runs tools/install_torchcodec_rocm.sh (TORCHCODEC_BRANCH=v0.10.0), whose comment requires revalidation on ROCm base/vLLM rebases. PR validation covers static ARG tests and the canary, not TorchCodec/ABI — please confirm the AMD image-build log for this head (e.g. post-177ac10) shows the TorchCodec/VideoDecoder step succeeding against the v0.29.0 base, or update that pin if it does not.

Evidence: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; docker/Dockerfile.rocm:15 ARG VLLM_VERSION_OR_COMMIT_HASH=v0.29.0; docker/Dockerfile.rocm:45 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')"; docker/Dockerfile.rocm:51-59 still COPY tools/install_torchcodec_rocm.sh then bash /tmp/install_torchcodec_rocm.sh after the canary; unchanged by this diff, present in the PR-time tree: tools/install_torchcodec_rocm.sh:11-13 # Keep this pinned for ROCm/PyTorch ABI compatibility. When rebasing the ROCm / base image or vLLM, revalidate this version against the active torch build. / TORCHCODEC_BRANCH="${TORCHCODEC_BRANCH:-v0.10.0}"; tests/buildkite/test_rocm_dockerfile.py has no TorchCodec assertions (grep torchcodec: no matches).

[andyluo7]

This remains an exact-head AMD evidence gate rather than a speculative pin change. The requested build for 2e850d4 must show tools/install_torchcodec_rocm.sh completing and the final VideoDecoder import succeeding against the v0.29.0 base. I am leaving this thread open until that artifact exists; if the step fails, the TorchCodec pin will be adjusted from the captured failure.

[andyluo7]

Updated for exact head f8af7e2d146cfa78629c36bd9ffd463f4f518b39. This remains the canonical open runtime-evidence thread: the AMD image log must show tools/install_torchcodec_rocm.sh completing and the final from torchcodec.decoders import VideoDecoder import succeeding against the vLLM 0.29 base. If it fails, the pin will be changed from the captured failure rather than speculatively.

[andyluo7]

Runtime evidence is now available from AMD Buildkite #11682 on commit f8af7e2: the image used vllm/vllm-openai-rocm:v0.29.0, printed Validated vLLM 0.29.0, completed the TorchCodec v0.10.0 build and installation, and completed Docker step 16 containing the final VideoDecoder import. A Z-Image diffusion worker also initialized and began loading model weights. Commit 15d3c10 changes only documentation and Dockerfile comments; I am leaving this canonical thread open until the final exact-head AMD rerun confirms the new head.

[andyluo7]

Follow-up for exact head 2e850d4:

• Added a consumer lock for the live amd-build template: it must use docker/Dockerfile.rocm and must not override BASE_IMAGE or USE_NIGHTLY_BUILD.
• Updated the PR body to the current 5-test result and clarified that the Dockerfile default also affects user source builds.
• Tracked the separate published-image documentation/release gap in [Docs][ROCm] Fix unavailable prebuilt image tags in installation guide #7405. Docker Hub currently has neither the documented vllm/vllm-omni-rocm:v0.28.0 tag nor a v0.29.0 replacement, so this PR does not substitute an unverified tag.
• Kept the API canary import-only because it targets the demonstrated symbol-availability incompatibility; functional behavior remains covered by the AMD diffusion runtime gate.

Validation:

• focused tests from repository cwd: 5 passed
• focused tests from /tmp: 5 passed
• changed-file pre-commit hooks: passed
• git diff --check: passed

The TorchCodec review thread remains intentionally open pending exact-head hardware evidence. The AMD image log must show install_torchcodec_rocm.sh and the final VideoDecoder import succeeding before that thread is closed.

@vllm-omni-review-bot please re-review exact head 2e850d4.

@yenuo26 @NickCao, this supersedes earlier SHA-specific requests. Please run AMD merge validation against exact head 2e850d4 and retain the vLLM version/API, TorchCodec/VideoDecoder, diffusion-startup, and #7396 Stable Audio evidence.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	5 finding(s) below
Security	no finding reported
Docs / comments	3 finding(s) below
Behavior / compatibility	no finding reported
Correctness	no finding reported

Validated:

• [resolved] ReviewBot cwd / CI naming / canary startswith / USE_NIGHTLY=0 / template default locks addressed at head 2e850d4; residual: no in-test docker build execution.
• [resolved] CUDA/XPU/NPU 0.28 drift vs Dockerfile.ci deferred by author; residual siblings still unaligned
• [claim-verified] Early canary imports compute_layout_strides and KVCacheLayout after the nightly block (Dockerfile.rocm:31-45), matching vllm_omni/diffusion/diffusion_kv/layout.py:23-24 consumers.
• [claim-verified] CUDA/XPU/NPU pins unchanged at head: Dockerfile.cuda:1 v0.28.0, Dockerfile.xpu VLLM_VERSION=v0.28.0, Dockerfile.npu VLLM_ASCEND_VERSION=v0.28.0 (author-deferred).
• [validated] requirements/setup do not install/replace vllm; canary-before-omni-install cannot be invalidated by uv pip install ".[dev]" (requirements/common.txt, rocm.txt; setup.py get_install_requires)
• [validated] pytestmark core_model+cpu present (test_rocm_dockerfile.py:8); collected by AMD/CUDA catch-all tests/ -m 'core_model and cpu' (.buildkite/amd/test-amd-merge.yml:52, .buildkite/cuda/test-ready.yml:34).

Keep three items centered on the ROCm 0.29 base alignment: TorchCodec pin/log confirmation after the ABI rebase (minor), tighter --build-arg matching in the new AMD template lock (nit), and ROCm install-doc note/override for the new default (minor). Drop the uncertain amd-build canary log ask as already covered by PR acceptance criteria, and drop the test-file relocation nit as intentional co-location with the Dockerfile-enforced module.

Verdict: COMMENT

Findings

• [P2] cwd-independent REPO_ROOT via Path(file).resolve().parents[2] (tests/buildk… — ``
  cwd-independent REPO_ROOT via Path(file).resolve().parents[2] (tests/buildkite/test_rocm_dockerfile.py:10); residual: siblings like test_amd_pipeline.py still use cwd-relative Path — out of this PR scope.

Evidence: tests/buildkite/test_rocm_dockerfile.py:10 REPO_ROOT = Path(__file__).resolve().parents[2] — cwd-independent. Unchanged by this diff, present in the PR-time tree: tests/buildkite/test_amd_pipeline.py:12 AMD_MERGE_PIPELINE = Path(".buildkite/amd/test-amd-merge.yml") — cwd-relative sibling.

• [P2] Prior ReviewBot locks are present at this head (REPO_ROOT via parents[2], VLLM_… — ``
  Prior ReviewBot locks are present at this head (REPO_ROOT via parents[2], VLLM_BASE_TAG tag alignment, canary startswith, USE_NIGHTLY_BUILD=0, AMD template omitting BASE_IMAGE/USE_NIGHTLY overrides). Residual: tests/buildkite/test_rocm_dockerfile.py only read_text-parses Dockerfile/template strings and never executes docker build.

Evidence: tests/buildkite/test_rocm_dockerfile.py:46-53 template = AMD_TEMPLATE.read_text(encoding="utf-8") / build_commands = [line.strip() for line in template.splitlines() if '"docker build ' in line] / assert "BASE_IMAGE" not in build_command / assert "USE_NIGHTLY_BUILD" not in build_command — inspects the build command string only; grep of this file finds no subprocess/docker/os.system/run invocation. Unchanged-by-claim but supporting locks in the same file: :10 REPO_ROOT = Path(__file__).resolve().parents[2]; :42 assert _docker_arg(ROCM_DOCKERFILE, "USE_NIGHTLY_BUILD") == "0"; :72 _line_index(lines, canary) (startswith). Unchanged by this residual, present in PR-time tree: docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0; .buildkite/amd/test-template-amd-omni.j2:77 docker build uses Dockerfile.rocm defaults.

• [P2] REPO_ROOT is cwd-independent via Path(file).resolve().parents[2] (tests/bui… — test_rocm_dockerfile.py:10
  REPO_ROOT is cwd-independent via Path(file).resolve().parents[2] (tests/buildkite/test_rocm_dockerfile.py:10). Residual: these tests only statically parse Dockerfile/AMD-template text and never execute docker build.

Evidence: tests/buildkite/test_rocm_dockerfile.py:10 REPO_ROOT = Path(__file__).resolve().parents[2]; tests/buildkite/test_rocm_dockerfile.py:47 build_commands = [line.strip() for line in template.splitlines() if '"docker build ' in line] — template string filter only; no subprocess/docker build invocation in this file

• **[P2] Canary is on the shared ROCm image (AMD suite + docker build -f docker/Dockerf…** — Dockerfile.rocm:45Canary is on the shared ROCm image (AMD suite +docker build -f docker/Dockerfile.rocm` source builds) and fails loud by design — see docker/Dockerfile.rocm:43-45. Residual: that image is not AR-split, so AR rebuilds still run the diffusion KV import canary.

Evidence: docker/Dockerfile.rocm:43-45 # Fail during image construction instead of letting every AMD test fail at / # runtime when the base image and vLLM-Omni target incompatible vLLM APIs. / RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')" — fail-loud canary on the Dockerfile.rocm base stage. Unchanged by this diff beyond the new canary itself, present in the PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ." — single shared AMD image build. Unchanged by this diff, present in the PR-time tree: vllm_omni/diffusion/diffusion_kv/layout.py:23-24 from vllm.v1.kv_cache_interface import KVCacheSpec, KVCacheTensor, compute_layout_strides / from vllm.v1.kv_cache_layout import KVCacheLayout — canary symbols are diffusion-KV consumers, so AR-shared rebuilds still pay that import tax.

• [P2] Land or track sibling pin bumps on Dockerfile.cuda:1, Dockerfile.xpu:21, an… — docker/Dockerfile.cuda:1
  Land or track sibling pin bumps on Dockerfile.cuda:1, Dockerfile.xpu:21, and Dockerfile.npu:2 (still v0.28.0) now that Dockerfile.ci and Dockerfile.rocm are on v0.29.0 — residual cross-platform drift should not stay silent.

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0. Contrast with pinned tree: docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0; docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0.

• **[P2] This PR flips docker/Dockerfile.rocm default BASE_IMAGE to vllm/vllm-opena…** — docs/getting_started/installation/gpu/rocm.inc.mdThis PR flipsdocker/Dockerfile.rocmdefaultBASE_IMAGEtovllm/vllm-openai-rocm:v0.29.0and adds an import canary that fails older bases, so the documented bare build indocs/getting_started/installation/gpu/rocm.inc.md(line 75) now silently produces a 0.29 stack while the adjacent pre-built section still showsvllm/vllm-omni-rocm:v0.28.0. CUDA’s sibling already documents --build-arg BASE_IMAGE=…:v0.29.0; add a one-line note that the source build defaults to v0.29.0 (Hub retag remains #7405) plus a CUDA-style optional --build-arg BASE_IMAGE=` example so rebase/debug overrides of older ROCm bases fail with the explained canary rather than a surprise.

Evidence: unchanged by this diff, present in the PR-time tree: rocm.inc.md:75 DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.rocm -t vllm-omni-rocm . — bare build, no BASE_IMAGE note; Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; Dockerfile.rocm:45 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')"; unchanged by this diff: rocm.inc.md:135 vllm/vllm-omni-rocm:v0.28.0 \; unchanged by this diff: cuda.inc.md:136 --build-arg BASE_IMAGE=vllm/vllm-openai:v0.29.0 \

Suggestion: #### Build docker image

The default base is vllm/vllm-openai-rocm:v0.29.0 (aligned with docker/Dockerfile.ci). Older bases fail the image-build API canary. Published vllm/vllm-omni-rocm Hub tags remain on 0.28 until #7405.

DOCKER_BUILDKIT=1 docker build -f docker/Dockerfile.rocm -t vllm-omni-rocm .

If you want to specify the base vLLM version:

DOCKER_BUILDKIT=1 docker build \
  -f docker/Dockerfile.rocm \
  --build-arg BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0 \
  -t vllm-omni-rocm .

• [P3] test_amd_build_uses_rocm_dockerfile_defaults (lines 52–53) locks AMD builds to… — tests/buildkite/test_rocm_dockerfile.py
  test_amd_build_uses_rocm_dockerfile_defaults (lines 52–53) locks AMD builds to Dockerfile.rocm defaults by asserting raw substrings BASE_IMAGE and USE_NIGHTLY_BUILD are absent from the build line. Prefer assert "--build-arg BASE_IMAGE" not in build_command and assert "--build-arg USE_NIGHTLY_BUILD" not in build_command so a future image/tag/path that merely contains those tokens cannot false-fail the lock.

Evidence: tests/buildkite/test_rocm_dockerfile.py:52-53 assert "BASE_IMAGE" not in build_command / assert "USE_NIGHTLY_BUILD" not in build_command — bare token lock. Unchanged by this diff, present in the PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 - "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ." — no --build-arg BASE_IMAGE / --build-arg USE_NIGHTLY_BUILD; intended forbidden overrides are those --build-arg forms.

Suggestion: assert "--build-arg BASE_IMAGE" not in build_command
assert "--build-arg USE_NIGHTLY_BUILD" not in build_command

[hsliuustc0106]

Findings from a frozen snapshot at 2e850d4 (base main @ 5a386d78). Read-only review — no code from the head was executed.

[P2] TorchCodec ABI pin is not revalidated on the base-image rebase — docker/Dockerfile.rocm:47-65, tools/install_torchcodec_rocm.sh:11-13

This diff moves the ROCm base from v0.28.0 to v0.29.0, which changes the torch/ROCm ABI under the TorchCodec layer, but that layer and its pin are unchanged. The script states its own contract:

# Keep this pinned for ROCm/PyTorch ABI compatibility. When rebasing the ROCm base image or vLLM, revalidate this version against the active torch build. — TORCHCODEC_BRANCH="${TORCHCODEC_BRANCH:-v0.10.0}"

This PR is exactly the "rebasing the ROCm base image" trigger, and no revalidation evidence is attached. Smallest fix: attach the amd-build result for 2e850d4 showing tools/install_torchcodec_rocm.sh completing and the final from torchcodec.decoders import VideoDecoder import succeeding against the v0.29.0 base — or bump TORCHCODEC_BRANCH from the captured failure.

[P3] The alignment contract is enforced for ROCm only, and binds a user-facing image to a CI-only image's tag — tests/buildkite/test_rocm_dockerfile.py:24-38, docker/Dockerfile.cuda:1

test_rocm_base_tracks_ci_vllm_release pins Dockerfile.rocm's BASE_IMAGE to docker/Dockerfile.ci's VLLM_BASE_TAG, and the new comment codifies that as the invariant. Two consequences need an explicit decision:

1. Dockerfile.rocm is user-facing (per the PR body: users building it without an override), yet it is now bound to a CI-only image's tag. If Dockerfile.ci moves to an unreleased tag for CI reasons, this test forces the user-facing ROCm default to follow.
2. The sibling user-facing image docker/Dockerfile.cuda:1 still defaults to vllm/vllm-openai:v0.28.0 while Dockerfile.ci is v0.29.0. Since vllm_omni/diffusion/diffusion_kv/layout.py:23-24 imports compute_layout_strides / KVCacheLayout (v0.29 APIs), the same import failure fixed here for AMD remains reachable for anyone building Dockerfile.cuda.

This is pre-existing rather than introduced here, so treating it as a scope question: bump Dockerfile.cuda in this change, or state in the PR that CUDA is tracked separately and open a follow-up. If the intent is "the Dockerfile.ci tag is the project-wide vLLM release of record," say so and the coupling is fine.

Verified sound (checked, no issue)

• The canary targets the real APIs. docker/Dockerfile.rocm:45 imports exactly what production imports — vllm_omni/diffusion/diffusion_kv/layout.py:23 (compute_layout_strides from vllm.v1.kv_cache_interface) and :24 (KVCacheLayout from vllm.v1.kv_cache_layout). Not a guessed path.
• The new tests do run in CI. tests/buildkite/ is not in the ignore list of the CPU sweep — .buildkite/cuda/test-ready.yml:34 and the AMD Simple · Other Test step both run pytest -sv tests/ -m 'core_model and cpu' --ignore={diffusion,model_executor,entrypoints,engine}, so test_rocm_dockerfile.py is collected on every PR.
• Markers are legal. core_model and cpu are registered in pyproject.toml:250,266, satisfying --strict-markers (pyproject.toml:244).
• Every test assumption holds at the frozen head. Dockerfile.ci declares ARG VLLM_BASE_TAG= exactly once (:7, v0.29.0); ROCm has BASE_IMAGE=…:v0.29.0, USE_NIGHTLY_BUILD=0, VLLM_VERSION_OR_COMMIT_HASH=v0.29.0; the AMD template has exactly one docker build line, it uses -f docker/Dockerfile.rocm, and passes no BASE_IMAGE / USE_NIGHTLY_BUILD override (.buildkite/amd/test-template-amd-omni.j2:77).
• The AMD build is a real pre-merge gate. The template defines key: "amd-build" with soft_fail: false (:73-93), and every AMD test step carries depends_on: amd-build.
• REPO_ROOT is cwd-independent (parents[2]), and .buildkite/ is not excluded by Dockerfile.ci.dockerignore (shipped via COPY --link . .), so the file the tests read exists in CI.

Validation gaps

• The PR's own stated gate has not reported. No amd-build check-run or status exists on 2e850d4. I could not independently confirm that vllm/vllm-openai-rocm:v0.29.0 is published (Docker Hub was unreachable from my environment), so the canary's actual import success is unproven until the AMD build runs. A skipped hardware gate is a gap, not a pass.
• The local command quoted in the PR body uses -o addopts='', which disables --strict-markers. Harmless here since the markers are registered, but it means that run did not exercise the repo's real addopts.

[andyluo7]

Follow-up for exact head f8af7e2d146cfa78629c36bd9ffd463f4f518b39:

• Tightened the AMD template lock to reject the actual override forms --build-arg BASE_IMAGE / --build-arg=BASE_IMAGE and --build-arg USE_NIGHTLY_BUILD / --build-arg=USE_NIGHTLY_BUILD, without false-failing on unrelated text.
• Updated the ROCm source-build documentation to identify vllm/vllm-openai-rocm:v0.29.0 as the default, explain the image-build API canary, distinguish it from prebuilt vllm/vllm-omni-rocm images, and show an explicit BASE_IMAGE override.
• Opened [CI] Reconcile CUDA, XPU, and NPU vLLM image pins after 0.29 transition #7410 to track the separate CUDA/XPU/NPU vLLM-version contracts and remaining 0.28 defaults. The published ROCm Omni-image/tag documentation gap remains [Docs][ROCm] Fix unavailable prebuilt image tags in installation guide #7405.
• Consolidated the duplicate TorchCodec findings. The canonical thread remains open until an exact-head AMD image log proves install_torchcodec_rocm.sh and the final VideoDecoder import succeed.

Validation:

• focused test from repository cwd with --strict-markers: 5 passed
• focused test from /tmp with --strict-markers: 5 passed
• all changed-file pre-commit hooks, including Markdown lint: passed
• compileall and git diff --check: passed

@vllm-omni-review-bot please re-review exact head f8af7e2d146cfa78629c36bd9ffd463f4f518b39.

@yenuo26 @hsliuustc0106 @NickCao, please run one AMD merge validation against this exact head. Required evidence is:

1. the blocking amd-build prints Validated vLLM 0.29.0 and completes;
2. tools/install_torchcodec_rocm.sh and the final VideoDecoder import succeed;
3. runtime reports vLLM-Omni 0.29 with vLLM 0.29 and no major/minor mismatch warning;
4. a diffusion job reaches model startup;
5. any quarantined Stable Audio log is retained for [ROCm][CI] Validate Stable Audio layerwise-offload memory after vLLM 0.29 image alignment #7396 and evaluated separately from this image-fix gate.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	6 finding(s) below
Security	no finding reported
Docs / comments	4 finding(s) below
Behavior / compatibility	no finding reported
Correctness	no finding reported

Validated:

• [resolved] CUDA/XPU/NPU still on v0.28 (Dockerfile.cuda:1, Dockerfile.xpu:21, Dockerfile.npu:2) — deferred to #7410; residual: those platforms can still drift vs Dockerfile.ci.
• [resolved] Prior ReviewBot cwd/CI naming/canary startswith/USE_NIGHTLY=0/template override locks present at head f8af7e2.
• [resolved] USE_NIGHTLY_BUILD=0 lock + template no-override locks — test:41-54; residual: rendered minijinja output not re-checked beyond j2 source
• [resolved] cwd-independent REPO_ROOT / CI naming / canary startswith / USE_NIGHTLY=0 / template locks — present at head; residual: tests are string/content locks, not docker execution.
• [claim-verified] Canary RUN imports compute_layout_strides and KVCacheLayout after the nightly if/fi (docker/Dockerfile.rocm:31-45), matching vllm_omni/diffusion/diffusion_kv/layout.py:23-24.
• [claim-verified] requirements/*.txt and setup.py do not declare vllm, so later uv pip install ".[dev]" does not replace the base-image vLLM the canary checked.

Keep all six with centrality on the ROCm 0.29 BASE_IMAGE flip: fleet-wide inheritance plus exact-head AMD runtime proof (4), canary order locked before TorchCodec not only after nightly fi (0), and docs for coexisting 0.28/0.29 plus #7405 on prebuilt examples (1, 5). Demote the canary version-prefix assert to nit as optional beyond the import canary and static tag lock (2); keep the misleading nightly-default test rename as nit (3). No drops or dups.

Verdict: COMMENT

Findings

• [P2] CUDA/XPU/NPU images remain on v0.28 while CI/ROCm are on v0.29 (docker/Dockerfi… — Dockerfile.cuda:1
  CUDA/XPU/NPU images remain on v0.28 while CI/ROCm are on v0.29 (docker/Dockerfile.cuda:1, docker/Dockerfile.xpu:21, docker/Dockerfile.npu:2). Unchanged by this diff; deferred to #7410. Residual: those platforms can still drift vs docker/Dockerfile.ci because tests/buildkite/test_rocm_dockerfile.py only gates ROCm.

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0; contrast docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0 and (in this diff) docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0

• [P2] cwd-independent REPO_ROOT via Path(file).resolve().parents[2] at tests/buil… — test_rocm_dockerfile.py:10
  cwd-independent REPO_ROOT via Path(file).resolve().parents[2] at tests/buildkite/test_rocm_dockerfile.py:10; residual: tests only statically parse Dockerfile/AMD template text and never execute docker build.

Evidence: tests/buildkite/test_rocm_dockerfile.py:10 REPO_ROOT = Path(__file__).resolve().parents[2] — cwd-independent root. tests/buildkite/test_rocm_dockerfile.py:45-54 test_amd_build_uses_rocm_dockerfile_defaults only does AMD_TEMPLATE.read_text(...) and filters lines with "docker build " then asserts substrings; no subprocess/docker invocation anywhere in the file.

• [P2] Published vllm/vllm-omni-rocm:v0.28.0 examples are retained (rocm.inc.md:160,… — rocm.inc.md:84
  Published vllm/vllm-omni-rocm:v0.28.0 examples are retained (rocm.inc.md:160,181) with an explicit #7405 published-tag-availability callout (rocm.inc.md:84-86); residual: no v0.29 omni-rocm published tag in these examples.

Evidence: docs/getting_started/installation/gpu/rocm.inc.md:84-86 image-build canary. This upstream base is distinct from the prebuilt / ``vllm/vllm-omni-rocm images discussed below; their published-tag availability / `is tracked in #7405.`; docs/getting_started/installation/gpu/rocm.inc.md:160 ` vllm/vllm-omni-rocm:v0.28.0 `; also line 181 ` vllm/vllm-omni-rocm:v0.28.0`

• [P2] Locks hold in tests/buildkite/test_rocm_dockerfile.py:41-54 (USE_NIGHTLY_BUILD=… — ``
  Locks hold in tests/buildkite/test_rocm_dockerfile.py:41-54 (USE_NIGHTLY_BUILD=0 + AMD j2 build command must not pass --build-arg BASE_IMAGE/USE_NIGHTLY_BUILD). Residual: that test only reads .buildkite/amd/test-template-amd-omni.j2 source; live AMD CI renders via minijinja-cli in bootstrap-amd-omni.sh and that rendered output is not re-checked.

Evidence: tests/buildkite/test_rocm_dockerfile.py:41-42 def test_rocm_defaults_to_prebuilt_base_image() -> None: / assert _docker_arg(ROCM_DOCKERFILE, "USE_NIGHTLY_BUILD") == "0"; tests/buildkite/test_rocm_dockerfile.py:46-54 template = AMD_TEMPLATE.read_text(encoding="utf-8") … for arg_name in ("BASE_IMAGE", "USE_NIGHTLY_BUILD"): / assert f"--build-arg {arg_name}" not in build_command; unchanged by this diff, present in PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ."; unchanged by this diff, present in PR-time tree: .buildkite/amd/scripts/bootstrap-amd-omni.sh:186 minijinja-cli test-template.j2 "$TEST_YAML" \ — render path exists but no test asserts rendered output for these locks

• [P2] This PR adds the fail-fast canary RUN immediately before the TorchCodec build l… — tests/buildkite/test_rocm_dockerfile.py
  This PR adds the fail-fast canary RUN immediately before the TorchCodec build layer (COPY tools/install_torchcodec_rocm.sh), but test_rocm_dockerfile_contains_vllm_api_canary only requires an intervening nightly fi before canary_index (tests/buildkite/test_rocm_dockerfile.py:75-77). Extend the assertion so canary_index is also strictly before that TorchCodec COPY line; otherwise the canary can drift below the expensive layer (or past COPY .) and the regression still passes while the early-fail claim is lost.

Evidence: tests/buildkite/test_rocm_dockerfile.py:75-77 assert any(line.strip() == "fi" for line in lines[nightly_start + 1 : canary_index]), ( — only lower-bound after nightly; no upper bound vs TorchCodec. docker/Dockerfile.rocm:45 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')" then docker/Dockerfile.rocm:51 COPY tools/install_torchcodec_rocm.sh /tmp/install_torchcodec_rocm.sh.

Suggestion: assert any(line.strip() == "fi" for line in lines[nightly_start + 1 : canary_index]), (
"vLLM API canary must follow the optional nightly source reinstall block"
)
torchcodec_copy = _line_index(lines, "COPY tools/install_torchcodec_rocm.sh")
assert canary_index < torchcodec_copy, (
"vLLM API canary must run before the TorchCodec build layer"
)

[andyluo7]

Follow-up for exact head 15d3c106799d3a90db64df8bdf8a51f961094dec:

• Added the ROCm major/minor vLLM compatibility contract before the installation paths.
• Clarified that BASE_IMAGE governs Docker builds and that non-Docker source installs must install the matching platform-specific vLLM release explicitly; no global vllm>=0.29 dependency was added.
• Clarified that VLLM_VERSION_OR_COMMIT_HASH is consumed only by the opt-in USE_NIGHTLY_BUILD=1 source-rebuild path, while BASE_IMAGE pins the normal prebuilt runtime.
• No executable Docker command, dependency, test threshold, or CUDA/XPU/NPU path changed.

Validation:

• focused static test from the repository working directory: 5 passed
• the same focused test from /tmp: 5 passed
• changed-file pre-commit hooks, including Markdown lint: passed
• git diff --check: passed

AMD #11682 validated parent f8af7e2d146cfa78629c36bd9ffd463f4f518b39: the vLLM 0.29 canary passed, TorchCodec built and installed, the final VideoDecoder import layer completed, and a Z-Image diffusion worker reached model initialization and weight loading. The final exact-head AMD rerun will target 15d3c106799d3a90db64df8bdf8a51f961094dec.

@vllm-omni-review-bot please re-review this exact head.

[vllm-omni-review-bot]

Omni ReviewBot review

Scan:

Category	Result
Tests / verification	4 finding(s) below
Security	no finding reported
Docs / comments	7 finding(s) below
Behavior / compatibility	no finding reported
Correctness	1 finding(s) below

Validated:

• [resolved] TorchCodec ABI revalidation (hsliuustc0106): not re-raised; residual is still the script’s early-exit if base already imports VideoDecoder — author cited AMD #11682 TorchCodec v0.10.0 build on parent f8af7e2
• [resolved] CUDA/XPU/NPU still on v0.28 (Dockerfile.cuda:1, Dockerfile.xpu:21, Dockerfile.npu:2) — deferred by author to #7410; residual siblings remain unaligned with Dockerfile.ci
• [resolved] Prior review items on Path parents[2], CI_DOCKERFILE naming, canary startswith, USE_NIGHTLY=0 lock, template --build-arg forms present at head
• [resolved] CUDA/XPU/NPU v0.28 drift — author #7410; residual siblings still unaligned by design.
• [resolved] USE_NIGHTLY_BUILD=0 lock + unquoted template override bans + canary after nightly — present at head (test:41-54, Dockerfile:14,45).
• [resolved] Prebuilt vllm/vllm-omni-rocm:v0.28.0 left in place — intentional per #7405/PR body; residual is missing release-only disclaimer (comment below), not a tag bump ask.

Keep four items: the unrendered set-up-using-python version contract (major), the pre-built-images missing until-#7405 / backwards “below” note (minor, absorbing 0 and 1), the AMD exact-head canary verification ask (minor question, absorbing 5), and the BASE_IMAGE override test-integrity hole (minor). Drop the test_rocm_* rename nit as pure taxonomy polish.

Verdict: REQUEST CHANGES

Findings

• [P2] Residual (not blocking): tools/install_torchcodec_rocm.sh still early-exits if… — ``
  Residual (not blocking): tools/install_torchcodec_rocm.sh still early-exits if the base image already imports VideoDecoder, so a preinstalled TorchCodec can skip the ROCm ABI rebuild the Dockerfile intends. Pin remains TORCHCODEC_BRANCH=v0.10.0. Script unchanged by this PR.

Evidence: unchanged by this diff, present in the PR-time tree: tools/install_torchcodec_rocm.sh:13 TORCHCODEC_BRANCH="${TORCHCODEC_BRANCH:-v0.10.0}"; tools/install_torchcodec_rocm.sh:22-25 if verify_torchcodec >/dev/null 2>&1; then / echo "TorchCodec is already installed and working." / exit 0 / fi — early-exit skips rebuild when base already imports VideoDecoder; Dockerfile.rocm:59 invokes bash /tmp/install_torchcodec_rocm.sh after copying that script.

• [P2] Residual siblings remain on v0.28 while CI is v0.29: docker/Dockerfile.cuda:1,… — Dockerfile.cuda:1
  Residual siblings remain on v0.28 while CI is v0.29: docker/Dockerfile.cuda:1, docker/Dockerfile.xpu:21, docker/Dockerfile.npu:2 vs docker/Dockerfile.ci:7 — ROCm-only in this PR; tracked under #7410.

Evidence: unchanged by this diff, present in the PR-time tree: docker/Dockerfile.cuda:1 ARG BASE_IMAGE=vllm/vllm-openai:v0.28.0; docker/Dockerfile.xpu:21 ARG VLLM_VERSION=v0.28.0; docker/Dockerfile.npu:2 ARG VLLM_ASCEND_VERSION=v0.28.0; docker/Dockerfile.ci:7 ARG VLLM_BASE_TAG=v0.29.0 (ROCm updated in-diff: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0)

• [P2] Leaving vllm/vllm-omni-rocm:v0.28.0 is intentional (#7405 / source-build note) — ``
  Leaving vllm/vllm-omni-rocm:v0.28.0 is intentional (#7405 / source-build note). Residual: `pre-built-images` still has no release-only disclaimer next to the v0.28.0 examples (unlike wheels at line 26)—docs clarity only, not a tag-bump ask.

Evidence: docs/getting_started/installation/gpu/rocm.inc.md:169 vllm/vllm-omni-rocm:v0.28.0 \ and :190 vllm/vllm-omni-rocm:v0.28.0 — prebuilt examples still on 0.28.0. Contrast wheels disclaimer at :26 These pre-built wheel instructions install the published vLLM-Omni 0.28.0 release. For the 0.29 development line, use the source-install instructions below. — no equivalent beside the prebuilt-images commands (:149–191). Intentional-not-bump: :93–95 This upstream base is distinct from the prebuilt vllm/vllm-omni-rocm images discussed below; their published-tag availability is tracked in [#7405](...). and docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0 (upstream base only).

[vllm-omni-review-bot]

[P1] This diff places the new BASE_IMAGE-vs-non-Docker / “vllm-omni does not insta…

This diff places the new BASE_IMAGE-vs-non-Docker / “vllm-omni does not install vLLM” version-contract prose under # --8<-- [start:set-up-using-python] in rocm.inc.md (lines 11–20). Cited gpu.md includes only requirements, pre-built-wheels, build-wheel-from-source, pre-built-images, and build-docker — not set-up-using-python — so that unique contract never appears on the published GPU install page (README’s short 0.28/0.29 note does not replace the BASE_IMAGE/no-dep sentences). Move the prose into a rendered snippet (pre-built-wheels and/or build-wheel-from-source), or add an explicit --8<-- …:set-up-using-python include in gpu.md.

Evidence: rocm.inc.md:9 # --8<-- [start:set-up-using-python] then 17–20 BASE_IMAGE/no-dep prose; gpu.md:47/67/87/105 include only pre-built-wheels / build-wheel-from-source / pre-built-images / build-docker — no set-up-using-python include.

Suggestion: The Dockerfile's BASE_IMAGE pin applies only to Docker builds. The
vllm-omni package does not install vLLM as a dependency, so non-Docker source
installs must install the matching ROCm vLLM release explicitly before
installing vLLM-Omni, as shown below.

Installation of vLLM

[vllm-omni-review-bot]

[P2] The fail-fast canary at docker/Dockerfile.rocm:45 (Validated vLLM …) only r…

The fail-fast canary at docker/Dockerfile.rocm:45 (Validated vLLM …) only runs during image construction. L1 tests/buildkite/test_rocm_dockerfile.py only asserts Dockerfile/template text (including canary string presence). Unchanged amd-build at .buildkite/amd/test-template-amd-omni.j2:77 builds -f docker/Dockerfile.rocm without overriding BASE_IMAGE/USE_NIGHTLY_BUILD, so that job is the real gate for vllm/vllm-openai-rocm:v0.29.0. Please link a green amd-build for 15d3c106799d3a90db64df8bdf8a51f961094dec showing Validated vLLM 0.29.0, or explicitly accept parent #11682 plus this docs-only tip before merge.

Evidence: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; docker/Dockerfile.rocm:45 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')"; tests/buildkite/test_rocm_dockerfile.py:64-76 def test_rocm_dockerfile_contains_vllm_api_canary() only _line_index/assert on Dockerfile text for the canary import string (no image build); unchanged by this diff, present in the PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ." (no --build-arg BASE_IMAGE / USE_NIGHTLY_BUILD).

[vllm-omni-review-bot]

[P2] Rename tests/buildkite/test_rocm_dockerfile.py to an amd-prefixed sibling (e.g

Rename tests/buildkite/test_rocm_dockerfile.py to an amd-prefixed sibling (e.g. test_amd_rocm_dockerfile.py alongside test_amd_{bootstrap,pipeline,suite_selection}.py). This file locks the AMD CI template .buildkite/amd/test-template-amd-omni.j2 but breaks the existing test_amd_* AMD-contract naming.

Evidence: tests/buildkite/test_rocm_dockerfile.py:11 AMD_TEMPLATE = REPO_ROOT / ".buildkite/amd/test-template-amd-omni.j2" — new file path is test_rocm_dockerfile.py while locking the AMD template. Unchanged by this diff, present in the PR-time tree: siblings tests/buildkite/test_amd_bootstrap.py, tests/buildkite/test_amd_pipeline.py, tests/buildkite/test_amd_suite_selection.py use the test_amd_* AMD-contract naming.

[vllm-omni-review-bot]

[P2] The image canary only imports compute_layout_strides and KVCacheLayout (sam…

The image canary only imports compute_layout_strides and KVCacheLayout (same 0.29 pair as vllm_omni/diffusion/diffusion_kv/layout.py:23-24); a green canary still leaves other missing 0.29 APIs beyond those two imports unguarded. Is that two-symbol surface the accepted fail-fast contract for this bump, or should the RUN gate cover additional 0.29 call sites?

Evidence: docker/Dockerfile.rocm:45 RUN python3 -c "import vllm; from vllm.v1.kv_cache_interface import compute_layout_strides; from vllm.v1.kv_cache_layout import KVCacheLayout; print(f'Validated vLLM {vllm.__version__}')" — canary gates only those two symbols. Unchanged by this diff beyond alignment (present in PR-time tree): vllm_omni/diffusion/diffusion_kv/layout.py:23-24 from vllm.v1.kv_cache_interface import KVCacheSpec, KVCacheTensor, compute_layout_strides / from vllm.v1.kv_cache_layout import KVCacheLayout — same compute_layout_strides + KVCacheLayout pair the canary mirrors.

[vllm-omni-review-bot]

[P2] Default BASE_IMAGE (and paired VLLM_VERSION_OR_COMMIT_HASH) bump is inherit…

Default BASE_IMAGE (and paired VLLM_VERSION_OR_COMMIT_HASH) bump is inherited on rebuild by every consumer of docker/Dockerfile.rocm — AMD amd-build (.buildkite/amd/test-template-amd-omni.j2 builds without --build-arg BASE_IMAGE/USE_NIGHTLY_BUILD; merge suite steps depends_on: amd-build), user docker build -f docker/Dockerfile.rocm, and recipes that cite that Dockerfile (Ming-TTS, SenseNova, Qwen3-TTS, Stable-Audio, OmniVoice, MammothModa2). Accept that shared default for those recipe paths, or call out any that must stay pinned off the new base.

Evidence: docker/Dockerfile.rocm:4 ARG BASE_IMAGE=vllm/vllm-openai-rocm:v0.29.0; docker/Dockerfile.rocm:15 ARG VLLM_VERSION_OR_COMMIT_HASH=v0.29.0. Unchanged by this diff, present in the PR-time tree: .buildkite/amd/test-template-amd-omni.j2:77 "docker build -f docker/Dockerfile.rocm -t {{ docker_image_amd }} --target test --build-arg ARG_PYTORCH_ROCM_ARCH=gfx942 --progress plain ." (no BASE_IMAGE/USE_NIGHTLY_BUILD override). PR adds tests/buildkite/test_rocm_dockerfile.py:52-54 asserting AMD build must not pass --build-arg BASE_IMAGE/USE_NIGHTLY_BUILD. Unchanged by this diff: recipes cite docker/Dockerfile.rocm (e.g. recipes/inclusionAI/Ming-omni-tts.md:197, recipes/SenseNova/SenseNova-U1.md:186, recipes/Qwen/Qwen3-TTS.md:359, recipes/StabilityAI/Stable-Audio-Open.md:174, recipes/k2-fsa/OmniVoice.md:47, recipes/MammothModa2/MammothModa2.md:140).

[hsliuustc0106]

Reviewed 15d3c10. No actionable findings. The five focused static regression tests and diff hygiene checks pass. AMD image-build and diffusion startup validation remain outstanding; Docker is unavailable in the local review environment.