[CI][ROCm] Make image dependencies cache-safe

[AndreasKaratzas]

This PR makes ROCm image builds dependency-aware and makes their caches safer to reuse across CI and release jobs.

• Add a generated ROCm source-build lock so test/runtime dependency churn does not invalidate base, csrc, or Rust builds.
• Harden cache keys, manifests, forced refreshes, scoped writes, diagnostics, and digest-qualified image handoffs.
• Fix dependency and ABI issues found during validation: pinned-source Decord, ABI-aware TorchCodec rebuilds, AITER's undeclared PyYAML runtime import, and LMCache's locked NumPy plus complete runtime roots.
• Preserve native-build parallelism, pin AITER v0.1.21.post1 to its peeled commit with FlyDSL 0.3.2, checksum the ROCm Triton source, honor the configured Rust toolchain, and extend CI build triggers.
• Keep credential-bearing SCCACHE_DOWNLOAD_URL values out of release BuildKit arguments and image history.
• Explicitly install libdw1 in the final ROCm image so the copied profiler SDK remains loadable after Python-only installation cleanup.

Test Plan

Scope	Validation
Static contracts	Existing metadata pytest suite and all applicable pre-commit hooks
Cache protocol	Offline fake-S3 matrix covering misses, corruption, force bypass, outages, interrupted writes, and concurrent writers
Images	Earlier ROCm base, ci_base, full test and production-default final images with cache repeats, as supporting evidence
Runtime	Offline package/PyYAML/import/linkage/native-payload checks plus one-GCD gfx90a Torch and AITER native-core/ABI imports

Commands run

.venv/bin/python -m pytest -q tests/tools/test_docker_build_metadata_args.py
mapfile -t pr_files < <(git diff --name-only "$(git merge-base origin/main HEAD)"..HEAD)
.venv/bin/pre-commit run --files "${pr_files[@]}"

The base and ci_base validation used the production refresh and Bake paths against an isolated local registry. Image results predate the dependency-naming/security follow-up and remain supporting evidence.

Test Result

Check	Result
Existing metadata tests	PASS — 13/13
Pre-commit and static validation	PASS
Fake-S3 cache protocol	PASS — 92/92
Earlier ROCm base	PASS (supporting evidence) — offline package/PyYAML/import/linkage/native-payload and gfx90a Torch/AITER checks passed; repeat returned the same digest with zero BuildKit vertices
Earlier ci_base	PASS (supporting evidence) — package/import/core-linkage and gfx90a checks passed; repeat returned immediately with zero BuildKit vertices
Earlier full test image	PASS (supporting evidence) — package, import, linkage, artifact, and cache-repeat checks passed
Earlier production-default final image with LMCache	PASS (supporting evidence) — uv/pip, runtime-root, import, native-linkage, and gfx90a core checks passed

The Python-only Installation failure in Buildkite build 12676 exposed an additional runtime dependency: purging the repository-setup packages left libdw1 eligible for the test's later apt autoremove, and Torch then failed to import libdw.so.1. The final image now installs libdw1 explicitly. For this follow-up, applicable pre-commit hooks, git diff --cached --check, and shell syntax validation passed; a controlled APT simulation using copied metadata confirmed that manual ownership preserves libdw1, with host package state unchanged. Cache review confirmed the Dockerfile change invalidates the base image identity. Rebuilt-image execution of the installation test remains unverified.

AI assistance was used for the dependency audit, validation, and drafting.

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @AndreasKaratzas.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[AndreasKaratzas]

/amd-ci run

[github-actions]

✅ Triggered Buildkite AMD CI #12647 for commit 01509d083644.

[AndreasKaratzas]

/amd-ci run

[github-actions]

✅ Triggered Buildkite AMD CI #12652 for commit d094e57118bd.

[AndreasKaratzas]

/amd-ci run

[github-actions]

✅ Triggered Buildkite AMD CI #12676 for commit 1c212c77ca41.

[AndreasKaratzas]

/amd-ci run

[github-actions]

✅ Triggered Buildkite AMD CI #12698 for commit 50990b54cafa.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @AndreasKaratzas.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork