Skip to content

ci(lmcache): publish LMCache wheels as Docker Hub images instead of GitHub Releases - #2394

Closed
yhl-amd wants to merge 4 commits into
feat/dsv4-lmcache-mpfrom
feat/lmcache-wheel-dockerhub
Closed

yhl-amd wants to merge 4 commits into
feat/dsv4-lmcache-mpfrom
feat/lmcache-wheel-dockerhub

Conversation

@yhl-amd

@yhl-amd yhl-amd commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Stacked on #2250 (feat/dsv4-lmcache-mp).

Why

lmcache-rocm-wheel.yaml published wheels as GitHub Releases. A release needs a git tag, and lmcache-v0.5.6.dev98-g05fc77a0-rocm-torch210 was created on main's history, so setuptools-scm's git describe picked it up and failed to parse it. pip install -e . then failed in every Pre Checkin run. That tag and release have been deleted, which also broke #2250's pinned releases/download/... URL.

What

  • Publish: the wheel is pushed as a FROM scratch image that holds only the wheel at /, tagged rocm/atom-dev:lmcache-v<version>-g<sha8>-rocm-torch210. It uses the repo's existing Docker Hub credentials (docker-auth action + docker_push_retry.sh, same as docker-release.yaml) and refuses to overwrite an existing tag. Nothing appears under Releases or Tags.
  • source_run_id input: publishes the wheel that an earlier run built, without rebuilding, as long as its artifact is still retained. The wheel must match lmcache_commit and the ABI suffix.
  • Dockerfiles: ARG LMCACHE_WHEEL_IMAGE (tag@digest) → FROM ${LMCACHE_WHEEL_IMAGE} AS lmcache_wheel → COPY --from=lmcache_wheel / /tmp/lmcache-wheel/. The sha256 check is kept, and lmcache.__version__ is still asserted, now parsed from the wheel name.
  • bump script / bump-pr job: pin --image (digest required) instead of --release. The Requires-Dist diff reads the old wheel out of the old image.

Pinned wheel

rocm/atom-dev:lmcache-v0.5.6.dev98-g05fc77a0-rocm-torch210@sha256:d3cfe74f42d78a188992cae98efbe23053610216e7b247632d6be772e9d465d6, published from run 35984043135's artifact by https://github.com/ROCm/ATOM/actions/runs/36002407218. The wheel sha256 is a5fe8f3f5b9dee602ac7d11241f65a1640cd3d26c101f2d1d0e0d8aee88b7aab, byte-identical to the previously validated release asset.

Validation

  • actionlint 1.7.7 clean; black clean on the bump script
  • The publish run above succeeded. An anonymous docker pull of the digest contains exactly the one wheel, with a matching sha256.
  • Locally: a FROM ${ARG} AS stage + COPY --from consumer build copies only the wheel. The Dockerfile's wheel-name/version/sha256 shell logic was run against the real wheel (and rejects 2 wheels). The publish job's wheel-check script was run against the real artifact layout. The bump script was run on both Dockerfiles and rejects non-digest refs.
  • Not run: a full ATOM image build, and the bump-pr job (it needs the new pin on main first).

🤖 Generated with Claude Code

Honglie Yi and others added 3 commits September 24, 2026 12:57
A GitHub Release needs a git tag, and the lmcache-v...-rocm-torch210 tag
on main was picked up by setuptools-scm's git describe, breaking
`pip install -e .` for every CI job. Publish the wheel as a FROM scratch
image, rocm/atom-dev:lmcache-v<version>-g<sha8>-rocm-torch210, and pin
it in the Dockerfiles by digest (LMCACHE_WHEEL_IMAGE) plus the wheel
sha256. Nothing appears under Releases or Tags.

source_run_id publishes the wheel an earlier run built instead of
rebuilding, so a validated wheel can be pinned byte for byte.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Published from run 35984043135's wheel by
https://github.com/ROCm/ATOM/actions/runs/36002407218 (same sha256 as the
previously validated release asset).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lmcache_wheel is now the full /tmp/lmcache-wheel/... path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Same fixes as #2395: check-inputs job (full commit SHA on every path,
numeric source_run_id, source_run_id requires publish), exactly one
artifact resolved per publish (build's own attempt, or the latest
unexpired attempt of a successful build of this workflow on the default
or current branch), only "no such manifest" allows a push, and Docker
Hub credentials scoped to the push step.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@yhl-amd

yhl-amd commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #2395, which now carries the Dockerfile and bump-script changes straight to main. When merging main into feat/dsv4-lmcache-mp, take main's side for docker/Dockerfile, docker/atom_release.dockerfile, .github/workflows/lmcache-rocm-wheel.yaml and .github/scripts/bump_lmcache_wheel_pin.py.

@yhl-amd yhl-amd closed this Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant