[moe-berri]

TLDR

Problem this solves:

• Worker release tags can move after an approved build.
• Worker dependencies lack hashes and image vulnerability coverage.

How it solves it:

• Helm installs and advertises the approved worker digest.
• Development builds publish to a separate worker package.
• Worker uses pinned Wolfi/uv images and hashed Python dependencies.
• Both native architectures receive runtime, hash-tampering, and vulnerability checks.

User Flow

Before: a chart install resolves the worker's release tag again.

1. Install the chart and enable the Lens worker.
2. Later restart the worker after its image tag changes.
3. The restart can run different content from the approved release.

After: the published chart carries its approved worker digest.

1. Install the chart and enable the Lens worker.
2. Later restart the worker after its image tag changes.
3. The restart still requests the approved content; registration advertises that same digest.

Pre-Submission checklist

• Added meaningful regression tests
• Focused release tests and all 172 Helm tests pass locally
• Repository make check passes locally
• Current-tip hosted image, Compose, and native architecture checks pass
• Bot reviews completed and findings addressed

Screenshots / Proof of Fix

Run helm template proof helm/litellm --set database.writer.host=localhost --set database.writer.dbname=test --set lensWorker.enabled=true --set lensWorker.tokenSecret.name=test --show-only templates/lens/deployment.yaml with each case's image settings below. The accompanying publisher #291 supplies and verifies the digest before chart publication; the empty source-chart default is never published by that pipeline.

Before (dfdd496)

Approved image after tag replacement

1. Add --set backend.image.tag=replaced-release --set lensWorker.image.digest=sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.
2. Rendered image: ghcr.io/berriai/litellm-lens-worker:replaced-release. The supplied digest is ignored.

Source commit image

1. Add --set backend.image.tag=sha-0123456789abcdef.
2. Rendered image: ghcr.io/berriai/litellm-lens-worker:sha-0123456789abcdef.

After (1454f5c)

Approved image after tag replacement

1. Run the same render command and digest settings.
2. Rendered image: ghcr.io/berriai/litellm-lens-worker@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. Backend registration advertises that same digest.

Source commit image

1. Run the same source-tag render command.
2. Rendered image: ghcr.io/berriai/litellm-lens-worker-dev:sha-0123456789abcdef. Private registry overrides remain intact.

Both native image jobs at this tip passed real image builds, imports, non-root/read-only temporary storage, deliberately corrupted dependency hashes, and Grype scanning. Greptile reports 5/5 with both findings addressed; Veria, Bugbot, and the broader Buildkite end-to-end suite pass. The full hosted Compose installation and restart passes at 1454f5c: real PostgreSQL/ClickHouse/LiteLLM/worker startup, authenticated heartbeat, trace readback, direct database access denied, persisted trace/token/identity/billing after restart, wrong-release rejection, and refusal of unversioned setup/claims followed by recovery. The publisher's real OCI tests also replaced a public version tag after approval: both mirrors, real signatures/SPDX attestations, verification, and this chart continued using the approved multiarch digest. An intentionally broken helper that ignored the digest stopped chart publication.

Type

🐛 Bug Fix
🚄 Infrastructure

Caveats

High

• Admin must revoke LiteLLM writes to the release package.
• Publisher Sagemaker streaming #291 must pass recorded digests into chart packaging.

Medium

• New development package needs public visibility and LiteLLM publishing access.
• Source charts retain tag fallback; published charts require approved digests.

All current-tip checks are complete: 115 successful, one skipped, and no failures. The full-image vulnerability scan also passes. No release or deployment has been dispatched. Package permissions and dev-package public visibility remain operator prerequisites as described above.

[moe-berri]

@greptileai @veria-ai Please review the current tip for release integrity: Helm digest precedence, separation of dev and release image destinations, worker dependency hash enforcement, build context completeness, and native runtime compatibility.

[moe-berri]

bugbot run

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

[High risk] Rebuilds the worker container image and changes its registry location.

No outstanding review finding blocks merging this PR.

Summary

The PR pins published Lens worker images by approved digest, separates development images from release images, and adds hashed dependencies and native-architecture image checks. The changes since the previous review make source charts select the development package for commit tags. Both previous Greptile threads are resolved; the source chart’s empty digest is intentional, with publication enforcement described in the companion publisher.

_{Reviews (3) · Last reviewed commit: "fix(lens): select the dev worker package..."}

[cursor]

Stale Bugbot comment from a previous run.

[moe-berri]

@greptileai @veria-ai Please review 1454f5c. The SHA-tagged source-chart fallback is fixed and covered by three rendered Helm cases. The required publisher half is now pushed in BerriAI/project-releaser#291 at 334bb07: it requires the recorded digest, checks the registry by digest, and rejects a chart whose rendered worker ignores it. Real tag-replacement tests passed for mirror/sign/attest/verify/chart, including an intentionally broken Helm helper. The package ACL and dockerhub branch-policy changes remain explicit admin prerequisites.

[moe-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1454f5c. Configure here.}

[moe-berri]

@greptileai Please re-review the remaining digest-default finding with the companion publisher enforcement described in my inline reply. Publisher #291 at 334bb07 requires the approved digest and verifies the rendered image before packaging; missing digest and an ignored digest both fail in real OCI tests. The source value is intentionally unstamped until an approved build exists.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lens_release_integrity (1454f5c) with main (0ed1c08)¹}

Footnotes

1. No successful run was found on main (4b67a2b) during the generation of this report, so 0ed1c08 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩