Skip to content

Make native package releases immutable and complete - #15

Merged
ndizazzo merged 3 commits into
mainfrom
codex/immutable-release-evidence
Jul 29, 2026
Merged

Make native package releases immutable and complete#15
ndizazzo merged 3 commits into
mainfrom
codex/immutable-release-evidence

Conversation

@ndizazzo

@ndizazzo ndizazzo commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • scan and verify each exact native package as the SPDX subject
  • namespace per-row BuildKit provenance and assemble one verified 11-subject release statement during full dry runs
  • reject filtered publication and make existing releases exact no-ops only; create new releases once without clobbering
  • document the immutable evidence and release gates

Validation

  • focused evidence/matrix/workflow tests: 17/17 pass
  • image-matrix coverage: 100% lines, branches, and functions
  • 11 release rows / 11 unique package filenames
  • shellcheck, actionlint, workflow policy scan, and git diff check pass
  • Ubuntu Dockerfile buildx check passes

Local environment notes

Two untouched client-readiness-smoke cases remain timing-sensitive on macOS, and Docker Desktop cannot resolve the advertised archlinux:base-devel amd64 manifest locally. Linux PR CI remains the authoritative gate for those existing checks. The TODO QA item stays open until PR CI passes.

Summary by CodeRabbit

  • New Features

    • Added comprehensive release evidence assembly and verification for native packages.
    • Added SPDX SBOM subject validation tied to exact package files and checksums.
    • Added immutable release publishing with exact asset and metadata verification.
  • Bug Fixes

    • Prevented filtered publishing runs and accidental release asset replacement.
  • Documentation

    • Updated publishing, signing, readiness, and release-checklist guidance for stricter verification requirements.
  • Tests

    • Added coverage for evidence assembly, SBOM validation, release comparisons, and workflow safeguards.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@ndizazzo, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: cdadf38d-ba30-4b70-b671-eb3fe7364ee0

📥 Commits

Reviewing files that changed from the base of the PR and between 5edea73 and e555359.

📒 Files selected for processing (6)
  • .github/workflows/images-release.yml
  • TODO.md
  • docs/publishing.md
  • scripts/release-evidence.ts
  • tests/release-evidence.test.ts
  • tests/workflow-provenance.test.ts
📝 Walkthrough

Walkthrough

Changes

Native release evidence and immutable publication

Layer / File(s) Summary
Package and SBOM evidence contracts
scripts/image-matrix.ts, scripts/verify-sbom-subject.ts, tests/image-matrix.test.ts, tests/sbom-subject.test.ts
Matrix rows now carry package filenames, while SBOM verification checks exact package names and SHA-256 sidecars.
Evidence assembly and verification
scripts/release-evidence.ts, tests/release-evidence.test.ts
Release evidence assembly validates 11 package rows, emits aggregate provenance and checksums, and rejects asset or release metadata drift.
Workflow package evidence integration
.github/workflows/images-precheck.yml, .github/workflows/images-release.yml, tests/workflow-provenance.test.ts
The workflow gates filtered publishing, namespaces BuildKit provenance, generates pinned SBOMs, and verifies package subjects.
Immutable release assembly and publication
.github/workflows/images-release.yml, docs/*, TODO.md
Release assembly and publication now re-verify exact assets, create releases without clobbering, allow exact-match no-ops, and record readiness state.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Workflow
  participant NativeArtifacts
  participant release-evidence.ts
  participant GitHubRelease
  Workflow->>NativeArtifacts: download package and Homebrew artifacts
  Workflow->>release-evidence.ts: assemble and verify release evidence
  release-evidence.ts-->>Workflow: return provenance, checksums, and package subjects
  Workflow->>GitHubRelease: compare existing release or create immutable release
  GitHubRelease-->>Workflow: return release metadata and asset digests
  Workflow->>release-evidence.ts: verify published release contents
Loading

Possibly related PRs

  • Mesh-LLM/mesh-packaging#4: Both changes modify release asset staging, checksum generation, provenance handling, and upload behavior.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: making native package releases immutable and complete.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/immutable-release-evidence

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (3)
tests/workflow-provenance.test.ts (1)

84-86: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Use a substring check instead of an escaped dynamic RegExp.

All snippets are fully escaped literals, so the regex adds nothing but the dynamic-regexp lint warning.

♻️ Proposed simplification
-    assert.match(workflow, new RegExp(snippet.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")));
+    assert.ok(workflow.includes(snippet), `workflow is missing ${snippet}`);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/workflow-provenance.test.ts` around lines 84 - 86, Replace the escaped
dynamic RegExp assertion in the workflow snippet checks with a direct substring
assertion using the existing workflow and snippet values. Preserve the current
behavior of verifying that each literal snippet occurs within workflow.

Source: Linters/SAST tools

.github/workflows/images-release.yml (1)

794-801: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Consider persist-credentials: false on checkout in the publishing job.

This job holds contents: write and id-token: write; leaving the checkout token in .git/config widens the blast radius of any subsequent step, and the checkout is only needed for the scripts. Same applies to the release-assembly checkout at line 744.

🔒 Proposed hardening
-      - uses: actions/checkout@v7
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/images-release.yml around lines 794 - 801, Update the
checkout steps in both the publishing job and the release-assembly job to set
persist-credentials to false, including the checkout near the artifact download
and the one referenced around the release-assembly section. Keep the existing
checkout behavior otherwise unchanged.

Source: Linters/SAST tools

scripts/release-evidence.ts (1)

141-158: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Release completeness counts are duplicated instead of derived. The 11-row matrix requirement and the 47-asset publication check encode the same contract (4 * rows + 3) in two places, plus a mirrored string in tests/workflow-provenance.test.ts. Adding a distro or backend requires synchronized edits, and a mismatch only surfaces at publish time.

  • scripts/release-evidence.ts#L141-L158: name the expected row count as a shared constant (or accept it as a CLI/matrix-derived input) rather than inlining 11.
  • .github/workflows/images-release.yml#L873-L877: derive the expected asset count from the assembled evidence/matrix instead of the literal 47.

As per coding guidelines: "When adding a distro or backend, update matrix validation in scripts/image-matrix.ts" — keeping the completeness contract in one derived place preserves that single update point.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/release-evidence.ts` around lines 141 - 158, Update matrixRows in
scripts/release-evidence.ts to use a shared or matrix-derived expected row count
instead of the literal 11, preserving validation of the complete matrix. Update
.github/workflows/images-release.yml lines 873-877 to calculate the expected
asset count from the assembled evidence or matrix using the 4 × rows + 3
contract, replacing the literal 47; keep the completeness contract derived from
the matrix validation in scripts/image-matrix.ts.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/images-release.yml:
- Around line 309-315: Update the provenance handling around the “Namespace and
require BuildKit package provenance” step so the emitted in-toto subject uses
the exact matrix package filename expected by statementSubject(), rather than a
Docker PURL. Ensure each native-package provenance subjectName matches
package_file before renaming or assembling the evidence, while preserving the
existing provenance existence check.

In `@docs/publishing.md`:
- Around line 21-23: Update the native package SPDX contract sentence in the
publishing documentation to include `.apk` alongside `.deb` and `.pkg.tar.zst`,
preserving the requirement that SPDX scans and subjects reference the exact
package file.

In `@scripts/release-evidence.ts`:
- Around line 358-369: Update the asset-processing loop in the release
comparison flow around requireString(asset.digest, ...) to tolerate missing or
null digests: skip those assets from digest comparison or emit a non-fatal
warning instead of throwing. Preserve duplicate-name and count/mismatch
validation for assets with available digests.

---

Nitpick comments:
In @.github/workflows/images-release.yml:
- Around line 794-801: Update the checkout steps in both the publishing job and
the release-assembly job to set persist-credentials to false, including the
checkout near the artifact download and the one referenced around the
release-assembly section. Keep the existing checkout behavior otherwise
unchanged.

In `@scripts/release-evidence.ts`:
- Around line 141-158: Update matrixRows in scripts/release-evidence.ts to use a
shared or matrix-derived expected row count instead of the literal 11,
preserving validation of the complete matrix. Update
.github/workflows/images-release.yml lines 873-877 to calculate the expected
asset count from the assembled evidence or matrix using the 4 × rows + 3
contract, replacing the literal 47; keep the completeness contract derived from
the matrix validation in scripts/image-matrix.ts.

In `@tests/workflow-provenance.test.ts`:
- Around line 84-86: Replace the escaped dynamic RegExp assertion in the
workflow snippet checks with a direct substring assertion using the existing
workflow and snippet values. Preserve the current behavior of verifying that
each literal snippet occurs within workflow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 89828ede-7bb9-4b52-96cb-3366faccf086

📥 Commits

Reviewing files that changed from the base of the PR and between b6192e8 and 5edea73.

📒 Files selected for processing (13)
  • .github/workflows/images-precheck.yml
  • .github/workflows/images-release.yml
  • TODO.md
  • docs/package-signing.md
  • docs/publishing.md
  • docs/release-checklist.md
  • scripts/image-matrix.ts
  • scripts/release-evidence.ts
  • scripts/verify-sbom-subject.ts
  • tests/image-matrix.test.ts
  • tests/release-evidence.test.ts
  • tests/sbom-subject.test.ts
  • tests/workflow-provenance.test.ts

Comment thread .github/workflows/images-release.yml Outdated
Comment thread docs/publishing.md Outdated
Comment thread scripts/release-evidence.ts
@ndizazzo
ndizazzo merged commit abdcbcd into main Jul 29, 2026
2 checks passed
@ndizazzo
ndizazzo deleted the codex/immutable-release-evidence branch July 29, 2026 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant