fix(ci): aggressively reclaim disk before Trivy (no-space DB download failure) - #1728
Conversation
… failure) GHCR run 27042677318: 'Build archon' built+pushed fine but the Trivy scan FAILED with 'failed to download vulnerability DB: ... no space left on device' on kvm4-1 — the scanner could not fetch its ~95MB DB. False-negative gate (image is fine, gate fails for infra reasons), masking the real CVE signal. The prior step only pruned BuildKit cache + dangling images. Strengthen it: - docker image prune -af (image is RE-PULLED in the next step, so removing all unused images is safe; reclaims GBs of layers) - drop stale/corrupt .cache/trivy DB so the download restarts clean - sweep /tmp SBOM/scan scratch (stereoscope-*, sbom-action-*, trivy-*) that docker prune does not reclaim (same class as #1719's build-host sweep) - log df -h before/after for observability Deliberately NOT 'docker volume prune' — these runners may co-host fleet data-store volumes; pruning volumes could destroy stopped-service data. Unblocks Trivy verification of the archon/open-notebook CVE dep bumps (#1726). kvm4 runners are chronically disk-tight — flagged for operator hygiene. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Too much diff to scan? Review this PR in Change Stack to start with the highest-impact changes. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe PR updates the "Free disk space before Trivy" step in the integrations-ghcr GitHub Actions workflow to reclaim root filesystem space more aggressively on self-hosted runners. It now logs disk usage before and after, performs aggressive Docker pruning, explicitly deletes cached Trivy database and fanal directories, and expands temporary directory cleanup to include Trivy scratch artifacts. ChangesTrivy Pre-Scan Disk Cleanup
Possibly related PRs
Poem
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8080bcd83
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Sweep the SBOM/scan scratch that the docker prune does NOT reclaim | ||
| # (same class as the build-host sweep in #1719) — frees the root volume. | ||
| for pat in 'stereoscope-*' 'sbom-action-*' 'trivy-*' 'codeql-packaging*' 'codeql.*'; do | ||
| find /tmp -maxdepth 1 -name "$pat" -mtime +0 -exec rm -rf {} + 2>/dev/null || true |
There was a problem hiding this comment.
In the build-publish job, this sweep runs immediately after Generate SBOMs and before the Trivy DB pull. find -mtime +0 only matches entries older than a full day (checked GNU findutils 4.9.0 locally: a newly-created stereoscope-* dir was not printed, a 25h-old one was), so any large /tmp/stereoscope-* or /tmp/sbom-action-* tree just left by the SBOM scan is skipped. On the disk-full path this change is trying to fix, that leaves the root volume full and the following docker pull/Trivy DB download can still fail with no space left on device.
Useful? React with 👍 / 👎.
1 participant
Problem
GHCR run
27042677318—Build archonbuilt + pushed successfully, but the Trivy gate failed with:Trivy couldn't fetch its ~95 MB vuln DB on kvm4-1 — it never scanned a package. This is a false-negative gate: the image is fine (CVE dep bumps from #1726 present), but the gate fails for an infra reason, masking the real security signal. 3rd disk-related GHCR failure (after #1717 kvm2 deadlock, #1719 /tmp stereoscope sweep).
Fix
Free disk space before Trivyonly pruned BuildKit cache + dangling images. Strengthen it (the image is re-pulled in the next step, so removing unused images is safe):docker image prune -af(reclaims GBs of layers).cache/trivyDB so the download restarts clean/tmpSBOM/scan scratch (stereoscope-*,sbom-action-*,trivy-*) that docker prune doesn't reclaim (same class as fix(ci): self-clean build-host /tmp SBOM scratch in build-publish (Codex P1 fast-follow #1717) #1719)df -hbefore/afterDeliberately excludes volume pruning — these runners may co-host fleet data-store volumes (neo4j/supabase-db/etc.); removing volumes could destroy a temporarily-stopped service's data. Image-layer reclaim frees far more than the DB needs.
Verification
YAML parses (build-publish 31 steps). Unblocks Trivy verification of the archon/open-notebook CVE remediation (#1726).
Note (operator)
kvm4 self-hosted runners are chronically disk-tight — this is a workflow-side mitigation; the runner hosts need disk hygiene (old
_workdirs, caches). Flagged for operator follow-up.🤖 Generated with Claude Code
Summary by CodeRabbit