Skip to content

ci(reborn): share one Rust cache across the closure (fix per-crate cache LRU eviction) - #5118

Merged
serrrfirat merged 1 commit into
mainfrom
firat/ci-shared-registry-cache
Jun 21, 2026
Merged

serrrfirat merged 1 commit into
mainfrom
firat/ci-shared-registry-cache

Conversation

@serrrfirat

Copy link
Copy Markdown
Collaborator

Problem (root cause of the closure's re-downloads + flakes)

The closure's crate-tests matrix used a per-crate cache key (reborn-tests-<crate>), so it produced ~60 caches (registry + target each, ~0.4–0.7 GB) plus 4 per-partition root caches (~1 GB each) → ~30+ GB competing for GitHub's ~10 GB per-repo cache LRU.

They evict each other almost as fast as they're written. Live snapshot when we investigated:

24 total repo caches, 18.0 GB  (limit ~10 GB → actively evicting)
only 10 of 64 closure crates had a cache present

So most jobs hit "No cache found" → re-download the crates.io registry from cold every run → and ~60 parallel cold downloads amplify transient registry flakes (the SSL_ERROR_SYSCALL / HTTP2 reds). It was LRU eviction, not code changes — the keys were valid; the entries were just evicted.

Fix

Switch both matrices to rust-cache shared-key:

  • crate-tests → shared-key: reborn-tests-crates — one entry for all crate jobs; the registry + shared-dep build is downloaded/compiled once and stays resident, not ~60×.
  • root partitions → shared-key: reborn-tests-root — all 4 partitions compile the identical root build, so one shared cache is strictly better than 4 copies.

Drops the reborn-tests cache footprint from ~30+ GB / 68 entries → ~2 entries that comfortably fit the LRU → the registry stays resident → no more cold re-downloads.

Trade-off

Crate-test target caching is now shared rather than per-crate, so a given crate may recompile its own (small) crate-specific code more often — but the shared dependency build (the bulk) and the registry (the download-relevant part) are always cached. The download/flake fix is the priority; if per-crate compile time regresses, we can add a separate per-crate target cache later.

Relationship to #5115

Complementary: this removes the cause of cold downloads (cache eviction); CARGO_NET_RETRY (#5115) remains the safety net for the now-rare cold download.

Automated agent-authored.

…r-crate caches

The closure's crate-tests matrix used a per-crate cache key
(`reborn-tests-<crate>`), producing ~60 caches that each bundle the
cargo registry + a target dir (~0.4-0.7 GB each), plus 4 per-partition
root caches (~1 GB each). That's ~30+ GB competing for GitHub's ~10 GB
per-repo cache LRU, so the caches evict each other almost as fast as
they're written: a live snapshot showed only 10 of 64 crates had a cache
present, summing to 18 GB and actively evicting. The result is "No cache
found" on most jobs -> the crates.io registry is re-downloaded from cold
every run, and 64 parallel cold downloads amplify transient registry
flakes (the SSL_ERROR_SYSCALL / HTTP2 reds we saw).

Switch both matrices to `shared-key`:
- crate-tests -> `shared-key: reborn-tests-crates` (one entry for all
  crate jobs; the registry + shared-dep build is downloaded/compiled
  once and resident, not ~60x).
- root partitions -> `shared-key: reborn-tests-root` (all 4 partitions
  compile the identical root build, so one cache is strictly better than
  4 copies).

This drops the reborn-tests cache footprint from ~30+ GB / 68 entries to
~2 entries that comfortably fit the LRU, so the registry stays resident
and stops being re-downloaded. Complements CARGO_NET_RETRY (#5115),
which remains the safety net for the now-rare cold download.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5118 June 21, 2026 21:45 Destroyed
@github-actions github-actions Bot added the scope: ci CI/CD workflows label Jun 21, 2026
@coderabbitai

coderabbitai Bot commented Jun 21, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 975bc19b-f8e9-40c2-a7a8-18e37aa9105b

📥 Commits

Reviewing files that changed from the base of the PR and between b1924ab and 171c73e.

📒 Files selected for processing (1)
  • .github/workflows/reborn-tests.yml

📝 Walkthrough

Summary by CodeRabbit

Release Notes

  • Chores
    • Optimized CI/CD cache configuration to improve test job efficiency.

Walkthrough

The reborn-tests workflow updates Rust cache keying in two jobs. The ~60 crate-test matrix jobs move from per-package keys to a single reborn-tests-crates key, and the four root parity partitions move from per-partition keys to a single reborn-tests-root key.

Changes

CI Cache Key Consolidation

Layer / File(s) Summary
Shared cache keys for crate-test and root parity jobs
.github/workflows/reborn-tests.yml
crate-test jobs replace reborn-tests-${{ matrix.package }} with constant reborn-tests-crates; root parity partitions replace reborn-tests-root-${{ matrix.partition }} with constant reborn-tests-root. Both changes collapse per-variant cache entries into a single shared entry.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

Two cache keys, once sixty-four,
Now one each—no more, no more.
Matrix variants share the pile,
Registry reused in style.
🦀 One key to rule them all.

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning Description includes root cause analysis, solution details, trade-offs, and relationship to prior work, but omits most template sections (Change Type, Validation, Security Impact, Blast Radius, Review track). Complete required sections: mark CI/Infrastructure as Change Type; confirm validation steps (cargo fmt/clippy if applicable); confirm no security/trust-boundary impact; specify blast radius and review track.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed Title follows Conventional Commits (ci scope), clearly describes the fix (shared cache strategy), and accurately reflects the main change in the workflow file.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added size: S 10-49 changed lines risk: medium Business logic, config, or moderate-risk modules contributor: core 20+ merged PRs labels Jun 21, 2026
@railway-app

railway-app Bot commented Jun 21, 2026

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-5118 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw 🕒 Building (View Logs) Web Jun 21, 2026 at 9:45 pm

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-5118 — 171c73e5 Deployed Jun 21, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: medium Business logic, config, or moderate-risk modules scope: ci CI/CD workflows size: S 10-49 changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant