Repository navigation
ci: park each pull request's build on its root and route its re-push back to it - #15020
Conversation
…back to it A root's kept DerivedData is replaced by the next admission on it, so a re-push usually found its previous build gone. `keep` now parks another pull request's build in pr-builds/pr-<n> (a rename; 2 per root, 6 h, only with 140 GiB free), `check` swaps it back in for that pull request, and warm-keys publishes it as `parked` so distance routing sends the re-push to the mini that holds it. A same-PR start with the PR's own package interface change ranks far rather than rebuild, since every other start rebuilds. Ported from #14804 without its age pricing, jobs-API lookup, contention multiplier or SwiftPM scratch, onto main's distance routing (#14949). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 26 minutes. View limit detailsLimit details: You’ve used all 10 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
All contributors have signed the CLA ✍️ ✅ |
CI failure attributionCI passes on Written by |
|
Toolbox g1 🔔 reviewed the failed run. The implementation checks passed, but the CLI product job was refused on |
…ed keys out of warm-keys From review: unpark only over another pull request's build (or none), and routing costs the parked build exactly when check would swap it in; keep clears the job's own stale slot; a park killed after its stamp is renamed into place; expired slots are not published; parked pr- keys no longer crowd MAX_WARM_KEYS (roots[].parked carries them). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the stamp on a failed park Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot is paused — on-demand spend limit reachedBugbot uses usage-based billing for this team and has hit its on-demand spend limit. A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue. |
|
Merge receipt for |
…isk floor (#15039) * ci: evict parked pull request builds oldest first instead of a free-disk floor #15020 parked a pull request's build only with 140 GiB free. There is no floor now: parking is bounded by 2 per root and 6 h, and when space runs out parked builds go first. `keep` that hits ENOSPC evicts every parked build on the mini, oldest first, and retries the clone once; `owned_build_state.py evict-parked STORE [COUNT]` does the same for disk tooling. Main builds are never parked, so eviction never touches one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: clean a partial clone on failure, strict evict-parked args (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
c490fb4 Add an opt-in subtle sidebar selection highlight (manaflow-ai#14890) b78dae5 ci: evict parked pull request builds oldest first instead of a free-disk floor (manaflow-ai#15039) 31b6fc9 test: close the TabManagers the session snapshot tests create (manaflow-ai#15038) e64eb93 ci: send every retry of an owned-mini job to Blacksmith, once (manaflow-ai#15035) 17f0f90 Test: pin persisted Kimi custom kind on exact remote restore (manaflow-ai#15034) f9d5bb5 Switch between cmux and cmux NIGHTLY from inside the app (manaflow-ai#14995) 756847a sidebar: keep the selected row clear of the footer (manaflow-ai#15013) 1431a1e One dialog per close, and Feed banner clicks open the agent (manaflow-ai#14960) 74d4719 ci: route cli-product-tests to the gui runners like the shards (manaflow-ai#15011) b78e8aa ci: park each pull request's build on its root and route its re-push back to it (manaflow-ai#15020) 60e45e7 web tests: run every child process asynchronously (manaflow-ai#15001) 55724f0 ci: side lanes take the light minis' side runners first (manaflow-ai#15019) 3be8bdd Restoring a remote workspace terminal hands it this machine's working directory (manaflow-ai#8634) a9e555c Reattach 0.64.25 tmux-profile SSH workspaces after upgrading (manaflow-ai#14938) fe20e20 fix: mark workspace-level notifications read on workspace focus (manaflow-ai#12427) # Conflicts: # .github/workflows/app-host-test-rerun.yml # .github/workflows/auth-refresh-tests.yml # .github/workflows/ci-macos.yml # .github/workflows/ci.yml # .github/workflows/cloud-command-deadlines.yml # .github/workflows/cloud-machine-tests.yml # .github/workflows/cloud-task-local-tests.yml # .github/workflows/cmux-tui.yml # .github/workflows/iroh-v2.yml # .github/workflows/relay-tls.yml # .github/workflows/reload-build.yml # .github/workflows/remote-daemon.yml # .github/workflows/terminal-hang-diagnostics.yml
Why
Compile admission p50 was 334 s over 290 owned admissions (2026-09-26 13Z to 09-27 14Z, every mini's
admissions.jsonl). 165 of them (57%) rebuilt the app, at p50 396 s against 123 s for those that did not. Split by cause:For a PR that changes a package interface, every start except its own previous build rebuilds the app. 69 re-pushes rebuilt for their own changes: 53 on another mini than their previous push, and 16 on the same mini after another PR's admission replaced the kept build. Starts from the same PR's build skipped the rebuild 11 times in 19 (7 of 12 with a package interface change).
The distance picker (#14949) cannot find those builds, since a root publishes only its current stamp.
Change
keepreplaces another PR's build, it now moves that build topr-builds/pr-<n>beside the root's store. It's a rename, not a copy. A root keeps at most 2 parked builds, for 6 h, and parks only with 140 GiB free. The slots also sweep leftovers from a killed park.checkwithCMUX_OWNED_PR(compile admission passes the PR number) swaps that PR's parked build back in and parks the current one. It never drops a main build (idle warming's) or a build it could not park: then the job starts from the kept build.keepclears the job's own stale slot.rootsentry inwarm-keyscarries itsparkedstamps (owned_warm_state.py checks the fields). They stay out ofkeys, so they never crowd out other roots' keys.distance_route()costs a root by its parked build of this PR whenevercheckwould swap it in.hook_root_cost()ranks a same-PR startfar, notrebuild, when the only package change is the PR's own. Main's diff from a parked base still makes itrebuild.glaeda's hook reads the parked stamp when it ranks roots on the mini (teamleaderleo/glaeda#1284).
Ported from #14804 without its age pricing, jobs-API lookup, contention multiplier or SwiftPM scratch, and rebuilt on main's distance routing.
Expected effect
If every one of the 87 re-push rebuilds had started from its own build at the observed same-PR hit rate (40-58%), compile p50 would have been 220-246 s instead of 334 s, and p90 452-462 s instead of 478 s. That's an upper bound: a busy mini's parked build is only used when the picker's wait limit allows it. ci-dash
/estimates.jsonby_startandsameprstarts will show the change.Tests
tests/test_ci_owned_build_state.py(park, unpark, fingerprint mismatch, disk floor, count and age caps),tests/test_ci_owned_warm_state.py(parked fields),tests/test_ci_warm_distance.py(a parked build draws a package-changing re-push to its mini; hook parity, also run against glaeda#1284's hook). Targeted-kruns pass locally; CI runs the files.🤖 Generated with Claude Code
Summary by cubic
Prevents a pull request's re-push from rebuilding the app when its previous build is still on a root. When
keepreplaces another PR's build, it now parks that build inpr-builds/pr-<n>(a rename, not a copy) instead of dropping it; admission'scheckswaps the PR's parked build back in, never dropping a main build, and also replaces a kept build no router can read (stale or missing stamp). Distance routing sends the re-push to the mini holding it. A same-PR package interface change ranksfarinstead ofrebuild, since every other start rebuilds.warm-keyspublishes parked stamps asparkedon each root'srootsentry, which glaeda's hook reads; parked builds don't count against the keys list.Written for commit 752934d. Summary will update on new commits.