ci: run the owned side lanes on side runners, not root runners - #14431
Merged
Merged
Conversation
The Claude wrapper and remote daemon macOS tests are light jobs with no canonical root, but on an owned pick they took the pool label, which the root runners carry too. From 06:30 to 09:00Z on 2026-09-25, 11 of 21 such jobs landed on a root runner (3,300 s of root-runner time), keeping a compile or product consumer off that mini's root; on cmux7s and cmux9s, which have one root, they blocked the mini's only compile. The picker now writes side_runner, the pool's glaeda-side-* label, when the pool has a root count and CI_OWNED_POOL_SLOTS leaves it machines beyond its root runners. Both side lanes take it before the pool label on attempt 1 and on the rescue's attempt 2. Without a root count nothing changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 5 minutes. View limit detailsLimit details: You’ve used all 10 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (6)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
marker_peaks took a run's root-runner share as its marker peak less its jobs on the pool label. Side lanes on the side label were not subtracted, so each run with side lanes reserved up to 2 more root runners than it uses. Count jobs on the pool's side label too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
teamleaderleo
added a commit
that referenced
this pull request
Sep 25, 2026
… bounds - ci.yml passes macos_pr_side_runner to ci-macos.yml, and swift-package-tests takes pr_side_runner before the pool label on attempt 1 and the rescue's attempt 2, like the Claude wrapper and remote daemon lanes (#14431), so it never holds a mini's root runner. - pr_runner_pool: a full suite with release_build false places the package lane as a third side lane, a peak of 12. SIDE_LANES is 3, MAX_RUN_JOBS 12, and FULL_RUN (unknown routing) carries the package lane. The replay charge stays 3 (admission plus the two usual side lanes). - Tests: the release SDK lane, self-hosted guard and change-area checks read the new runs-on and Xcode conditions; picker tests cover the 12 peak and the side-runner wiring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
teamleaderleo
added a commit
that referenced
this pull request
Sep 25, 2026
… bounds - ci.yml passes macos_pr_side_runner to ci-macos.yml, and swift-package-tests takes pr_side_runner before the pool label on attempt 1 and the rescue's attempt 2, like the Claude wrapper and remote daemon lanes (#14431), so it never holds a mini's root runner. - pr_runner_pool: a full suite with release_build false places the package lane as a third side lane, a peak of 12. SIDE_LANES is 3, MAX_RUN_JOBS 12, and FULL_RUN (unknown routing) carries the package lane. The replay charge stays 3 (admission plus the two usual side lanes). - Tests: the release SDK lane, self-hosted guard and change-area checks read the new runs-on and Xcode conditions; picker tests cover the 12 peak and the side-runner wiring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
teamleaderleo
added a commit
that referenced
this pull request
Sep 25, 2026
…ease helper (#14411) * ci: run swift-package-tests on owned minis when the run builds no Release helper swift-package-tests never reached the owned Macs: all 4 runs in the last two hours were on blacksmith-6vcpu-macos-15. Its only hard need for macOS 15 is the SDK 15 Release Ghostty helper, which runs only on a full suite with release_build. Every other run (a package change under the compile-only policy) is plain `swift test`, which glaeda already classes as light. - pr_runner_pool: a `swift-package` side lane in run_plan when the lane runs and builds no helper (package_lane_owned(); an unknown release_build counts as a helper build). ci.yml passes RUN_SWIFT_PACKAGES and RUN_RELEASE_BUILD. - ci-macos.yml: swift-package-tests takes the picked owned label on attempt 1 (and the rescue's attempt 2 after a refusal) of a same-repository pull request whose owned_jobs names it, with the lane's Xcode; everything else keeps the macOS 15 pool and pin. The rescue already covers it (CI marker). - docs/ci-runners.md: the lane, plus a table of every macOS job and whether it may take an owned Mac. - Tests: picker placement and wiring, dual-Xcode guard, Xcode pin tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: put swift-package-tests on the side label and count it in the run bounds - ci.yml passes macos_pr_side_runner to ci-macos.yml, and swift-package-tests takes pr_side_runner before the pool label on attempt 1 and the rescue's attempt 2, like the Claude wrapper and remote daemon lanes (#14431), so it never holds a mini's root runner. - pr_runner_pool: a full suite with release_build false places the package lane as a third side lane, a peak of 12. SIDE_LANES is 3, MAX_RUN_JOBS 12, and FULL_RUN (unknown routing) carries the package lane. The replay charge stays 3 (admission plus the two usual side lanes). - Tests: the release SDK lane, self-hosted guard and change-area checks read the new runs-on and Xcode conditions; picker tests cover the 12 peak and the side-runner wiring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
On an owned pick, the two picker-routed side lanes (
Claude wrapper regressionsin ci.yml and the macOS tests in remote-daemon.yml) now take the pool's side label,glaeda-side-<class>-xcode-<version>, instead of the pool label.pr_runner_pool.pywrites a newside_runneroutput. It is set only when the pick has a root count (root_runneris set) andCI_OWNED_POOL_SLOTSgives the pool more machines than root runners (std 36 vs 16 root, light 4 vs 2), so a side lane never waits on a label no runner carries.macos_pr_side_runnerand passespr_side_runnerto remote-daemon.yml. Both lanes take it before the pool label on attempt 1 and on the rescue's attempt 2, mirroring how the root jobs takepr_root_runner.Why
The pool label is carried by root and side runners alike, so GitHub handed these light jobs to a root runner about half the time. From 06:30 to 09:00Z on 2026-09-25, 11 of 21 pool-label side-lane jobs ran on a root runner, 3,300 s of root-runner time, while the root label was the saturated one (16/16 busy with jobs queued). On cmux7s and cmux9s, which have a single root, a wrapper job there blocked the mini's only compile.
What this does not change (measured, same window)
#filePathis baked into the product, restore takes it withglaeda-canonical-root take --wait), and the shards and lag also hold the mini's one gui token. The root label is what keeps root users per mini within its roots; on a side runner they would wait or be refused, not run sooner.Related: #14411 routes swift-package-tests as a third owned side lane through
pr_runner; once both land it can takepr_side_runnerthe same way.Repo variables
None changed.
Tests
python3 -m unittest tests.test_ci_pr_runner_pool(154 OK), plus test_ci_change_areas, test_ci_owned_pool_rescue, test_ci_fork_runner_routing, test_runner_label_policy, test_seed_derived_data, test_run_e2e, test_reuse_app_host_products: OKbash tests/test_ci_self_hosted_guard.sh: 38 PASSactionlinton ci.yml, ci-macos.yml, remote-daemon.yml: clean🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Routes the owned Claude wrapper and remote-daemon macOS test lanes to side runners instead of root runners, and fixes the queue janitor to count side-label jobs out of a run's root-runner share.
Previously these light lanes took the pool label, which root runners also carry, landing on a root runner about half the time (11 of 21 in a measured window) and blocking a compile or product consumer off that mini's root. The picker now emits
side_runner(glaeda-side-<class>-xcode-<version>) when the pick has a root count andCI_OWNED_POOL_SLOTSleaves machines beyond the root runners; both lanes take it before the pool label on attempt 1 and on the rescue's attempt 2. The janitor'smarker_peaksnow subtracts side-label jobs too, so runs with side lanes no longer reserve up to 2 extra root runners.changesoutput path.Written for commit f64c0c8. Summary will update on new commits.