ci: take the build-fleet host lock for nightly mini builds - #14233
Merged
Merged
Conversation
A std mini that runs the nightly producer is also a build-fleet controller worker. Build under /Users/Shared/cmux-build-fleet/bin/with-host-lock when it exists, so a nightly and a dev build take turns instead of splitting one 48 GB machine. Hosted and fork runs have no lock tool and build as before. A lock refusal (exit 75, free-disk floor) fails the producer with a clear error and the router falls back to Blacksmith. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 4 minutes. View limit detailsLimit details: You’ve used all 10 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
Say that the build seconds include any lock wait, word the exit 75 error for what it is, and say a lock wait falls back only when it overruns the execution budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rustybret
pushed a commit
to rustybret/bmux
that referenced
this pull request
Sep 24, 2026
cd7a4cf Prepare iOS 1.0.6 beta compatibility release (manaflow-ai#14112) 2d9b4e8 test: skip dead persistent-SSH restore tests and fix relay-less legacy fixtures after manaflow-ai#14216 (manaflow-ai#14222) df44058 ci: run focused cmuxTests against products CI already compiled (manaflow-ai#14229) 06ec6cb Stop unrelated defaults writes and pane geometry changes from re-evaluating chrome-heavy views (manaflow-ai#14058) 185d99e chore(cli): remove dead persistent SSH PTY startup path (manaflow-ai#14231) dddffea ci: take the build-fleet host lock for nightly mini builds (manaflow-ai#14233) f2106e5 test(cli): expect the client-side workspace ref resolution manaflow-ai#13964 added (manaflow-ai#14230) 59fa5b9 docs(ci): fix persistent-compile pilot runbook drift (manaflow-ai#14206) # Conflicts: # .github/workflows/app-host-test-rerun.yml # .github/workflows/nightly-mini-build.yml
teamleaderleo
added a commit
that referenced
this pull request
Sep 24, 2026
…14243) Reverts the owned-Mac nightly route (#14208, #14223, #14233). There is no separate nightly lane or fallback: nightlies build on Blacksmith until Glaeda routing (glaeda#1174) sends every job std > light > Blacksmith > GitHub-hosted. nightly.yml is back to its pre-lane Blacksmith path, keeping the later nightly Sparkle key change (#14215). Signing, notarization and publication are unchanged. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The nightly lane's runner is now on cmux15, a 48 GB M4 Pro (
cmux15-nightlyincmux-nightly-mini). cmux15 is also a worker for the build-fleet controller, which can start a dev build while a nightly is compiling. Both would then run at once and split the machine.The controller serializes its own builds with
/Users/Shared/cmux-build-fleet/bin/with-host-lock(a flock on/Users/Shared/cmux-build-fleet/host.lock). The producer now runs its xcodebuild under that same lock when the tool exists, so a nightly and a dev build take turns.bin/with-host-lock, notrecipes/apfs-with-host-lock.recipesis a symlink, and its lock path doesn't resolve.producer_failure, and the nightly compiles on Blacksmith.NIGHTLY_MAC_MINI_EXECUTION_SECONDS.docs/ci/mac-fleet.mdsays so.${lock[@]+"${lock[@]}"}, which is safe underset -u. Checked with/bin/bash.Lock details come from the fleet manifest session, which checked the live workers (cmux15, cmux13s) read-only.
Tests:
tests/test_ci_self_hosted_guard.sh,tests/test_nightly_mini_route.py, actionlint.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Nightly mini builds now take the build-fleet host lock, so a nightly and a dev build take turns on the shared 48 GB runner instead of splitting it. Hosted and fork runs have no lock tool and build exactly as before, and a lock refusal (exit 75, free-disk floor) fails the producer with a clear error so the nightly falls back to Blacksmith.
/Users/Shared/cmux-build-fleet/bin/with-host-lockwhen the tool exists, usingbin/not therecipes/symlink, whose lock path doesn't resolve.NIGHTLY_MAC_MINI_EXECUTION_SECONDS; it falls back only when the wait overruns the budget, and reported build seconds include the wait.Written for commit 8622dd2. Summary will update on new commits.