ci: optional compile-only pull request runs, and fail-fast merge groups - #13117
Merged
Merged
Conversation
…ps at the first failure A new changes output, full_suite, decides whether a run gets the whole macOS suite (app-host shards, package tests, the lag build, the Release build) or only compile admission. Merge groups and dispatches always get the suite. Pull requests get it too unless the repository variable CI_PULL_REQUEST_SUITE is "compile-only"; under that policy the full-ci label opts one pull request back in. Nothing changes until the variable is set, and it should be set together with enabling the merge queue, which then runs the suite on the commit that will land. The tests gate accepts a skipped suite only when full_suite is explicitly false, so a missing output cannot relax it. On merge groups the shard matrix fails fast, and a small job with actions: write and no checkout cancels the run at the first failed job. ci-status runs on cancellation and reports it, so the queue drops the entry without waiting for the remaining macOS jobs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
|
Warning Review limit reachedNext included review available in 26 seconds. View limit detailsLimit details: You’ve used all 10 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (3)
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review. 📝 WalkthroughWalkthroughChangesThe CI workflow now selects a full or compile-only macOS suite for pull requests. macOS jobs and the aggregate test gate use this selection. Merge-group runs stop shards and cancel after failures or timeouts. Tests cover routing, gating, and cancellation configuration. CI routing and execution control
Priority: ➖ Normal Estimated code review effort: 3 (Moderate) | ~25 minutes Change: Feature Sequence Diagram(s)sequenceDiagram
participant PullRequest
participant ChangesJob
participant GitHubAPI
participant ChooseCISuite
participant MacOSJobs
PullRequest->>ChangesJob: Trigger CI
ChangesJob->>GitHubAPI: Fetch labels
ChangesJob->>ChooseCISuite: Evaluate event, policy, and labels
ChooseCISuite-->>ChangesJob: Set full_suite
ChangesJob->>MacOSJobs: Route full or compile-only suite
🚥 Pre-merge checks | ✅ 24 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (24 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 11 functions across 2 files. (1 skipped: 1 unsupported.) ✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
The watcher holds actions: write. As a job in ci.yml it ran on merge_group from the queue's merge commit, so a queued pull request could edit the code holding that token. It is now its own workflow triggered by workflow_run, which always runs the default branch's copy, and ci.yml grants no write permission at all. It also stops when the CI run completes instead of inferring that from the job list, counts startup failures, and gives up visibly after ten consecutive Actions API errors instead of polling until its timeout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
teamleaderleo
enabled auto-merge (squash)
September 20, 2026 04:18
This was referenced Sep 20, 2026
teamleaderleo
added this pull request to the merge queue
Sep 20, 2026
This was referenced Sep 20, 2026
teamleaderleo
added this pull request to the merge queue
Sep 20, 2026
6 of 10 tasks
austinywang
removed this pull request from the merge queue due to a manual request
Sep 20, 2026
rustybret
pushed a commit
to rustybret/bmux
that referenced
this pull request
Sep 20, 2026
9da8b07 ci: run package tests only for packages a change can affect (manaflow-ai#13118) 1cd76eb ci: stop pull requests evicting the main cache seeds, and add an optional Warp cache store (manaflow-ai#13160) bd65a8a ci: optional compile-only pull request runs, and fail-fast merge groups (manaflow-ai#13117) 5517d3d test: preserve native terminal scrollbar visibility (manaflow-ai#12977) aa45187 Clarify shared Max RAM and vCPU allowance (manaflow-ai#13159)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nothing changes when this merges. It adds a switch, off by default, that lets pull requests run compile admission only and leaves the full macOS suite to the merge queue.
Why
Last 7 days of
ci.ymlpull-request runs: 1,111 runs for about 230 merges (74 succeeded, 433 failed, 604 cancelled by a newer push). About 80% of Mac time, roughly 53,000 Mac-minutes a week, goes to commits that never merge (shares estimated from 25 sampled runs per bucket).A merge queue runs CI again on every queued pull request, so with today's workflow each macOS pull request would pay for the full suite twice. With
compile-only, a merged pull request costs one full suite on the commit that lands plus about 21 Mac-minutes per push: about today's spend, andmaincan no longer break from two pull requests that each passed alone.The cost: test failures show up when a pull request is queued, not on each push, and a flaky shard now costs a queue slot.
Summary
CI_PULL_REQUEST_SUITE=compile-onlymakes pull requests skip the app-host shards, package tests, lag build and Release build. Merge groups and dispatches always run everything. Unset (today), pull requests run everything.full-cilabel restores the full suite for one pull request; labels are read live, so label and re-run.testsgate accepts a skipped suite only whenfull_suiteis explicitlyfalse.merge-group-fail-fast.ymlcancels a merge group's run at the first failed job so the queue drops it without waiting on the remaining Macs. It needsactions: write, so it runs fromworkflow_runon the default branch's copy (a queued pull request cannot edit it), checks out nothing, and uses a GitHub-hosted runner.ci.ymlgets no write permission.Rollout (maintainers)
CI_PULL_REQUEST_SUITE=compile-only, create thefull-cilabel, and add "Require merge queue" to the ruleset "main: block force-push or delete" (squash, status check timeout 360 min, merge limits min 1 / max 5, build concurrency 3, only merge non-failing pull requests).Testing
tests/test_ci_change_areas.py(Python 3.9 and 3.12): runs the realtestsgate script against compile-only, full, failed and missing-output inputs, covers the policy for every event, and asserts the fail-fast job never checks out code and is not aci-statusdependency. Pass.test_ci_self_hosted_guard.sh,test_ci_merge_queue_required_checks.py,test_ci_reusable_workflow_permissions.py,actionlint: pass.gh: failure on the second page of jobs (cancels), completed run (exits), API down (errors after ten tries).Related: #13095
🤖 Generated with Claude Code