Skip to content

docs(ci-runners): correct the runner table and document the lane variables - #13667

Merged
teamleaderleo merged 1 commit into
mainfrom
docs/ci-runners-accuracy
Sep 22, 2026
Merged

teamleaderleo merged 1 commit into
mainfrom
docs/ci-runners-accuracy

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

What

Documentation only. Corrects docs/ci-runners.md, which tests/test_ci_self_hosted_guard.sh points people at as the reference for which runner a job lands on.

Three things were wrong

1. The "Active value" column had drifted. It asserted values for five variables that had not been set for days. A column claiming live state, in a file that cannot observe live state, will always rot — so it is now "Intended steady state", with a pointer to gh variable list --repo manaflow-ai/cmux for what is actually set.

The intended values are unchanged from what the file already documented, except MACOS_RUNNER_15 and MACOS_RUNNER_DISPLAY, which now name the Blacksmith pool rather than Tart labels. That matches the file's own "Tart isolation and capacity" section, which already says macOS 26 and release builds stay on Blacksmith until a Tahoe image passes the runner and GUI canaries.

2. Three variables were undocumented. MACOS_RUNNER_PR (#13658) and MACOS_RUNNER_TESTS (#13662) were added and never written down — my omission in both PRs. MACOS_RUNNER_STREAMED_VALIDATION has been set for days with no row at all, and its two consumers disagree on the fallback (ios-streamed-validate.yml falls back to macOS 26, iroh-release-gate.yml to macOS 15), which is now stated rather than left to be discovered.

A new Lanes section explains why macOS jobs do not all follow one variable:

  • Required CI on main, the merge queue, releases and nightly follow the MACOS_RUNNER_* variables. This is where a queued runner blocks a merge or a ship.
  • Pull requests resolve MACOS_RUNNER_PR first. They are cancelled on supersession by design, so they are the wrong place for elastic paid capacity.
  • Manual test debugging resolves MACOS_RUNNER_TESTS and deliberately does not follow MACOS_RUNNER_15.

3. The Blacksmith cutover recipe pointed at a retired provider. It set MACOS_RUNNER_DISPLAY to depot-macos-latest. Depot is retired for cmux CI — test-depot.yml says so in its own comment — and depot-macos-latest is not in .github/actionlint.yaml's label allowlist. Running that recipe as written would point the GUI lane at a pool that no longer answers. It now names the same Blacksmith label as the workflow fallback.

Also

Records that fork pull requests resolve MACOS_RUNNER_PR and MACOS_RUNNER_TESTS to empty, so they land on the Blacksmith fallback — the same runner they used before those variables existed. That property is easy to forget and easy to break.

Test

tests/test_ci_self_hosted_guard.sh passes. No workflow or script changes.

Related

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Fixes the CI runner reference in docs/ci-runners.md — the doc tests/test_ci_self_hosted_guard.sh points people at — which had a stale "Active value" column, left three lane variables undocumented, and named a retired provider in the cutover recipe.

The column now reads "Intended steady state" and points readers at gh variable list for what is actually set; MACOS_RUNNER_15 and MACOS_RUNNER_DISPLAY now name the Blacksmith pool rather than Tart labels. Adds rows and a Lanes section for MACOS_RUNNER_PR, MACOS_RUNNER_TESTS, and MACOS_RUNNER_STREAMED_VALIDATION, explaining that required CI on main, the merge queue, releases and nightly follow the MACOS_RUNNER_* variables while pull requests and manual test lanes stay on Blacksmith by default. The cutover recipe set MACOS_RUNNER_DISPLAY to depot-macos-latest, a retired provider not in the actionlint label allowlist; it now uses the same Blacksmith label as the workflow fallback. Records that fork pull requests resolve the two new variables to empty and land on Blacksmith, the runner they used before. Documentation only; no workflow or script changes.

Written for commit 26fc399. Summary will update on new commits.

Review in cubic

…ables

`docs/ci-runners.md` is what `tests/test_ci_self_hosted_guard.sh` points people
at, so it is the reference for which runner a job lands on. Three things in it
were wrong or missing.

**The "Active value" column had drifted.** It claimed values for five variables
that had not been set for days. A column that asserts live state in a file that
cannot observe live state will always rot, so replace it with "Intended steady
state" and point readers at `gh variable list` for what is actually set. The
intended values are unchanged from what the file already documented, except
that `MACOS_RUNNER_15` and `MACOS_RUNNER_DISPLAY` now name the Blacksmith pool
rather than Tart labels, matching the "Tart isolation and capacity" section
below which already says macOS 26 and release builds stay on Blacksmith until a
Tahoe image is qualified.

**Three variables were undocumented.** `MACOS_RUNNER_PR` and
`MACOS_RUNNER_TESTS` were added with the pull-request and manual-test lane
changes and never written down, and `MACOS_RUNNER_STREAMED_VALIDATION` has been
set for days with no row at all. A new "Lanes" section explains why macOS jobs
do not all follow one variable: required CI on main blocks a merge or a ship,
pull requests are cancelled on supersession by design, and re-running one test
to chase a flake should never reach elastic paid capacity.

**The Blacksmith cutover recipe pointed at a retired provider.** It set
`MACOS_RUNNER_DISPLAY` to `depot-macos-latest`. Depot is retired for cmux CI --
`test-depot.yml` says so in its own comment -- and `depot-macos-latest` is not
in `.github/actionlint.yaml`'s label allowlist, so running that recipe as
written would point the GUI lane at a pool that no longer answers. It now names
the same Blacksmith label as the workflow fallback.

Also notes that fork pull requests resolve the two new variables to empty and
therefore land on the Blacksmith fallback, which is the runner they used before
those variables existed.

Documentation only; no workflow or script changes.
`tests/test_ci_self_hosted_guard.sh` passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 4 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 18fc9e2b-6e76-4afa-871c-04b0796069dc

📥 Commits

Reviewing files that changed from the base of the PR and between be71faf and 26fc399.

📒 Files selected for processing (1)
  • docs/ci-runners.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@teamleaderleo
teamleaderleo merged commit 0bb5a5f into main Sep 22, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant