ci: call a missing pinned Xcode a machine failure

[teamleaderleo]

A mini without the pinned Xcode could take a CI job, fail at Select Xcode, and leave the failure unattributed. CI now identifies that case as a machine failure and requeues an owned runner's failed job at the earliest point GitHub permits. Operators can check every pin on each mini with the new audit; the HQ fleet command is in cmuxterm-hq#984.

The selector emits one error with the runner name, installed Xcodes, and xcode-pin-missing. Both failure classifiers recognize that marker and the older messages from pre-fix PR heads. The owned pool rescue treats failed Select Xcode and Select helper Xcode steps as refusals even after a helper build or a long job. The audit checks pool versions from xcode-pins.txt and every configured CMUX_CI_XCODE_APP_* path against xcodebuild -version.

Validation: the new rescue regression failed before the fix and passes after it. Focused classifier, rescue, selection, and audit tests pass; python3 scripts/verify-local.py --all passed all 15 selected checks. The test execution registry now runs the audit test in the Linux guard lane. A live audit through the HQ branch reached all 15 minis: 13 pass; cmux15 and cmux-mac-mini still lack Xcode 26.3 while the fleet copy is in progress. This machine installation is tracked outside this PR.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: b98b422a-4214-4420-a27c-e7b2a5c4b3f6

📥 Commits

Reviewing files that changed from the base of the PR and between 6093e59 and f299005.

📒 Files selected for processing (12)

• .github/workflows/ci-guards.yml
• scripts/ci/classify_failures.py
• scripts/ci/machine_failure.py
• scripts/ci/owned_pool_rescue.py
• scripts/ci/xcode_pin_audit.py
• scripts/select-ci-xcode.sh
• tests/test-execution.toml
• tests/test_ci_classify_failures.py
• tests/test_ci_machine_failure.py
• tests/test_ci_owned_pool_rescue.py
• tests/test_ci_xcode_pin_audit.py
• tests/test_ci_xcode_selection_fast_path.sh

 ____________________________________________
< 99 little bugs in the code... I filed 100. >
 --------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

[github-actions]

Merge receipt for f29900566c: every check was green at merge (25 verified; 25 skipped by policy). Full suite runs on main after merge.

[github-actions]

Dogfood tours of f2990056

sidebar-and-chrome-tour at f2990056, on its merge 159c5e94 that CI built: failure (run)

Failed: DogfoodScenarioUITests.swift:169: Failed to get matching snapshot: No matches found for first query match sequence: Descendants matching type Window, given input App element pid: 44032

Key frames of sidebar-and-chrome-tour at f299005

_{Tours are picked by the paths globs in dogfood/scenarios/*.json; a Dogfood-tours: a, b line in the description picks them instead (none turns this off). Look at every frame before merging: a green tour only means no step failed.}