Skip to content

Re-apply Blacksmith macOS CI/CD runner migration - #4984

Merged
lawrencecchen merged 4 commits into
mainfrom
feat-blacksmith-runners-retry
May 29, 2026
Merged

lawrencecchen merged 4 commits into
mainfrom
feat-blacksmith-runners-retry

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented May 29, 2026 •

Copy link
Copy Markdown
Contributor

Re-applies the WarpBuild/Depot → Blacksmith migration from #4902, which was reverted by #4926.

The revert was purely a Blacksmith macOS capacity problem, not a code defect: at the time every Blacksmith macOS SKU sat queued 8m+ in a probe and 65m+ on live main CI, while GitHub-hosted runners picked up in under 10s, so every main push got CANCELLED.

Capacity is back. A push-triggered probe on this branch ran trivial jobs on all three Blacksmith macOS SKUs and they picked up in ~16s and finished successfully:

SKU pickup result
blacksmith-6vcpu-macos-15 ~16s success
blacksmith-6vcpu-macos-26 ~17s success
blacksmith-6vcpu-macos-latest ~16s success

What changed

Every macOS job moves to blacksmith-6vcpu-macos-{15,26,latest} (1:1 with the former 6x Warp jobs, macOS 15 vs 26 preserved):

  • ci.yml — tests, tests-build-and-lag, release-build, ui-regressions
  • build-ghosttykit.yml, nightly.yml, release.yml, test-depot.yml, tmux-corpus.yml (also drops the self-hosted label), ci-macos-compat.yml (matrix os)
  • perf-activation.yml — default + dispatch choices → Blacksmith, SPM cache keys scoped by runner, and the Cmd-Tab activation bench re-wrapped in launchctl asuser so it runs in the console Aqua session (Blacksmith runners run in launchd's system bootstrap, where CGWindowListCopyWindowInfo([.optionOnScreenOnly]) can't see on-screen windows)
  • test-e2e.yml — default → blacksmith-6vcpu-macos-15, keeps depot-macos-* as fallback dispatch options. The existing Depot identity guard now correctly skips on the Blacksmith default and still validates explicit Depot runs.
  • .github/actionlint.yaml — re-adds the Blacksmith label allowlist (keeps the Depot labels test-e2e still references)
  • tests/test_ci_self_hosted_guard.sh — asserts Blacksmith macOS runners on the paid jobs ([Security] HIGH-1: Self-hosted CI runner exposed to fork pull requests #385)

The temporary capacity probe used to gate this re-attempt is removed.

Test plan

  • ./tests/test_ci_self_hosted_guard.sh passes against this tree
  • Probe confirms all three Blacksmith macOS SKUs pick up in ~16s
  • This PR's macOS CI goes green on Blacksmith without queuing

🤖 Generated with Claude Code


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.


Note

Medium Risk
Changes which provider runs release/nightly and main CI and adds launchctl/sudo for perf benchmarks; mis-set repo variables or runner env differences could cause queueing or flaky GUI/visibility tests.

Overview
Routes all paid macOS CI/CD jobs through repo variables MACOS_RUNNER_15 and MACOS_RUNNER_26, with WarpBuild labels as workflow fallbacks when variables are unset—so Blacksmith vs Warp can be flipped via gh variable set/delete without a code change (docs/macos-ci-runners.md).

Workflow updates: ci.yml, release/nightly, ghosttykit, compat matrix, test-depot, and tmux-corpus (drops self-hosted label) use vars.MACOS_RUNNER_* || 'warp-…'. perf-activation.yml and test-e2e.yml add runner: auto (follows MACOS_RUNNER_15), Blacksmith/Warp dispatch options, runner-scoped SPM cache keys, and updated concurrency/run-name expressions; Depot choices and identity guard remain for explicit depot-macos-* runs.

Blacksmith-specific fix: Cmd-Tab bench in perf-activation.yml runs under launchctl asuser the console user so window visibility checks work outside the runner’s system bootstrap.

Tooling: .github/actionlint.yaml allowlists Blacksmith labels; tests/test_ci_self_hosted_guard.sh now requires vars.MACOS_RUNNER_* or Blacksmith/Warp labels (not Warp-only).

Reviewed by Cursor Bugbot for commit 2a06dc9. Bugbot is set up for automated code reviews on this repo. Configure here.


Summary by cubic

Re-applies the macOS CI/CD runner migration to Blacksmith and routes all macOS jobs through repo variables for fast Blacksmith↔WarpBuild switching. Capacity is back; Blacksmith SKUs pick up in ~16s, so main CI is unblocked.

  • Migration
    • Route all macOS jobs via MACOS_RUNNER_15/MACOS_RUNNER_26 with WarpBuild fallback; repo vars point to blacksmith-6vcpu-macos-{15,26} in ci.yml, build-ghosttykit.yml, ci-macos-compat.yml, nightly.yml, release.yml, test-depot.yml, and tmux-corpus.yml. Update .github/actionlint.yaml to allow Blacksmith (incl. blacksmith-6vcpu-macos-latest) and keep Warp/Depot.
    • test-e2e.yml: default runner is auto and resolves to vars.MACOS_RUNNER_15 then Warp; keeps depot-macos-* options and the identity guard; scope SPM cache keys and run metadata by the resolved runner.
    • perf-activation.yml: default runner is auto and resolves to vars.MACOS_RUNNER_15 then Warp; scope SPM cache keys by runner, and run the Cmd-Tab bench via launchctl asuser for Aqua visibility.
    • Guard: tests/test_ci_self_hosted_guard.sh now enforces use of vars.MACOS_RUNNER_* or a Blacksmith/Warp label (and updates the Depot identity assertion). Remove the temporary capacity probe.
    • Docs: docs/macos-ci-runners.md documents the switch (repo variables, no PR) and how manual perf/e2e runs follow auto → repo variable → Warp.

Written for commit 2a06dc9. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Chores
    • CI workflows now use configurable macOS runner variables with sensible fallbacks instead of fixed runner labels.
    • Workflow dispatch inputs and cache keys updated to respect the chosen runner, improving flexibility for different macOS environments.
    • CI guard tests enhanced to validate multiple paid macOS runner providers.
    • Documentation added explaining how to switch and manage macOS CI runners.

Review Change Stack

lawrencecchen and others added 2 commits May 29, 2026 00:48
Push-triggered probe (scoped to this branch) to check whether Blacksmith
macOS runners pick up jobs before re-attempting the migration reverted in
#4926. Delete before merge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Re-applies the WarpBuild/Depot -> Blacksmith migration that landed as
#4902 and was reverted by
#4926. The revert was purely a
Blacksmith macOS capacity problem (every SKU queued 8m+/65m+ while
GitHub-hosted picked up in <10s), not a defect. A push-triggered probe on
this branch confirmed all three Blacksmith macOS SKUs (15/26/latest) now
pick up jobs in ~16s, so the migration is safe to re-attempt.

Switches every macOS job to blacksmith-6vcpu-macos-{15,26,latest}:
- ci.yml (tests, tests-build-and-lag, release-build, ui-regressions)
- build-ghosttykit, nightly, release, test-depot, perf-activation,
  tmux-corpus (drops the self-hosted label), ci-macos-compat
- test-e2e default -> blacksmith-6vcpu-macos-15; keeps depot-macos-* as
  fallback dispatch options and the Depot identity guard (now correctly
  skipped on the Blacksmith default)
- re-adds .github/actionlint.yaml allowlist for the Blacksmith labels
- re-adds the launchctl asuser Aqua-session wrapper for the Cmd-Tab perf
  bench (Blacksmith runners run in launchd's system bootstrap)
- tests/test_ci_self_hosted_guard.sh asserts Blacksmith macOS runners

Removes the temporary capacity probe used to gate this re-attempt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented May 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment May 29, 2026 8:56am
cmux-staging Building Building Preview, Comment May 29, 2026 8:56am

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented May 29, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Workflows and CI validation now resolve macOS runners via repo variables MACOS_RUNNER_15/MACOS_RUNNER_26 with WarpBuild fallbacks; actionlint allowlist, multiple workflows, perf benchmark invocation, cache keys, docs, and the CI guard script were updated to match the new resolution.

Changes

Parameterized macOS Runner Selection

Layer / File(s) Summary
Actionlint self-hosted runner allowlist
.github/actionlint.yaml
Actionlint configuration now recognizes Blacksmith macOS runner label patterns (blacksmith-6vcpu-macos-15, blacksmith-6vcpu-macos-26, blacksmith-6vcpu-macos-latest) as valid self-hosted runners.
Runner variable configuration and usage documentation
docs/macos-ci-runners.md
Documentation introduces MACOS_RUNNER_15 and MACOS_RUNNER_26 repository variables, explains the fallback resolution pattern in workflow expressions, provides instructions for switching between runner providers via variable set/delete operations, and describes how manual workflow inputs interact with and override repo variables.
Standard workflow runner variable migrations
.github/workflows/build-ghosttykit.yml, .github/workflows/ci.yml, .github/workflows/ci-macos-compat.yml, .github/workflows/nightly.yml, .github/workflows/release.yml, .github/workflows/test-depot.yml, .github/workflows/tmux-corpus.yml
Nine workflows replace hardcoded macOS runner labels in runs-on and matrix os fields with variable expressions: `${{ vars.MACOS_RUNNER_15
E2E workflow input configuration and cache keys
.github/workflows/test-e2e.yml
Workflow name, runner input defaults and options, concurrency grouping, job runs-on, and Swift package cache keys are updated to use the variable-based macOS 15 runner expression instead of hardcoded Depot runner defaults. Input selection takes precedence over repository variables.
Performance activation workflow: runner configuration and benchmark execution
.github/workflows/perf-activation.yml
Runner input choices are expanded to include Blacksmith variants; runs-on resolution prioritizes manual input, then repository variable, then fallback; cache key incorporates the selected runner; Cmd-Tab benchmark execution is reworked to run under the console user's Aqua session via launchctl asuser with explicit PATH and DEVELOPER_DIR environment setup.
CI self-hosted guard script validation updates
tests/test_ci_self_hosted_guard.sh
check_warp_runner function is replaced with check_macos_runner to validate that macOS jobs reference repository variables or approved paid runner labels; E2E fallback condition is broadened to include variable routing; function call sites across workflows are rewired.

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly Related PRs

  • manaflow-ai/cmux#4926: Continues macOS runner configuration by updating workflow runner labels and the shared self-hosted guard validation script.
  • manaflow-ai/cmux#4902: Establishes initial Blacksmith macOS runner migration and introduces the perf-activation benchmark execution pattern with console-user session re-entry.
  • manaflow-ai/cmux#4922: Overlaps on macOS runner fallback logic and CI guard changes affecting test-e2e.yml and related validations.

Poem

🐰 I hopped through YAML, labels in paw,
Repo vars guide where runners now draw,
Blacksmith or Warp, fallbacks hum bright,
CI sleeps soundly through day and night. 🥕

🚥 Pre-merge checks | ✅ 17 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (17 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: re-applying the Blacksmith macOS CI/CD runner migration, which is the primary objective of this PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed PR contains no production Swift code changes—only GitHub Actions workflows, config files, documentation, and shell scripts. The actor isolation check is not applicable.
Cmux Swift Blocking Runtime ✅ Passed PR introduces no blocking/timing synchronization in production Swift code; changes are primarily GitHub Actions workflow configurations and CI infrastructure updates for runner migration.
Cmux No Hacky Sleeps ✅ Passed PR violates no-hacky-sleeps rule: workflow YAML is out of scope; shell script changes add no sleeps; Swift changes covered by separate rule.
Cmux Algorithmic Complexity ✅ Passed PR contains only GitHub Actions workflow configuration, linting rules, documentation, and CI test scaffolding—not production code subject to algorithmic complexity review.
Cmux Swift Concurrency ✅ Passed This PR modifies only GitHub Actions workflows, configuration, documentation, and shell scripts—zero Swift code changes. The custom check targets "cmux-owned Swift code" and is not applicable here.
Cmux Swift @Concurrent ✅ Passed PR contains no Swift source code changes; all changes are in GitHub Actions workflow YAML, shell scripts, and documentation. The @concurrent annotation check is not applicable.
Cmux Swift File And Package Boundaries ✅ Passed No Swift file changes detected. PR modifies GitHub Actions workflow YAML, shell scripts, and documentation only. The Swift file/package boundaries check does not apply.
Cmux Swift Logging ✅ Passed No production Swift source code changes in this PR—only CI/CD workflows, configuration, documentation, and shell test scripts. The logging rule applies only to Swift code, which is not modified.
Cmux User-Facing Error Privacy ✅ Passed All changes are in CI/CD infrastructure, tests, and docs—not production code. Error message in test-e2e.yml is in a workflow (operational runbook) shown only to developers, which the rule allows.
Cmux Full Internationalization ✅ Passed PR contains only CI/CD config, workflows, operational docs, and test scripts—no user-facing strings or i18n changes required.
Cmux Swiftui State Layout ✅ Passed PR contains no SwiftUI/Swift source code changes; it only modifies GitHub Actions workflows, CI configuration, documentation, and test scripts. SwiftUI state layout check is not applicable.
Cmux Architecture Rethink ✅ Passed PR contains no Swift source changes—only GitHub Actions YAML, documentation, and test scripts. The architectural review rule applies to Swift code changes, not CI infrastructure configuration.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed PR contains only GitHub Actions workflow YAML, configuration, documentation, and shell script changes—zero Swift source code modifications, so the Swift window close shortcuts check is not applicable.
Description check ✅ Passed The PR description comprehensively covers all required sections: summary of changes, rationale (capacity problem resolved), what changed (detailed workflow updates), test plan with checkmarks, and includes supporting documentation updates.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat-blacksmith-runners-retry

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented May 29, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR re-applies the WarpBuild → Blacksmith macOS runner migration after a temporary capacity-driven revert. The core change routes all paid macOS CI jobs through two repository variables (MACOS_RUNNER_15 / MACOS_RUNNER_26) with WarpBuild as a hard-coded fallback, so the active provider can be switched without a code change.

  • Runner indirection: Every runs-on: warp-macos-* across ci.yml, nightly.yml, release.yml, build-ghosttykit.yml, test-depot.yml, ci-macos-compat.yml, and tmux-corpus.yml is replaced with ${{ vars.MACOS_RUNNER_{15,26} || 'warp-macos-*' }}.
  • perf-activation.yml / test-e2e.yml: Default runner changed to auto (follows the repo variable); Blacksmith and Warp labels added as explicit dispatch choices; SPM cache keys are scoped per-runner; the Cmd-Tab activation bench is re-wrapped in launchctl asuser to run in the console Aqua session so CGWindowListCopyWindowInfo sees on-screen windows on Blacksmith.
  • Guard & docs: test_ci_self_hosted_guard.sh updated to accept vars.MACOS_RUNNER_*, Blacksmith, or Warp labels; .github/actionlint.yaml allowlists the new labels; new docs/macos-ci-runners.md documents the switch procedure.

Confidence Score: 5/5

Safe to merge — changes are limited to CI runner selection and tooling; no application code, signing logic, or production behaviour is touched.

All twelve changed files are CI/CD workflow YAML, a shell guard script, and documentation. Every macOS job gains a vars.MACOS_RUNNER_*-based fallback chain so the provider can be flipped without a commit. The launchctl asuser wrapping in perf-activation.yml is correctly scoped to the Aqua-session visibility problem on Blacksmith. The Depot identity guard in test-e2e.yml was updated in lockstep with the runner-resolution expression and still fires correctly on explicit Depot selections. No logic regressions were found.

No files require special attention.

Important Files Changed

Filename Overview
.github/workflows/ci.yml Four runs-on values updated from hardcoded Warp labels to vars.MACOS_RUNNER_{15,26} with Warp fallback; no logic changes.
.github/workflows/perf-activation.yml Default runner changed to auto, SPM cache keys scoped by runner label, and Cmd-Tab bench re-wrapped under launchctl asuser for Aqua session access on Blacksmith; all changes are well-reasoned CI-only adjustments.
.github/workflows/test-e2e.yml Default runner changed to auto (resolves via MACOS_RUNNER_15); Blacksmith/Warp dispatch choices added; Depot identity guard, SPM cache scoping, concurrency group, and run-name all updated to use the same resolver expression consistently.
.github/workflows/ci-macos-compat.yml Matrix os values now resolve via vars.MACOS_RUNNER_{15,26} expressions with Warp fallback; matrix.os propagates correctly to runs-on and cache keys.
tests/test_ci_self_hosted_guard.sh Guard refactored from check_warp_runner to check_macos_runner; AWK pattern broadened to accept vars.MACOS_RUNNER_*, Blacksmith, or Warp labels; Depot identity-guard assertion updated to match the new resolver expression.
docs/macos-ci-runners.md New doc describing the repo-variable switch mechanism, gh variable commands for Blacksmith/Warp, and the guard invariant.
.github/workflows/tmux-corpus.yml Runner updated from [self-hosted, warp-macos-15-arm64-6x] array to vars.MACOS_RUNNER_15 expression; redundant self-hosted label correctly dropped.
.github/actionlint.yaml Blacksmith SKU labels added to the self-hosted allowlist; existing Warp and Depot labels retained.
.github/workflows/build-ghosttykit.yml Single runs-on updated to vars.MACOS_RUNNER_15 with Warp fallback; no other changes.
.github/workflows/nightly.yml Single runs-on updated to vars.MACOS_RUNNER_26 with Warp fallback; no other changes.
.github/workflows/release.yml Single runs-on updated to vars.MACOS_RUNNER_26 with Warp fallback; no other changes.
.github/workflows/test-depot.yml Single runs-on updated to vars.MACOS_RUNNER_15 with Warp fallback; no other changes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Workflow triggered] --> B{inputs.runner set\nand not 'auto'?}
    B -- Yes --> C[Use inputs.runner directly\ne.g. blacksmith-6vcpu-macos-15\nwarp-macos-15-arm64-6x\ndepot-macos-latest]
    B -- No --> D{vars.MACOS_RUNNER_15\nrepo variable set?}
    D -- Yes --> E[Use vars.MACOS_RUNNER_15\ne.g. blacksmith-6vcpu-macos-15]
    D -- No --> F[Fallback to hardcoded\nwarp-macos-15-arm64-6x]
    C --> G{starts with\ndepot-macos-?}
    E --> G
    F --> G
    G -- Yes --> H[Run Depot identity guard\nvalidate runner.name]
    G -- No --> I[Proceed with job]
    H --> I
Loading

Reviews (3): Last reviewed commit: "ci: make manual perf/e2e runner default ..." | Re-trigger Greptile

@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/test_ci_self_hosted_guard.sh (1)

33-33: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Update awk patterns to check for the new default runner.

Lines 33 and 40 still check for depot-macos-latest, but the E2E workflow default was changed to blacksmith-6vcpu-macos-15 (as reflected in line 61). Since these awk checks validate that the run-name and concurrency grouping include the runner with its default fallback, they should check for the new default rather than the old one.

🔧 Proposed fix to align with the new default
    /^run-name:/ {
      saw_run_name=1
-      if ($0 ~ /inputs\.test_filter/ && ($0 ~ /inputs\.runner/ || $0 ~ /depot-macos-latest/) && ($0 ~ /inputs\.ref/ || $0 ~ /github\.ref_name/)) {
+      if ($0 ~ /inputs\.test_filter/ && ($0 ~ /inputs\.runner/ || $0 ~ /blacksmith-6vcpu-macos-15/) && ($0 ~ /inputs\.ref/ || $0 ~ /github\.ref_name/)) {
        saw_run_name_dynamic=1
      }
    }
    /^concurrency:/ { in_concurrency=1; next }
    in_concurrency && /^jobs:/ { in_concurrency=0 }
    in_concurrency && /cancel-in-progress:[[:space:]]*true/ { saw_cancel=1 }
-    in_concurrency && (/inputs\.runner/ || /depot-macos-latest/) { saw_runner=1 }
+    in_concurrency && (/inputs\.runner/ || /blacksmith-6vcpu-macos-15/) { saw_runner=1 }
    in_concurrency && /inputs\.test_filter/ { saw_test_filter=1 }

Also applies to: 40-40

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_ci_self_hosted_guard.sh` at line 33, Update the awk pattern checks
that still look for the old runner string "depot-macos-latest" to the new
default "blacksmith-6vcpu-macos-15" in the conditional that matches
inputs.test_filter/inputs.runner/github.ref_name (the if condition containing $0
~ /inputs\.test_filter/ && ($0 ~ /inputs\.runner/ || $0 ~ /depot-macos-latest/)
&& ($0 ~ /inputs\.ref/ || $0 ~ /github\.ref_name/)); change both occurrences
(lines checking for the runner in that awk expression and the similar one at the
other occurrence) so the fallback runner match uses /blacksmith-6vcpu-macos-15/
instead of /depot-macos-latest/.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/perf-activation.yml:
- Around line 142-153: The workflow step assumes an interactive Aqua session and
uses sudo/launchctl asuser which will fail on Blacksmith macOS runners; update
the step that invokes scripts/bench-window-visibility.swift to (1) detect
whether an interactive Aqua session exists (e.g., check if /dev/console user
equals the current user and/or whether launchctl asuser succeeds) and if not
skip or mark the bench as Unsupported on this runner, (2) avoid unconditional
sudo -n & inner sudo -u usage — run the swift script as the current runner user
when possible or gate the sudo calls behind a conditional that ensures
passwordless sudo is configured, and (3) ensure APP_PATH/home layout is
consistent by only switching to CONSOLE_USER when CONSOLE_USER == "$(whoami)" or
when explicitly required; reference the invocation of launchctl asuser, sudo -n,
and scripts/bench-window-visibility.swift with flags --cmd-tab-activation and
--cg-visibility when implementing these guards.

---

Outside diff comments:
In `@tests/test_ci_self_hosted_guard.sh`:
- Line 33: Update the awk pattern checks that still look for the old runner
string "depot-macos-latest" to the new default "blacksmith-6vcpu-macos-15" in
the conditional that matches inputs.test_filter/inputs.runner/github.ref_name
(the if condition containing $0 ~ /inputs\.test_filter/ && ($0 ~
/inputs\.runner/ || $0 ~ /depot-macos-latest/) && ($0 ~ /inputs\.ref/ || $0 ~
/github\.ref_name/)); change both occurrences (lines checking for the runner in
that awk expression and the similar one at the other occurrence) so the fallback
runner match uses /blacksmith-6vcpu-macos-15/ instead of /depot-macos-latest/.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 71dd8b45-4bfe-4560-aae5-cd2c25fd9b49

📥 Commits

Reviewing files that changed from the base of the PR and between bc35d13 and b3a6da0.

📒 Files selected for processing (11)
  • .github/actionlint.yaml
  • .github/workflows/build-ghosttykit.yml
  • .github/workflows/ci-macos-compat.yml
  • .github/workflows/ci.yml
  • .github/workflows/nightly.yml
  • .github/workflows/perf-activation.yml
  • .github/workflows/release.yml
  • .github/workflows/test-depot.yml
  • .github/workflows/test-e2e.yml
  • .github/workflows/tmux-corpus.yml
  • tests/test_ci_self_hosted_guard.sh

Comment thread .github/workflows/perf-activation.yml
Make the Blacksmith<->WarpBuild switch a one-step repo-variable change with no
PR, so a future Blacksmith macOS capacity outage (the cause of the #4926 revert)
can be reverted instantly.

Every paid macOS job now uses:
  runs-on: ${{ vars.MACOS_RUNNER_15 || 'warp-macos-15-arm64-6x' }}   (and _26)
An unset variable falls back to WarpBuild, so a missing var never breaks CI.
The repo variables are set to the blacksmith-6vcpu-macos-{15,26} labels.

- ci.yml, build-ghosttykit, nightly, release, test-depot, tmux-corpus,
  ci-macos-compat (matrix os) route runs-on through the vars
- perf-activation / test-e2e defaults become
  inputs.runner || vars.MACOS_RUNNER_15 || 'warp-...'; explicit dispatch
  choices and the Depot identity guard are unchanged
- test_ci_self_hosted_guard.sh now asserts each paid job references
  vars.MACOS_RUNNER_* or a Blacksmith/Warp label (never a free GitHub-hosted
  runner), the real intent of issue #385
- actionlint allowlist keeps the Warp fallback labels alongside Blacksmith/Depot
- docs/macos-ci-runners.md documents the switch procedure

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 316e73b. Configure here.

CONSOLE_UID="$(id -u "$CONSOLE_USER")"
sudo -n launchctl asuser "$CONSOLE_UID" sudo -n -u "$CONSOLE_USER" -E \
env PATH="$PATH" DEVELOPER_DIR="$DEVELOPER_DIR" \
bash -c "cd '$PWD' && swift scripts/bench-window-visibility.swift '$APP_PATH' 'com.cmuxterm.app.debug.$PERF_TAG' 30 --cmd-tab-activation --cg-visibility" \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unconditional sudo with no fallback unlike test-e2e pattern

Low Severity

The launchctl asuser wrapper unconditionally requires sudo -n without first checking availability, unlike the robust pattern in test-e2e.yml which guards with if sudo -n true 2>/dev/null and falls back to direct execution with a warning. The stat -f %Su /dev/console call also lacks 2>/dev/null || true error suppression and the $CONSOLE_USER != "root" safety check. Since perf-activation.yml triggers on pull_request and can fall back to WarpBuild runners (when vars.MACOS_RUNNER_15 is unset), this step will hard-fail on any runner without passwordless sudo, where the old direct swift invocation would have succeeded.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 316e73b. Configure here.

Codex review caught that perf-activation.yml and test-e2e.yml set the
workflow_dispatch `runner` input default to a blacksmith literal. GitHub
populates inputs.runner with that default, so inputs.runner || vars.MACOS_RUNNER
never reached the repo variable on manual runs, and there was no Warp option to
pick during an outage. Flipping the variable would not have redirected manual
runs.

Default the input to 'auto' and resolve it (and the empty pull_request case) to
vars.MACOS_RUNNER_15 then the warp fallback:
  (!inputs.runner || inputs.runner == 'auto')
    && (vars.MACOS_RUNNER_15 || 'warp-macos-15-arm64-6x')
    || inputs.runner
Add warp-macos-15/26 dropdown options so an operator can also select Warp
directly. Update the guard's e2e identity-guard assertion and the runner docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/perf-activation.yml (1)

138-154: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Bench step hardcodes the Blacksmith-specific launchctl asuser wrapper, but runs-on can still resolve to the warp fallback.

On pull_request events inputs.runner is empty, so Line 39 resolves to vars.MACOS_RUNNER_15 || 'warp-macos-15-arm64-6x'. When vars.MACOS_RUNNER_15 is unset the job lands on warp-macos-15-arm64-6x, yet this step now unconditionally re-enters via sudo -n launchctl asuser … sudo -n -u … (per the AI summary, this replaced a direct swift … invocation). If the warp fallback doesn't provide passwordless sudo plus a console Aqua session, sudo -n fails immediately and takes the whole job down on every PR in that state.

The Blacksmith path is verified working — this is strictly about the warp fallback that the PR intentionally keeps. Gating the wrapper on the resolved runner preserves the previous direct-invocation behavior off Blacksmith:

🛡️ Proposed fix: gate the Aqua re-entry to Blacksmith runners
       - name: Run Cmd-Tab activation benchmark
+        env:
+          RESOLVED_RUNNER: ${{ inputs.runner || vars.MACOS_RUNNER_15 || 'warp-macos-15-arm64-6x' }}
         run: |
           set -euo pipefail
           APP_PATH="$HOME/Library/Developer/Xcode/DerivedData/cmux-$PERF_TAG/Build/Products/Debug/cmux DEV $PERF_TAG.app"
-          # The GitHub Actions runner process runs in launchd's system
-          # bootstrap, not the console user's Aqua session, so windows
-          # created by apps launched via NSWorkspace.openApplication do
-          # not show up in CGWindowListCopyWindowInfo([.optionOnScreenOnly]).
-          # Re-enter the Aqua session via launchctl asuser before running
-          # the bench so visibility polling sees real on-screen windows.
-          CONSOLE_USER="$(stat -f %Su /dev/console)"
-          CONSOLE_UID="$(id -u "$CONSOLE_USER")"
-          sudo -n launchctl asuser "$CONSOLE_UID" sudo -n -u "$CONSOLE_USER" -E \
-            env PATH="$PATH" DEVELOPER_DIR="$DEVELOPER_DIR" \
-            bash -c "cd '$PWD' && swift scripts/bench-window-visibility.swift '$APP_PATH' 'com.cmuxterm.app.debug.$PERF_TAG' 30 --cmd-tab-activation --cg-visibility" \
-            > perf-results/cmd-tab-activation.txt
+          if [[ "$RESOLVED_RUNNER" == blacksmith-* ]]; then
+            # Blacksmith jobs run in launchd's system bootstrap, not the
+            # console user's Aqua session, so windows created via
+            # NSWorkspace.openApplication do not show up in
+            # CGWindowListCopyWindowInfo([.optionOnScreenOnly]). Re-enter the
+            # Aqua session via launchctl asuser so visibility polling sees them.
+            CONSOLE_USER="$(stat -f %Su /dev/console)"
+            CONSOLE_UID="$(id -u "$CONSOLE_USER")"
+            sudo -n launchctl asuser "$CONSOLE_UID" sudo -n -u "$CONSOLE_USER" -E \
+              env PATH="$PATH" DEVELOPER_DIR="$DEVELOPER_DIR" \
+              bash -c "cd '$PWD' && swift scripts/bench-window-visibility.swift '$APP_PATH' 'com.cmuxterm.app.debug.$PERF_TAG' 30 --cmd-tab-activation --cg-visibility" \
+              > perf-results/cmd-tab-activation.txt
+          else
+            swift scripts/bench-window-visibility.swift "$APP_PATH" "com.cmuxterm.app.debug.$PERF_TAG" 30 --cmd-tab-activation --cg-visibility \
+              > perf-results/cmd-tab-activation.txt
+          fi
           cat perf-results/cmd-tab-activation.txt

To confirm whether the wrapper is even viable on the fallback (in case you'd rather not branch):

Do WarpBuild macOS runners (warp-macos-15-arm64-6x) run jobs in a logged-in console Aqua session and provide passwordless sudo / launchctl asuser support?
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/perf-activation.yml around lines 138 - 154, The step
currently always uses the launchctl asuser + sudo wrapper (the long line
invoking sudo -n launchctl asuser "$CONSOLE_UID" sudo -n -u "$CONSOLE_USER" -E
bash -c "cd '$PWD' && swift scripts/bench-window-visibility.swift ..."), which
breaks on non-Blacksmith runners; change it to detect support and fall back to
the original direct swift invocation: use APP_PATH, CONSOLE_USER and CONSOLE_UID
as already computed, run a quick probe like sudo -n launchctl asuser
"$CONSOLE_UID" true (or another non-destructive check) and if that exits zero
run the existing wrapper invocation, otherwise run bash -c "cd '$PWD' && swift
scripts/bench-window-visibility.swift '$APP_PATH'
'com.cmuxterm.app.debug.$PERF_TAG' 30 --cmd-tab-activation --cg-visibility"
without the launchctl/sudo re-entry so the job doesn’t fail on warp runners.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.github/workflows/perf-activation.yml:
- Around line 138-154: The step currently always uses the launchctl asuser +
sudo wrapper (the long line invoking sudo -n launchctl asuser "$CONSOLE_UID"
sudo -n -u "$CONSOLE_USER" -E bash -c "cd '$PWD' && swift
scripts/bench-window-visibility.swift ..."), which breaks on non-Blacksmith
runners; change it to detect support and fall back to the original direct swift
invocation: use APP_PATH, CONSOLE_USER and CONSOLE_UID as already computed, run
a quick probe like sudo -n launchctl asuser "$CONSOLE_UID" true (or another
non-destructive check) and if that exits zero run the existing wrapper
invocation, otherwise run bash -c "cd '$PWD' && swift
scripts/bench-window-visibility.swift '$APP_PATH'
'com.cmuxterm.app.debug.$PERF_TAG' 30 --cmd-tab-activation --cg-visibility"
without the launchctl/sudo re-entry so the job doesn’t fail on warp runners.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 7833ec58-4e88-49fc-bfeb-30afee289624

📥 Commits

Reviewing files that changed from the base of the PR and between bc35d13 and 316e73b.

📒 Files selected for processing (12)
  • .github/actionlint.yaml
  • .github/workflows/build-ghosttykit.yml
  • .github/workflows/ci-macos-compat.yml
  • .github/workflows/ci.yml
  • .github/workflows/nightly.yml
  • .github/workflows/perf-activation.yml
  • .github/workflows/release.yml
  • .github/workflows/test-depot.yml
  • .github/workflows/test-e2e.yml
  • .github/workflows/tmux-corpus.yml
  • docs/macos-ci-runners.md
  • tests/test_ci_self_hosted_guard.sh

This branch was successfully deployed

1 active deployment
Preview – cmux — 2a06dc9a Deployed May 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant