Skip to content

Use Blacksmith as macOS CI default - #5019

Closed
lawrencecchen wants to merge 7 commits into
mainfrom
feat-blacksmith-ci
Closed

lawrencecchen wants to merge 7 commits into
mainfrom
feat-blacksmith-ci

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented May 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make Blacksmith the checked-in default for paid macOS CI jobs when MACOS_RUNNER_15 / MACOS_RUNNER_26 are unset. Remove Warp runner choices from manual workflows and actionlint config. Update runner docs and the CI guard to treat Blacksmith as the supported default.

I also set the live repo variables to blacksmith-6vcpu-macos-15 and blacksmith-6vcpu-macos-26, so new main runs use Blacksmith immediately.

Queue check

Testing

  • tests/test_ci_self_hosted_guard.sh
  • actionlint -shellcheck= -config-file .github/actionlint.yaml .github/workflows/ci.yml .github/workflows/ci-macos-compat.yml .github/workflows/build-ghosttykit.yml .github/workflows/nightly.yml .github/workflows/release.yml .github/workflows/test-depot.yml .github/workflows/tmux-corpus.yml .github/workflows/perf-activation.yml .github/workflows/test-e2e.yml

Full actionlint with shellcheck still reports existing shellcheck findings in unrelated workflow scripts.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag @codesmith with what you need. Autofix is disabled.


Note

Medium Risk
Changes where all paid macOS builds and tests run (release, nightly, CI) with no repo-variable escape hatch; misconfiguration or Blacksmith outages require a workflow PR to fix.

Overview
Paid macOS CI/CD now hardcodes Blacksmith runner labels in workflows instead of routing through MACOS_RUNNER_15 / MACOS_RUNNER_26 with Warp fallbacks. Typical jobs use blacksmith-6vcpu-macos-15; release/CI app SDK validation and compat’s macOS 26 matrix leg use blacksmith-6vcpu-macos-26. Warp labels are removed from actionlint and from manual runner choices in perf-activation.yml and test-e2e.yml; auto and related run-name, concurrency, and SPM cache keys resolve to blacksmith-6vcpu-macos-15.

docs/macos-ci-runners.md is rewritten for git-tracked labels and PR-based provider changes. Guard scripts require Blacksmith labels only and assert the macOS 15 helper / macOS 26 app split on release and CI lanes.

Reviewed by Cursor Bugbot for commit 638a8c0. Bugbot is set up for automated code reviews on this repo. Configure here.


Summary by cubic

Hardcodes Blacksmith as the paid macOS CI/CD runner and removes the MACOS_RUNNER_15/MACOS_RUNNER_26 override variables. Manual “auto” choices, cache keys, and run names now resolve to blacksmith-6vcpu-macos-15/blacksmith-6vcpu-macos-26; Warp labels are removed.

  • Refactors

    • Replaced all runs-on fallbacks with Blacksmith labels in CI, release, nightly, compat matrix, e2e, perf, GhosttyKit, Depot tests, and tmux workflows.
    • Dropped repo-variable routing; perf-activation.yml and test-e2e.yml “auto” now uses Blacksmith; updated run names, concurrency groups, and SPM cache keys to match.
    • Removed Warp labels from .github/actionlint.yaml and manual runner inputs; kept depot-macos-* options and the Depot identity guard.
  • Docs & Tests

    • Rewrote docs/macos-ci-runners.md to state runners are hardcoded in git; switching providers requires a PR; added actionlint label guidance.
    • Tightened tests/test_ci_self_hosted_guard.sh to require Blacksmith labels; updated tests/test_ci_release_sdk_lane.sh to assert Blacksmith defaults for macOS 15 helper and macOS 26 app builds.

Written for commit 638a8c0. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Chores

    • Switched macOS CI jobs to use hardcoded Blacksmith paid-runner defaults across workflows, nightly and release pipelines; removed Warp fallback and many variable-based overrides.
  • Documentation

    • Updated macOS CI runner guidance to reflect fixed Blacksmith defaults and revised operational instructions for changing runner labels.
  • Tests

    • Tightened CI runner guard tests and assertions to accept only repo-var or Blacksmith runner labels; updated failure messages.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@vercel

vercel Bot commented May 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment Jun 2, 2026 5:36am
cmux-staging Building Building Preview, Comment Jun 2, 2026 5:36am

@coderabbitai

coderabbitai Bot commented May 30, 2026 •

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

Replace WarpBuild macOS runner fallbacks with Blacksmith labels across workflows, remove Warp labels from the actionlint allowlist, update docs to reflect MACOS_RUNNER_* → Blacksmith fallbacks, and tighten CI guard tests to accept only Blacksmith or repo-variable runner references.

Changes

macOS runner migration from WarpBuild to Blacksmith

Layer / File(s) Summary
Runner allowlist and docs
.github/actionlint.yaml, docs/macos-ci-runners.md
Removed warp-macos-*-arm64-6x entries from the actionlint allowlist and updated documentation to describe MACOS_RUNNER_15/MACOS_RUNNER_26 with checked-in Blacksmith fallbacks and manual-run behavior.
Workflow runner defaults and cache keys
.github/workflows/build-ghosttykit.yml, .github/workflows/ci.yml, .github/workflows/ci-macos-compat.yml, .github/workflows/nightly.yml, .github/workflows/release.yml, .github/workflows/perf-activation.yml, .github/workflows/test-depot.yml, .github/workflows/test-e2e.yml, .github/workflows/tmux-corpus.yml
Multiple workflows updated their runs-on expressions or default runner/run-name to use blacksmith-6vcpu-macos-15 / blacksmith-6vcpu-macos-26 as applicable; Swift package cache keys/restore-keys and concurrency/group keys were adjusted to partition by the new runner identity.
Guard and validation updates
tests/test_ci_self_hosted_guard.sh, tests/test_ci_release_sdk_lane.sh
CI self-hosted runner guard tests and release-lane assertions now accept only vars.MACOS_RUNNER_* and blacksmith-<N>vcpu-macos-* labels and were updated to expect blacksmith-6vcpu-macos-15 / blacksmith-6vcpu-macos-26 where appropriate.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • manaflow-ai/cmux#5022: Overlapping runner-label changes for macOS publishing workflows (nightly/release runner defaults).
  • manaflow-ai/cmux#4984: Prior Blacksmith migration updating workflow defaults and the actionlint allowlist that this PR extends.
  • manaflow-ai/cmux#4926: Reverts similar runner-label changes back to WarpBuild in the same workflows and guard script.

Poem

🐰 I swapped my hops from Warp to Steel,
Labels aligned, the workflows feel real,
Docs tidy, guards snug in their nest,
Cache keys hum and runners do their best,
Builds march on, Blacksmith in the lead.


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error, 1 warning)

Check name Status Explanation Resolution
Cmux Swift Blocking Runtime ❌ Error Introduces DispatchSemaphore.wait(), DispatchGroup.wait(), and NSLock in production Swift code (CLI files) violating swift-blocking-runtime.md rules. Replace semaphore.wait()/DispatchGroup.wait() with async/await or callbacks. Use actor isolation instead of NSLock for CLIProcessOutputBuffer synchronization.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (16 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Use Blacksmith as macOS CI default' clearly and concisely summarizes the main change—making Blacksmith the default runner for macOS CI jobs, which is the primary objective across all modified files.
Description check ✅ Passed The description covers the Summary section explaining what changed and why, includes a Testing section with specific test commands, and provides a comprehensive overview. However, the Checklist section is completely missing and no demo video is applicable.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed PR contains no Swift code changes; only GitHub Actions workflows, configuration, documentation, and test scripts—custom check for Swift actor isolation is not applicable.
Cmux No Hacky Sleeps ✅ Passed PR modifies only GitHub Actions workflows and docs; no production runtime code. Existing workflow sleeps are CI orchestration, out of scope per the rule.
Cmux Algorithmic Complexity ✅ Passed All modified files are CI/CD configuration, documentation, or test harnesses. No production code with algorithmic complexity violations exists.
Cmux Swift Concurrency ✅ Passed PR modifies only GitHub Actions workflows, documentation, and bash scripts—no Swift source code changes. Swift concurrency check does not apply.
Cmux Swift @Concurrent ✅ Passed PR contains only CI/workflow configuration changes (.github/workflows/*.yml, docs, bash scripts), no Swift source code modifications. The @concurrent annotation check is not applicable.
Cmux Swift File And Package Boundaries ✅ Passed This PR modifies only GitHub Actions workflows, documentation, and bash test scripts—no production Swift files are changed, making the swift-file-package-boundaries check inapplicable.
Cmux Swift Logging ✅ Passed PR contains no Swift source code changes. Check applies only to production Swift changes; this PR modifies only CI/workflow YAML, documentation, and shell test scripts.
Cmux User-Facing Error Privacy ✅ Passed Changes are to CI infrastructure configuration, developer documentation, and test scripts—all explicitly allowed exceptions per rules. No user-facing error messages modified.
Cmux Full Internationalization ✅ Passed PR modifies only GitHub Actions CI workflows, operational documentation, and test scripts—no user-facing text, Swift source, string catalogs, or web localization files are changed.
Cmux Swiftui State Layout ✅ Passed PR contains no SwiftUI code changes—only CI infrastructure updates (workflow YAMLs, shell scripts, docs). SwiftUI state layout rules do not apply.
Cmux Architecture Rethink ✅ Passed This PR contains no Swift code changes—only GitHub Actions workflows, CI config, documentation, and shell test scripts. The swift-architectural-rethink rule is not applicable.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed This PR contains only CI/GitHub Actions configuration changes (YAML workflows, docs, shell scripts) with 0 Swift files modified, so the auxiliary window close shortcut rule does not apply.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat-blacksmith-ci

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@greptile-apps

greptile-apps Bot commented May 30, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR hardens the macOS CI runner choice by replacing all ${{ vars.MACOS_RUNNER_15 || 'warp-macos-15-arm64-6x' }} dynamic expressions with hardcoded blacksmith-6vcpu-macos-15 / blacksmith-6vcpu-macos-26 labels across every workflow, removing Warp from the actionlint allowlist, and updating the guard test and docs to match.

  • All 9 paid-macOS workflow files now reference Blacksmith labels directly; the repo-variable override layer is gone and any provider switch requires a PR.
  • test_ci_self_hosted_guard.sh tightened to accept only blacksmith-[0-9]+vcpu-macos- patterns; test_ci_release_sdk_lane.sh updated to assert the literal Blacksmith label in release/CI lane YAML.
  • perf-activation.yml and test-e2e.yml retain an auto dispatcher and a runner choice input (without Warp options), and their SPM cache keys now encode the Blacksmith label directly.

Confidence Score: 5/5

Safe to merge; all changes are CI infrastructure, with no application code modified.

Every workflow, guard test, and doc update is internally consistent. The auto dispatcher logic, Depot identity guard, SPM cache keys, concurrency groups, and actionlint allowlist all use the same hardcoded Blacksmith labels. No logic errors or mismatched expressions were found across the 13 changed files.

No files require special attention. The guard test and release-lane assertions were updated in lockstep with the workflow changes.

Important Files Changed

Filename Overview
.github/workflows/ci.yml Five runs-on expressions replaced with hardcoded Blacksmith labels; added explanatory comment block on the macOS 15 vs 26 split. Changes are internally consistent.
.github/workflows/test-e2e.yml All six occurrences of the dynamic runner expression updated to the hardcoded Blacksmith label; run-name, concurrency group, Depot identity guard, and SPM cache key all updated consistently.
.github/workflows/perf-activation.yml Auto-resolution and SPM cache key updated to hardcoded Blacksmith label; Warp options removed from runner choice dropdown. Logic is correct.
.github/workflows/release.yml Two runs-on expressions hardcoded to Blacksmith (15 for CLI helper, 26 for app build). No logic changes.
.github/workflows/ci-macos-compat.yml Matrix os values hardcoded to Blacksmith labels; guard test updated to accept the new pattern. Clean change.
tests/test_ci_self_hosted_guard.sh awk patterns narrowed to accept only blacksmith-[0-9]+vcpu-macos-; Depot guard string-match updated to hardcoded Blacksmith expression. Consistent with workflow changes.
docs/macos-ci-runners.md Docs rewritten to reflect hardcoded Blacksmith defaults and PR-based override process; gh variable set emergency escape hatch commands removed.
.github/actionlint.yaml Warp labels removed from the self-hosted runner allowlist; comment updated to reflect hardcoded workflow labels.
tests/test_ci_release_sdk_lane.sh Four require_job_contains assertions updated to match the new literal Blacksmith label strings instead of the old vars.MACOS_RUNNER_* expressions.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Workflow triggered] --> B{inputs.runner}
    B -->|empty or 'auto'| C[blacksmith-6vcpu-macos-15]
    B -->|explicit choice| D{Choice}
    D --> E[blacksmith-6vcpu-macos-15]
    D --> F[blacksmith-6vcpu-macos-26]
    D --> G[blacksmith-6vcpu-macos-latest]
    D --> H[depot-macos-latest / depot-macos-14]
    C --> I{starts with depot-macos-?}
    E --> I
    F --> I
    G --> I
    H --> I
    I -->|Yes| J[Validate Depot runner identity]
    I -->|No| K[Skip identity guard]
    J --> L[Run job on selected runner]
    K --> L
Loading

Reviews (5): Last reviewed commit: "Hardcode Blacksmith runners; drop MACOS_..." | Re-trigger Greptile

Comment thread docs/macos-ci-runners.md Outdated
Comment on lines 21 to 26
Remove the override and use the checked-in Blacksmith defaults:

```bash
gh variable delete MACOS_RUNNER_15 --repo manaflow-ai/cmux
gh variable delete MACOS_RUNNER_26 --repo manaflow-ai/cmux
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 No documented escape hatch for Blacksmith capacity outages

The PR description explicitly cites a past event where Blacksmith queues hit 55–126 minutes and caused a full revert (PR #4926). The old docs told operators to gh variable set … warp-macos-15-arm64-6x; that guidance is now gone, and Warp is removed from all workflow dropdowns and the actionlint allowlist. If Blacksmith queues surge again, an operator in a time-sensitive release has no documented runner to fall back to. The only option—setting MACOS_RUNNER_15/MACOS_RUNNER_26 to a different provider—is correct in principle, but the docs no longer say what labels are valid alternatives or where to find them.

Consider adding a brief "Emergency override" section that names at least one alternative runner label (e.g., a Warp or GitHub-hosted fallback) and notes that the label must also be added to .github/actionlint.yaml before use.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Paused merge after the refreshed PR run stayed queued on Blacksmith macOS runners for over 11 minutes without a runner assignment. Live repo variables have been restored to Warp for now: MACOS_RUNNER_15=warp-macos-15-arm64-6x, MACOS_RUNNER_26=warp-macos-26-arm64-6x. Current Warp probe assigned a runner in about 1 second. Keeping this PR open for a later Blacksmith retry.

# Conflicts:
#	.github/workflows/nightly.yml
#	.github/workflows/release.yml
#	docs/macos-ci-runners.md

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 13 files

Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 13 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/tmux-corpus.yml">

<violation number="1" location=".github/workflows/tmux-corpus.yml:64">
P2: Hardcoded runner label removes the `vars.MACOS_RUNNER_15` override, contradicting the PR's stated design of "Auto resolution now follows MACOS_RUNNER_15 first, then Blacksmith." The original pattern allowed operators to redirect this job by setting a repo variable if Blacksmith has queue or capacity issues; the new hardcoded label removes that flexibility entirely.</violation>
</file>

<file name="docs/macos-ci-runners.md">

<violation number="1" location="docs/macos-ci-runners.md:3">
P2: The docs overstate runner behavior: Blacksmith is the default, but manual workflows can still run on Depot runners.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

terminal-nightly:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ${{ vars.MACOS_RUNNER_15 || 'warp-macos-15-arm64-6x' }}
runs-on: blacksmith-6vcpu-macos-15

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Hardcoded runner label removes the vars.MACOS_RUNNER_15 override, contradicting the PR's stated design of "Auto resolution now follows MACOS_RUNNER_15 first, then Blacksmith." The original pattern allowed operators to redirect this job by setting a repo variable if Blacksmith has queue or capacity issues; the new hardcoded label removes that flexibility entirely.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/tmux-corpus.yml, line 64:

<comment>Hardcoded runner label removes the `vars.MACOS_RUNNER_15` override, contradicting the PR's stated design of "Auto resolution now follows MACOS_RUNNER_15 first, then Blacksmith." The original pattern allowed operators to redirect this job by setting a repo variable if Blacksmith has queue or capacity issues; the new hardcoded label removes that flexibility entirely.</comment>

<file context>
@@ -61,7 +61,7 @@ jobs:
   terminal-nightly:
     if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
-    runs-on: ${{ vars.MACOS_RUNNER_15 || 'blacksmith-6vcpu-macos-15' }}
+    runs-on: blacksmith-6vcpu-macos-15
     timeout-minutes: 30
     steps:
</file context>
Suggested change
runs-on: blacksmith-6vcpu-macos-15
runs-on: ${{ vars.MACOS_RUNNER_15 || 'blacksmith-6vcpu-macos-15' }}

Comment thread docs/macos-ci-runners.md
# macOS CI runners

All paid macOS CI/CD jobs pick their runner from two repository variables instead of a hardcoded label:
All paid macOS CI/CD jobs run on Blacksmith. The runner labels are hardcoded in the workflows (git-tracked); there is no repo-variable override.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The docs overstate runner behavior: Blacksmith is the default, but manual workflows can still run on Depot runners.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At docs/macos-ci-runners.md, line 3:

<comment>The docs overstate runner behavior: Blacksmith is the default, but manual workflows can still run on Depot runners.</comment>

<file context>
@@ -1,46 +1,22 @@
 # macOS CI runners
 
-All paid macOS CI/CD jobs pick their runner from two repository variables instead of a hardcoded label:
+All paid macOS CI/CD jobs run on Blacksmith. The runner labels are hardcoded in the workflows (git-tracked); there is no repo-variable override.
 
-- `MACOS_RUNNER_15` for jobs that build the real universal Release app, including nightly, stable release, `release-build`, and the e2e/perf defaults.
</file context>
Suggested change
All paid macOS CI/CD jobs run on Blacksmith. The runner labels are hardcoded in the workflows (git-tracked); there is no repo-variable override.
All paid macOS CI/CD jobs default to Blacksmith. Runner labels are git-tracked in workflows, manual workflows can still choose explicit runner options, and there is no repo-variable override.

@lawrencecchen lawrencecchen added the stale-revisit Closed after 30+ days without activity; preserved for possible revisit or reopening. label Sep 23, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in cmux backlog Sep 23, 2026

This branch was successfully deployed

1 active deployment
Preview – cmux — 638a8c0d Deployed Jun 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale-revisit Closed after 30+ days without activity; preserved for possible revisit or reopening.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants