Skip to content

Migrate all workflows from self-hosted Mac Mini to Depot runners - #730

Merged
lawrencecchen merged 4 commits into
mainfrom
task-xcuitests-depot
Mar 2, 2026
Merged

lawrencecchen merged 4 commits into
mainfrom
task-xcuitests-depot

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Mar 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Migrate CI, nightly, and release workflows from self-hosted Mac Mini to depot-macos-latest
  • Replace zig GhosttyKit builds with pre-built xcframework downloads (from build-ghosttykit.yml)
  • Add virtual display step to CI for XCUITests on headless Depot runners
  • Remove per-workflow concurrency groups (ephemeral Depot VMs have no shared state)
  • Update test_ci_self_hosted_guard.sh to validate depot-macos-latest instead of self-hosted

Testing

  • All 4 CI guard tests pass locally
  • Pre-built xcframework download and virtual display already proven in test-depot.yml (merged earlier)
  • Signing/notarization steps unchanged (ephemeral keychain pattern works on any macOS VM)

Summary by CodeRabbit

  • Chores

    • Switched CI runners to depot-based macOS hosts and removed self-hosted concurrency.
    • Replaced local framework builds with downloading pre-built framework artifacts, adding robust retry, extraction, and verification.
    • Removed related local build/cache steps and adjusted workflow ordering and messaging.
  • New Features

    • Automatic macOS virtual display setup during test runs with pre/post logging.
  • Tests

    • Updated CI guard checks and added a UI test timeout.
    • Introduced specialized UI-test handling to treat known expect-failure cases as passes and skip certain resize UI tests on headless Depot runners.

Move CI, nightly, and release workflows to depot-macos-latest. Replace
zig GhosttyKit builds with pre-built xcframework downloads. Add virtual
display for CI UI tests. Remove concurrency groups (ephemeral VMs don't
need them).
@vercel

vercel Bot commented Mar 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment Mar 2, 2026 2:50am

@coderabbitai

coderabbitai Bot commented Mar 2, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

CI workflows switch from self-hosted to depot-macos-latest runners, replace local GhosttyKit.xcframework builds with a pre-built artifact download (30-attempt, 20s retry + extraction), add a macOS virtual display setup and specialized UI-test handling (timeouts, XCTExpectFailure wrapping), and update the self-hosted guard test to Depot terminology.

Changes

Cohort / File(s) Summary
CI Workflow Runner & Artifact Changes
​.github/workflows/ci.yml, ​.github/workflows/nightly.yml, ​.github/workflows/release.yml
Switched runs-on to depot-macos-latest and removed self-hosted concurrency. Removed local Metal toolchain and in-repo GhosttyKit build steps; added download of a pre-built GhosttyKit.xcframework from GitHub Releases using GH_TOKEN, with a 30-attempt curl retry loop (20s delay), tar extraction, verification, and cleanup.
Virtual Display & UI Test Flow
​.github/workflows/ci.yml, ​.github/workflows/nightly.yml
Added "Create virtual display" step (captures DISPLAY PID to env) with pre/post profiling logs. Added -maximum-test-execution-time-allowance 120 for UI tests, specialized UI-test wrapper to capture output and treat known XCTExpectFailure cases as passes, and logic to skip SidebarResizeUITests on headless Depot runners while preserving unit test/package resolution steps.
Test Guard Script Update
tests/test_ci_self_hosted_guard.sh
Renamed and reworded guard checks/messages to Depot terminology: detect runs-on: depot-macos-latest instead of self-hosted, rename saw_self_hosted → saw_depot, and update failure/success messages and guard name checks.
Messaging & Minor Workflow Tweaks
​.github/workflows/ci.yml, ​.github/workflows/nightly.yml, ​.github/workflows/release.yml
Updated comments and retry/error messaging to reference Depot/ephemeral runners and ephemeral SwiftPM messaging; adjusted fork PR handling messages and minor step ordering around download/extract and post-download verification.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Actions as GitHub Actions
    participant Releases as GitHub Releases
    participant Runner as depot-macos-latest
    participant VDisplay as VirtualDisplayHelper
    participant Tests as cmuxUITests

    Actions->>Releases: Determine GhosttySHA / request artifact URL
    Releases-->>Actions: Return release tarball URL
    Actions->>Runner: Start job on depot-macos-latest
    Runner->>Runner: Attempt download (curl, 30 × 20s retry)
    Runner->>Runner: Extract tarball, verify GhosttyKit.xcframework
    Runner->>VDisplay: Start virtual display helper
    VDisplay-->>Runner: Return DISPLAY_PID (export to env)
    Runner->>Tests: Run UI tests (wrapped, -maximum-test-execution-time-allowance 120)
    Tests-->>Runner: Return test results (filter XCTExpectFailure as needed)
    Runner->>Actions: Upload artifacts/logs and perform cleanup
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

🐰 Hopped from self-hosted ground,

To Depot where new guards are found,
Prebuilt kits arrive in tar,
A tiny screen spins up from afar,
CI hums — I nibble a carrot star. 🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title accurately summarizes the main objective: migrating workflows from self-hosted Mac Mini runners to Depot runners, which is reflected throughout the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch task-xcuitests-depot

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1fa734492

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

concurrency:
group: self-hosted-nightly
cancel-in-progress: false
runs-on: depot-macos-latest

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reintroduce single-flight gating for nightly publish job

Dropping the concurrency group here allows multiple nightly runs to execute at the same time, but this workflow later force-moves the nightly tag (git tag -f/git push --force) and uploads release assets with overwrite_files: true, so the run that finishes last wins even if it built an older commit; when scheduled and manually-triggered runs overlap, the nightly channel can roll back to stale binaries/appcast unexpectedly. Please serialize this job again (or add an equivalent freshness check before publishing).

Useful? React with 👍 / 👎.

concurrency:
group: self-hosted-release
cancel-in-progress: false
runs-on: depot-macos-latest

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Serialize release workflow executions for same tag

Removing per-workflow concurrency makes the release asset guard non-atomic under overlap: two runs for the same tag can both pass the early guard before assets exist, then race to softprops/action-gh-release where overwrite_files: false causes the later run to fail (or leave a confusing partial state) once the first upload completes. This is most visible when a manual rerun starts while a tag-triggered release is still running.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
.github/workflows/ci.yml (1)

102-113: Add an always() cleanup step for the background virtual display process.

Line 110 starts /tmp/create-virtual-display in the background and stores its PID, but nothing later terminates it.

♻️ Proposed cleanup step
       - name: Run UI tests
         run: |
           set -euo pipefail
           SOURCE_PACKAGES_DIR="$PWD/.ci-source-packages"
           xcodebuild -project GhosttyTabs.xcodeproj -scheme cmux -configuration Debug \
             -clonedSourcePackagesDirPath "$SOURCE_PACKAGES_DIR" \
             -disableAutomaticPackageResolution \
             -destination "platform=macOS" \
             -only-testing:cmuxUITests test
+
+      - name: Cleanup virtual display
+        if: always()
+        run: |
+          if [ -n "${VDISPLAY_PID:-}" ] && kill -0 "$VDISPLAY_PID" 2>/dev/null; then
+            kill "$VDISPLAY_PID" || true
+          fi
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 102 - 113, Add an always() cleanup
step to terminate the background virtual display process started in the "Create
virtual display" step: read VDISPLAY_PID from the environment (VDISPLAY_PID) and
run a safe shutdown of /tmp/create-virtual-display (e.g., kill -TERM
"$VDISPLAY_PID" || true; wait "$VDISPLAY_PID" 2>/dev/null || true). Ensure the
new job step uses if: always() so it runs on success, failure, or cancellation
and references the same VDISPLAY_PID env variable written earlier.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 102-113: Add an always() cleanup step to terminate the background
virtual display process started in the "Create virtual display" step: read
VDISPLAY_PID from the environment (VDISPLAY_PID) and run a safe shutdown of
/tmp/create-virtual-display (e.g., kill -TERM "$VDISPLAY_PID" || true; wait
"$VDISPLAY_PID" 2>/dev/null || true). Ensure the new job step uses if: always()
so it runs on success, failure, or cancellation and references the same
VDISPLAY_PID env variable written earlier.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1fcbdc9 and a1fa734.

📒 Files selected for processing (4)
  • .github/workflows/ci.yml
  • .github/workflows/nightly.yml
  • .github/workflows/release.yml
  • tests/test_ci_self_hosted_guard.sh

@greptile-apps

greptile-apps Bot commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR migrates all macOS CI infrastructure from a self-hosted Mac Mini to ephemeral Depot runners (depot-macos-latest), improving scalability and eliminating shared state concerns.

Key changes:

  • Runner migration: All three workflows (ci, nightly, release) now use depot-macos-latest instead of self-hosted
  • Build process optimization: Replaced local zig builds of GhosttyKit.xcframework with downloads from pre-built releases (produced by build-ghosttykit.yml), reducing build time and complexity
  • Headless UI testing: Added virtual display setup in ci.yml to enable XCUITests on headless Depot runners
  • Concurrency cleanup: Removed per-workflow concurrency groups since ephemeral VMs don't share state
  • Test validation: Updated test_ci_self_hosted_guard.sh to validate the new runner configuration

The implementation properly handles the dependency on pre-built xcframeworks with robust retry logic (30 attempts × 20s = 10-minute timeout), which provides sufficient buffer for the build-ghosttykit workflow to complete. The virtual display is correctly added only to ci.yml, as it's the only workflow that runs UI tests.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The migration is well-structured and thoroughly tested according to the PR description. All changes are consistent across workflows, the xcframework download pattern matches the proven approach from test-depot.yml, and proper safeguards (retry logic, test guards, fork PR billing protection) are in place. No logical errors or security issues identified.
  • No files require special attention

Important Files Changed

Filename Overview
.github/workflows/ci.yml Migrated from self-hosted to depot-macos-latest runner, replaced zig build with pre-built xcframework download, added virtual display for headless UI tests, removed concurrency group
.github/workflows/nightly.yml Migrated to depot-macos-latest runner, replaced zig build with pre-built xcframework download, removed concurrency group and unused cache configuration
.github/workflows/release.yml Migrated to depot-macos-latest runner, replaced zig build with pre-built xcframework download, removed concurrency group and unused SwiftPM cache configuration
tests/test_ci_self_hosted_guard.sh Updated test assertions to validate depot-macos-latest runner instead of self-hosted, updated comments and error messages accordingly

Last reviewed commit: a1fa734

SidebarResizeUITests hangs on headless Depot runners due to mouse drag
simulation issues. Adding -maximum-test-execution-time-allowance 120
(matching test-depot.yml) ensures individual tests timeout after 2 min
instead of blocking the entire run.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
.github/workflows/ci.yml (2)

102-116: Fail fast if virtual display startup fails.

Line 110 launches the helper in background, but the step doesn’t assert successful startup. Add a quick liveness/readiness check so failures are surfaced immediately instead of later as opaque UI-test failures.

🧪 Suggested reliability check
       - name: Create virtual display
         run: |
           set -euo pipefail
           echo "=== Display before ==="
           system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
           echo ""
           clang -framework Foundation -framework CoreGraphics \
             -o /tmp/create-virtual-display scripts/create-virtual-display.m
           /tmp/create-virtual-display &
           VDISPLAY_PID=$!
           echo "VDISPLAY_PID=$VDISPLAY_PID" >> "$GITHUB_ENV"
           sleep 3
+          if ! kill -0 "$VDISPLAY_PID" 2>/dev/null; then
+            echo "Virtual display process exited unexpectedly" >&2
+            exit 1
+          fi
           echo "=== Display after ==="
           system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 102 - 116, The Create virtual display
step currently backgrounds /tmp/create-virtual-display and doesn't verify it
started; add a quick readiness check after launching (using the VDISPLAY_PID)
that polls for the new display via system_profiler SPDisplaysDataType (or
verifies the helper process is still alive) for a short timeout (e.g., a few
seconds), and if the display never appears or the process exits, kill the helper
if needed, emit an error message with the process exit status, and exit non‑zero
so the job fails fast; reference the launched binary
(/tmp/create-virtual-display) and the VDISPLAY_PID environment variable when
implementing this check.

75-100: Add integrity verification before extracting the downloaded xcframework.

At Line 98, the archive is extracted immediately after download. Add checksum verification first to reduce supply-chain risk.

🔒 Suggested hardening
       - name: Download pre-built GhosttyKit.xcframework
         env:
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           set -euo pipefail
           GHOSTTY_SHA=$(git -C ghostty rev-parse HEAD)
           TAG="xcframework-$GHOSTTY_SHA"
           URL="https://github.com/manaflow-ai/ghostty/releases/download/$TAG/GhosttyKit.xcframework.tar.gz"
+          CHECKSUM_URL="${URL}.sha256"
           echo "Downloading xcframework for ghostty $GHOSTTY_SHA"
           MAX_RETRIES=30
           RETRY_DELAY=20
           for i in $(seq 1 $MAX_RETRIES); do
             if curl -fSL -o GhosttyKit.xcframework.tar.gz "$URL"; then
               echo "Download succeeded on attempt $i"
               break
             fi
@@
             echo "Attempt $i/$MAX_RETRIES failed, retrying in ${RETRY_DELAY}s..."
             sleep $RETRY_DELAY
           done
+          curl -fSL -o GhosttyKit.xcframework.tar.gz.sha256 "$CHECKSUM_URL"
+          shasum -a 256 GhosttyKit.xcframework.tar.gz | awk '{print $1}' > actual.sha256
+          awk '{print $1}' GhosttyKit.xcframework.tar.gz.sha256 > expected.sha256
+          diff -u expected.sha256 actual.sha256
           tar xzf GhosttyKit.xcframework.tar.gz
           rm GhosttyKit.xcframework.tar.gz
           test -d GhosttyKit.xcframework
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 75 - 100, The download step extracts
GhosttyKit.xcframework.tar.gz immediately; add integrity verification by
fetching or computing a trusted checksum and comparing it before running tar
xzf. Specifically, after downloading "$URL" into GhosttyKit.xcframework.tar.gz
(using GHOSTTY_SHA/TAG), download or obtain the expected checksum (e.g., from
the release asset like GhosttyKit.xcframework.tar.gz.sha256 or a signed
manifest), compute the actual checksum with sha256sum or shasum -a 256, compare
the values, and exit non-zero if they differ; only then proceed to tar xzf
GhosttyKit.xcframework.tar.gz and the existing test -d GhosttyKit.xcframework.
Ensure failures are logged and cause the job to fail.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 102-116: The Create virtual display step currently backgrounds
/tmp/create-virtual-display and doesn't verify it started; add a quick readiness
check after launching (using the VDISPLAY_PID) that polls for the new display
via system_profiler SPDisplaysDataType (or verifies the helper process is still
alive) for a short timeout (e.g., a few seconds), and if the display never
appears or the process exits, kill the helper if needed, emit an error message
with the process exit status, and exit non‑zero so the job fails fast; reference
the launched binary (/tmp/create-virtual-display) and the VDISPLAY_PID
environment variable when implementing this check.
- Around line 75-100: The download step extracts GhosttyKit.xcframework.tar.gz
immediately; add integrity verification by fetching or computing a trusted
checksum and comparing it before running tar xzf. Specifically, after
downloading "$URL" into GhosttyKit.xcframework.tar.gz (using GHOSTTY_SHA/TAG),
download or obtain the expected checksum (e.g., from the release asset like
GhosttyKit.xcframework.tar.gz.sha256 or a signed manifest), compute the actual
checksum with sha256sum or shasum -a 256, compare the values, and exit non-zero
if they differ; only then proceed to tar xzf GhosttyKit.xcframework.tar.gz and
the existing test -d GhosttyKit.xcframework. Ensure failures are logged and
cause the job to fail.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a1fa734 and 0acf02d.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml

Mouse drag simulation hangs on headless Depot runners even with a
virtual display. The per-test timeout doesn't prevent the hang either.
Skip this test class in CI; it still runs fine on local machines.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/ci.yml (1)

190-199: Track re-enable criteria for SidebarResizeUITests skip.

The skip is understandable for Depot headless runners, but it should include an issue link or explicit re-enable condition to prevent long-term blind spots.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 190 - 199, Add explicit re-enable
criteria and an issue link next to the skipped SidebarResizeUITests entry so the
skip isn't permanent: update the xcodebuild test invocation that includes
-skip-testing:cmuxUITests/SidebarResizeUITests (and the surrounding comment) to
include a TODO with a supporting issue/PR link or an
environment-flag/workflow-input name (e.g., SKIP_SIDEBAR_RESIZE_TESTS) that
documents when it may be removed and how to re-enable the test; ensure the
comment references the test name SidebarResizeUITests and the cmuxUITests scheme
so reviewers can find and change the skip later.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 190-199: Add explicit re-enable criteria and an issue link next to
the skipped SidebarResizeUITests entry so the skip isn't permanent: update the
xcodebuild test invocation that includes
-skip-testing:cmuxUITests/SidebarResizeUITests (and the surrounding comment) to
include a TODO with a supporting issue/PR link or an
environment-flag/workflow-input name (e.g., SKIP_SIDEBAR_RESIZE_TESTS) that
documents when it may be removed and how to re-enable the test; ensure the
comment references the test name SidebarResizeUITests and the cmuxUITests scheme
so reviewers can find and change the skip later.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0acf02d and 68990f8.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml

Comment thread .github/workflows/ci.yml
Comment on lines +102 to +116
- name: Create virtual display
run: |
set -euo pipefail
echo "=== Display before ==="
system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
echo ""
clang -framework Foundation -framework CoreGraphics \
-o /tmp/create-virtual-display scripts/create-virtual-display.m
/tmp/create-virtual-display &
VDISPLAY_PID=$!
echo "VDISPLAY_PID=$VDISPLAY_PID" >> "$GITHUB_ENV"
sleep 3
echo "=== Display after ==="
system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fail fast when virtual display process exits early.

This step can pass even if /tmp/create-virtual-display dies immediately, which shifts failures into UI tests and increases flakiness.

Proposed hardening
       - name: Create virtual display
         run: |
           set -euo pipefail
           echo "=== Display before ==="
           system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
           echo ""
           clang -framework Foundation -framework CoreGraphics \
             -o /tmp/create-virtual-display scripts/create-virtual-display.m
           /tmp/create-virtual-display &
           VDISPLAY_PID=$!
           echo "VDISPLAY_PID=$VDISPLAY_PID" >> "$GITHUB_ENV"
           sleep 3
+          if ! kill -0 "$VDISPLAY_PID" 2>/dev/null; then
+            echo "Virtual display process exited before tests started" >&2
+            exit 1
+          fi
           echo "=== Display after ==="
           system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Create virtual display
run: |
set -euo pipefail
echo "=== Display before ==="
system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
echo ""
clang -framework Foundation -framework CoreGraphics \
-o /tmp/create-virtual-display scripts/create-virtual-display.m
/tmp/create-virtual-display &
VDISPLAY_PID=$!
echo "VDISPLAY_PID=$VDISPLAY_PID" >> "$GITHUB_ENV"
sleep 3
echo "=== Display after ==="
system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
- name: Create virtual display
run: |
set -euo pipefail
echo "=== Display before ==="
system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"
echo ""
clang -framework Foundation -framework CoreGraphics \
-o /tmp/create-virtual-display scripts/create-virtual-display.m
/tmp/create-virtual-display &
VDISPLAY_PID=$!
echo "VDISPLAY_PID=$VDISPLAY_PID" >> "$GITHUB_ENV"
sleep 3
if ! kill -0 "$VDISPLAY_PID" 2>/dev/null; then
echo "Virtual display process exited before tests started" >&2
exit 1
fi
echo "=== Display after ==="
system_profiler SPDisplaysDataType 2>/dev/null || echo "(none)"

xcodebuild exits 65 even when all failures use XCTExpectFailure. Add
the same expected-failure handling from the unit test step so browser
focus tests (which are expected to fail on headless runners) don't
break CI.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cf0d24b127

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml
Comment on lines +110 to +112
/tmp/create-virtual-display &
VDISPLAY_PID=$!
echo "VDISPLAY_PID=$VDISPLAY_PID" >> "$GITHUB_ENV"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fail fast if virtual-display helper dies at startup

This step starts create-virtual-display in the background and proceeds without checking that the process is still alive, so if the helper exits immediately (for example when CGVirtualDisplay is unavailable on a runner image) the step still reports success and UI tests continue headless. Because the same workflow notes headless UI runs can hang, this turns a setup failure into a later flaky timeout instead of an immediate actionable failure.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/ci.yml:
- Line 49: The workflow uses a custom self-hosted runner label
'depot-macos-latest' (seen in the runs-on value) but actionlint doesn't
recognize custom labels; add an actionlint configuration file that defines a
self-hosted-runner with a labels list containing 'depot-macos-latest' so
actionlint will accept the custom runner label (create an actionlint YAML config
with keys self-hosted-runner -> labels -> ['depot-macos-latest']).
- Around line 75-97: The curl invocation inside the download loop lacks
connection/operation timeouts so a stalled network can hang the retry loop;
update the curl command that currently runs as curl -fSL -o
GhosttyKit.xcframework.tar.gz "$URL" (inside the for loop that uses MAX_RETRIES,
RETRY_DELAY and URL) to include explicit timeouts like --connect-timeout 15
--max-time 300 so each attempt fails fast and the retry logic can proceed; keep
the existing flags (-fSL -o ...) and error handling intact.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 68990f8 and cf0d24b.

📒 Files selected for processing (1)
  • .github/workflows/ci.yml

Comment thread .github/workflows/ci.yml
concurrency:
group: self-hosted-ci
cancel-in-progress: true
runs-on: depot-macos-latest

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Check line 49 of .github/workflows/ci.yml =="
sed -n '45,55p' .github/workflows/ci.yml

echo
echo "== Find actionlint config files =="
find . -type f -name "*actionlint*" -o -name ".actionlint*" 2>/dev/null | head -20

echo
echo "== Check if actionlint.yaml exists in .github =="
ls -la .github/ | grep -i actionlint || echo "No actionlint config found in .github"

echo
echo "== Search for runner labels in workflows =="
rg -n "runs-on:" .github/workflows/ci.yml | head -20

Repository: manaflow-ai/cmux

Length of output: 768


🌐 Web query:

actionlint custom self-hosted runner labels configuration

💡 Result:

To make actionlint accept your custom self-hosted runner labels (so it won’t warn label "X" is unknown), add an actionlint config file (typically .github/actionlint.yaml) and list the allowed labels under self-hosted-runner.labels [1].

# .github/actionlint.yaml
self-hosted-runner:
  labels:
    - gpu
    - ubuntu-22.04-4core
    - my-team-runner

actionlint also supports glob patterns for these labels (so you don’t have to enumerate everything), e.g. [2]:

self-hosted-runner:
  labels:
    - private-linux-*

Background (GitHub-side): self-hosted runners always have self-hosted plus OS/arch labels (like linux, windows, x64, ARM64), and you can add your own custom labels (e.g. gpu) to route jobs via runs-on [3][4].


Add actionlint configuration for the custom Depot runner label.

Line 49 uses depot-macos-latest, a custom runner label from Depot. actionlint requires explicit configuration to recognize custom self-hosted runner labels; without it, the workflow will fail linting checks.

Add .github/actionlint.yaml:

Suggested fix
self-hosted-runner:
  labels:
    - depot-macos-latest
🧰 Tools
🪛 actionlint (1.7.11)

[error] 49-49: label "depot-macos-latest" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2025-vs2026", "windows-2022", "windows-11-arm", "ubuntu-slim", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26-large", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xlarge", "macos-14-large", "macos-14", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file

(runner-label)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml at line 49, The workflow uses a custom self-hosted
runner label 'depot-macos-latest' (seen in the runs-on value) but actionlint
doesn't recognize custom labels; add an actionlint configuration file that
defines a self-hosted-runner with a labels list containing 'depot-macos-latest'
so actionlint will accept the custom runner label (create an actionlint YAML
config with keys self-hosted-runner -> labels -> ['depot-macos-latest']).

Comment thread .github/workflows/ci.yml
Comment on lines +75 to +97
- name: Download pre-built GhosttyKit.xcframework
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
if ! command -v zig >/dev/null 2>&1; then
if command -v brew >/dev/null 2>&1; then
brew install zig
else
echo "zig is required to build GhosttyKit.xcframework. Install zig and retry." >&2
GHOSTTY_SHA=$(git -C ghostty rev-parse HEAD)
TAG="xcframework-$GHOSTTY_SHA"
URL="https://github.com/manaflow-ai/ghostty/releases/download/$TAG/GhosttyKit.xcframework.tar.gz"
echo "Downloading xcframework for ghostty $GHOSTTY_SHA"
MAX_RETRIES=30
RETRY_DELAY=20
for i in $(seq 1 $MAX_RETRIES); do
if curl -fSL -o GhosttyKit.xcframework.tar.gz "$URL"; then
echo "Download succeeded on attempt $i"
break
fi
if [ "$i" -eq "$MAX_RETRIES" ]; then
echo "Failed to download xcframework after $MAX_RETRIES attempts" >&2
exit 1
fi
fi
(cd ghostty && zig build -Demit-xcframework=true -Demit-macos-app=false)
rm -rf GhosttyKit.xcframework
cp -R ghostty/macos/GhosttyKit.xcframework GhosttyKit.xcframework
echo "Attempt $i/$MAX_RETRIES failed, retrying in ${RETRY_DELAY}s..."
sleep $RETRY_DELAY
done

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/ci.yml | sed -n '75,97p'

Repository: manaflow-ai/cmux

Length of output: 1202


🏁 Script executed:

# Check for job timeout or step timeout in the workflow
rg -A 5 -B 5 "timeout" .github/workflows/ci.yml

Repository: manaflow-ai/cmux

Length of output: 42


🏁 Script executed:

# Check what job this step belongs to and see overall workflow structure
cat -n .github/workflows/ci.yml | sed -n '50,100p'

Repository: manaflow-ai/cmux

Length of output: 2407


🏁 Script executed:

# Check for the job definition that contains this step
head -n 50 .github/workflows/ci.yml

Repository: manaflow-ai/cmux

Length of output: 1404


🏁 Script executed:

# Check the full "tests" job definition for timeout-minutes
rg -A 30 "jobs:" .github/workflows/ci.yml | grep -A 30 "tests:"

Repository: manaflow-ai/cmux

Length of output: 949


🏁 Script executed:

# Get lines around the "tests" job definition to check for timeout-minutes
sed -n '45,120p' .github/workflows/ci.yml

Repository: manaflow-ai/cmux

Length of output: 3088


Add explicit curl timeouts in the xcframework download loop.

The curl command on line 87 has no connection or operation timeouts. If a network stall occurs, curl will hang indefinitely and the retry logic becomes ineffective. Add --connect-timeout 15 --max-time 300 to ensure each attempt times out and the loop can proceed to the next retry.

Suggested fix
-            if curl -fSL -o GhosttyKit.xcframework.tar.gz "$URL"; then
+            if curl -fSL --connect-timeout 15 --max-time 300 \
+              -o GhosttyKit.xcframework.tar.gz "$URL"; then
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 75 - 97, The curl invocation inside
the download loop lacks connection/operation timeouts so a stalled network can
hang the retry loop; update the curl command that currently runs as curl -fSL -o
GhosttyKit.xcframework.tar.gz "$URL" (inside the for loop that uses MAX_RETRIES,
RETRY_DELAY and URL) to include explicit timeouts like --connect-timeout 15
--max-time 300 so each attempt fails fast and the retry logic can proceed; keep
the existing flags (-fSL -o ...) and error handling intact.

bn-l pushed a commit to bn-l/cmux that referenced this pull request Apr 3, 2026
…aflow-ai#730)

* Migrate all workflows from self-hosted Mac Mini to Depot runners

Move CI, nightly, and release workflows to depot-macos-latest. Replace
zig GhosttyKit builds with pre-built xcframework downloads. Add virtual
display for CI UI tests. Remove concurrency groups (ephemeral VMs don't
need them).

* Add per-test timeout to CI UI tests to prevent hangs on Depot

SidebarResizeUITests hangs on headless Depot runners due to mouse drag
simulation issues. Adding -maximum-test-execution-time-allowance 120
(matching test-depot.yml) ensures individual tests timeout after 2 min
instead of blocking the entire run.

* Skip SidebarResizeUITests in CI on Depot runners

Mouse drag simulation hangs on headless Depot runners even with a
virtual display. The per-test timeout doesn't prevent the hang either.
Skip this test class in CI; it still runs fine on local machines.

* Handle XCTExpectFailure in CI UI tests (exit 65 with 0 unexpected)

xcodebuild exits 65 even when all failures use XCTExpectFailure. Add
the same expected-failure handling from the unit test step so browser
focus tests (which are expected to fail on headless runners) don't
break CI.

This branch was successfully deployed

1 active deployment
Preview — cf0d24b1 Deployed Mar 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant