ci(dev tip): tip-dmg-build waits on the fleet, not a stalled Blacksmith queue by teamleaderleo · Pull Request #25902 · manaflow-ai/cmux

teamleaderleo · 2026-10-27T04:11:49Z

Follow-up to #25880. DEV publish 40445038889 placed tip-dmg-build on the owned pool first, but no mini was free at pick time, so the job fell through to blacksmith-6vcpu-macos-26. It then queued 25m07s there for 2m57s of work.

This applies #25847's rule (merged as 4512284), which verify already uses, to tip-dmg-build through the same --wait-owned path in e2e_runner_pool.py. The runner job's two-machine pick is now pick 2 --wait-owned. simple_pool_picker.pick then works like this:

An owned pool has two free machines: verify and tip-dmg-build both take it, as before.
A Blacksmith label has two free, moving slots: tip-dmg-build takes Blacksmith macOS, and verify alone tries pick 1 --wait-owned, as before.
Neither: both wait on the owned pool with the shortest queue for its size (owned glaeda-std/light), and the run's marker counts 2 jobs. The owned-pool rescue watches that queue, and any re-run (attempt 2) takes Blacksmith. So nothing waits on one pool with no fallback.

Nothing changes for fork runs, the bot's re-runs or tip-deltas (Sparkle and R2 keys), and nothing else in the job does either: still no secrets, and the same artifact hand-off to the Linux signer. No hosted macOS label is added.

The diff is 2 files and 16 lines: the workflow pick step and the test_cmux_next_dev_tip_feed.py assertion. tests/test_ci_pr_runner_pool.py::test_auto_pick_waits_on_the_fleet_only_with_wait_owned already covers how the flag is wired.

Risk

In the third case, tip-dmg-build now queues on an owned mini instead of Blacksmith, which could delay verify on the same pool. The rescue's queue budget bounds that wait. As in #25880, the job holds no secret, token or environment.

Tests run locally: tests/test_cmux_next_dev_tip_feed.py PASS, and the --wait-owned unit test passes.

Agent session: agent session

🤖 Generated with Claude Code

^{Need help on this PR? Tag @codesmith-bot with what you need.}

Summary by CodeRabbit

Chores
- macOS verification and build jobs now wait until the required runner capacity is available instead of switching to a single-machine fallback.
- When an owned Mac is selected, both jobs use it; otherwise, the build uses Blacksmith macOS.

…th queue The runner job's two-machine pick now passes --wait-owned (#25847's rule for verify): with no two owned Macs free and no Blacksmith label with two free, moving slots, verify and tip-dmg-build both wait on the owned pool with the shortest queue, which the owned-pool rescue watches. Run 40445038889's tip-dmg-build queued 25 min on Blacksmith macOS for 3 min of work. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-cc8bff1c3ddb4ec353c5

cursor · 2026-10-27T04:11:54Z

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

teamleaderleo · 2026-10-27T04:12:29Z

Owned by an agent session · session

github-actions · 2026-10-27T04:14:33Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

coderabbitai · 2026-10-27T04:20:36Z

No actionable comments were generated in the recent review.

Recent review info

Run configuration

Source mode: Auto
Repository: manaflow-ai/cmux
Branch: tip-dmg-wait-owned
File: .coderabbit.yaml
Commit: 1d70a13
Layers: Repository YAML
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 2dedd737-194e-4239-b44b-cff85a8b5dc6

Commits

Reviewing files that changed from the base of the PR and between a8694ba and 1d70a13.

Files selected for processing: 2

.github/workflows/cmux-next-dev-daily.yml
tests/test_cmux_next_dev_tip_feed.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

Walkthrough

The workflow now waits for two owned Macs before assigning owned runners. If that placement is unavailable, both jobs use Blacksmith runners. The runner-pool test checks the updated selection behavior.

Changes: DEV tip runner placement

Layer / File(s)	Summary
Two-runner placement and fallback `.github/workflows/cmux-next-dev-daily.yml`, `tests/test_cmux_next_dev_tip_feed.py`	The workflow requests two owned Macs and removes the one-Mac fallback for verify. The test expects the updated picker call and retains the one-machine fallback assertion.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Possibly related issues

Issue 14196 concerns macOS CI queue time and pool capacity. This change updates runner selection between the owned fleet and Blacksmith.
Issue 22303 concerns queued cmux-next runs and runner availability. This change changes runner routing in the DEV workflow.
Issue 13325 covers owned Macs, Blacksmith fallback, and fleet reuse. This change updates that fallback behavior for the DEV tip build.

Possibly related PRs

manaflow-ai/cmux#25442: Adds the runner-selection job in the same workflow, whose picker call this change updates.
manaflow-ai/cmux#25880: Adds pool-placement logic for tip-dmg-build; this change updates its picker call and fallback.
manaflow-ai/cmux#25847: Adds the --wait-owned flag used by the updated picker call.

Merge Risk: Minimal · up to 1d70a

No actionable merge-blocking risk is established; the runner placement change is ready to merge.

Pre-merge checks: 25

Check name	Status	Explanation
Title check		The title clearly and concisely describes the primary CI behavior change: tip-dmg-build waits on the owned fleet instead of stalling in a Blacksmith queue.
Description check		The description gives a detailed problem statement, resulting behavior, risk assessment, scope, and tests run. It does not use the template headings or include explicit Changelog, Proof, or Checklist …
Docstring Coverage		No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within survi…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input		PASS. The diff changes only the DEV daily GitHub workflow's macOS runner-pool selection and its assertions. It adds `pick 2 --wait-owned` and updates test expectations. It does not change Cloud termin…
Cmux Swift Actor Isolation		The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. The authoritative diff contains no Swift files or Swift production changes, so it …
Cmux Swift Blocking Runtime		PASS: The pull request changes only a GitHub Actions workflow and a Python test. The reviewed diff contains no Swift files and introduces no Swift blocking or timing-based synchronization.
Cmux Browser Automation Off-Main		PASS: The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. The diff contains CI runner-pool selection and assertions, with no `browser.…
Cmux Expensive Synchronous Load		PASS: The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. It introduces no production Swift code or synchronous agent-history load.
Cmux Cache Substitution Correctness		PASS: The pull request changes only a GitHub Actions workflow and a Python test. It introduces no production Swift, TypeScript, or JavaScript change, and it does not substitute a cache for an authorit…
Cmux No Hacky Sleeps		PASS: The PR changes only a GitHub Actions workflow and a test. The workflow uses `--wait-owned` for pool selection, but workflow YAML is explicitly out of scope when the wait is CI orchestration. No …
Cmux Algorithmic Complexity		PASS. The production diff only changes the workflow command from `pick 2` to `pick 2 --wait-owned` and updates comments. It adds no loop, collection scan, sort, filter, join, or per-target rescan. The…
Cmux Swift Concurrency		The pull request changes only a GitHub Actions workflow and a Python test. The authoritative diff contains no Swift or other Apple source files, so it introduces no legacy Swift async, Combine, comple…
Cmux Swift `@Concurrent`		The pull request changes only a GitHub Actions workflow and a Python test. The authoritative diff contains no Swift files or Swift code, so the `@concurrent` criteria do not apply.
Cmux Swift Package Boundaries		PASS: The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. It introduces no production Swift changes, so the Swift package-boundary rul…
Cmux Swiftpm Lockfiles		The PR changes only the macOS runner-pool selection in `.github/workflows/cmux-next-dev-daily.yml` and related assertions in `tests/test_cmux_next_dev_tip_feed.py`. It does not change `Package.swift`,…
Cmux Swift Logging		PASS. The pull request changes only a GitHub Actions workflow and a Python test. It adds no Swift code or logging statements, so the Swift logging rule does not apply.
Cmux User-Facing Error Privacy		The diff changes only an internal GitHub Actions runner-pool workflow and its test. The new `echo` text is CI/operator diagnostic output and names Blacksmith, which the rule explicitly permits for int…
Cmux Full Internationalization		PASS: The pull request changes only a GitHub Actions workflow and its Python test. The workflow edits runner-pool selection and operational comments; the test edits assertions. No user-facing Swift te…
Cmux Swiftui State Layout		PASS: The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. The diff contains no SwiftUI code, `ObservableObject`, `@Published`, `@Obser…
Cmux Architecture Rethink		The diff changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. It contains no Swift architecture change, lifecycle ownership change, or Swift timing work…
Cmux Swift Auxiliary Window Close Shortcuts		PASS: The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. It adds no Swift code and does not add or change any cmux-owned window. The …
Cmux Source Artifacts		The PR changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. Both are intentional workflow and test source files. The diff adds no logs, screenshots, rec…
Cmux No Test Or Debug Seam In Production Source		PASS. The pull request changes only `.github/workflows/cmux-next-dev-daily.yml` and `tests/test_cmux_next_dev_tip_feed.py`. It changes no Swift file under a production `Sources/` path, so it cannot in…

Full details: Description check

Explanation

The description gives a detailed problem statement, resulting behavior, risk assessment, scope, and tests run. It does not use the template headings or include explicit Changelog, Proof, or Checklist sections, but the core review information is complete.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 1 files. (1 skipped: 1 unsupported.)

Full details: Cmux Cloud Persistent Session And Early Input

Explanation

PASS. The diff changes only the DEV daily GitHub workflow's macOS runner-pool selection and its assertions. It adds pick 2 --wait-owned and updates test expectations. It does not change Cloud terminal creation, cmux-tui clients, transports, manual renderers, PTY readiness, input routing, authentication, idempotency, revision fences, or leases.

Full details: Cmux Swift Actor Isolation

Explanation

The pull request changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. The authoritative diff contains no Swift files or Swift production changes, so it does not introduce or worsen Swift actor isolation issues.

Full details: Cmux Browser Automation Off-Main

Explanation

PASS: The pull request changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. The diff contains CI runner-pool selection and assertions, with no browser.* socket commands or changes to Sources/TerminalController.swift, ControlCommandExecutionPolicy.swift, worker routing, WebKit, or AppKit. The browser automation rule is not applicable.

Full details: Cmux Cache Substitution Correctness

Explanation

PASS: The pull request changes only a GitHub Actions workflow and a Python test. It introduces no production Swift, TypeScript, or JavaScript change, and it does not substitute a cache for an authoritative read in a persistence, history, undo, or snapshot path.

Full details: Cmux No Hacky Sleeps

Explanation

PASS: The PR changes only a GitHub Actions workflow and a test. The workflow uses --wait-owned for pool selection, but workflow YAML is explicitly out of scope when the wait is CI orchestration. No covered production TypeScript, JavaScript, shell, or build/runtime script changes introduce or worsen fixed sleeps, timers, polling, or wall-clock synchronization.

Full details: Cmux Algorithmic Complexity

Explanation

PASS. The production diff only changes the workflow command from pick 2 to pick 2 --wait-owned and updates comments. It adds no loop, collection scan, sort, filter, join, or per-target rescan. The picker handles a fixed runner-pool state and the changed job count is the constant 2; the other changed file is test code.

Full details: Cmux Swift Concurrency

Explanation

The pull request changes only a GitHub Actions workflow and a Python test. The authoritative diff contains no Swift or other Apple source files, so it introduces no legacy Swift async, Combine, completion-handler, or fire-and-forget Task pattern.

Full details: Cmux Swift Package Boundaries

Explanation

PASS: The pull request changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. It introduces no production Swift changes, so the Swift package-boundary rule does not apply.

Full details: Cmux Swiftpm Lockfiles

Explanation

The PR changes only the macOS runner-pool selection in .github/workflows/cmux-next-dev-daily.yml and related assertions in tests/test_cmux_next_dev_tip_feed.py. It does not change Package.swift, SwiftPM dependencies, .gitignore, Xcode package references, or any Package.resolved file. The SwiftPM lockfile conditions do not apply.

Full details: Cmux User-Facing Error Privacy

Explanation

The diff changes only an internal GitHub Actions runner-pool workflow and its test. The new echo text is CI/operator diagnostic output and names Blacksmith, which the rule explicitly permits for internal CI and build tools. No changed text reaches a cmux app UI, product CLI, or product API, and no secret, credential, raw upstream error, or sensitive payload is exposed.

Full details: Cmux Full Internationalization

Explanation

PASS: The pull request changes only a GitHub Actions workflow and its Python test. The workflow edits runner-pool selection and operational comments; the test edits assertions. No user-facing Swift text, string catalog, web UI, metadata, API response, rendered markdown, changelog, or locale-specific data changes are present.

Full details: Cmux Swiftui State Layout

Explanation

PASS: The pull request changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. The diff contains no SwiftUI code, ObservableObject, @Published, @Observable, GeometryReader, or SwiftUI view changes. The SwiftUI state layout check is not applicable.

Full details: Cmux Architecture Rethink

Explanation

The diff changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. It contains no Swift architecture change, lifecycle ownership change, or Swift timing workaround. The check is not applicable.

Full details: Cmux Swift Auxiliary Window Close Shortcuts

Explanation

PASS: The pull request changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. It adds no Swift code and does not add or change any cmux-owned window. The auxiliary-window close-shortcut rule is not applicable.

Full details: Cmux Source Artifacts

Explanation

The PR changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. Both are intentional workflow and test source files. The diff adds no logs, screenshots, recordings, temporary directories, caches, build output, dependency checkout, or broad artifact directory.

Full details: Cmux No Test Or Debug Seam In Production Source

Explanation

PASS. The pull request changes only .github/workflows/cmux-next-dev-daily.yml and tests/test_cmux_next_dev_tip_feed.py. It changes no Swift file under a production Sources/ path, so it cannot introduce a test or debug seam covered by this check.

Finishing Touches: 💡 1

🛠️ Fix failing CI checks: 💡

Commit to this branch
Create a new PR

📝 Generate docstrings

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-10-27T04:21:41Z

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Merge receipt for 1d70a135c1: every check was green at merge (10 verified; 16 skipped by policy). Full suite runs on main after merge.

teamleaderleo requested a review from lawrencecchen as a code owner October 27, 2026 04:11

teamleaderleo merged commit 01d0feb into main Oct 27, 2026
65 of 66 checks passed

teamleaderleo deleted the tip-dmg-wait-owned branch October 27, 2026 04:21

teamleaderleo mentioned this pull request Oct 27, 2026

ci(dev tip): the DEV pick counts Blacksmith room only on its jobs' own label #25913

Merged

This was referenced Oct 31, 2026

ci(dev-daily): verify and tip-dmg-build pick only std owned Macs #28803

Merged

ci(dev publisher): --wait-owned never picks a label RUNNER_POOLS marks dead; verify and tip-dmg-build fall back to owned Macs #30087

Merged

Navigation Menu

Repository navigation

ci(dev tip): tip-dmg-build waits on the fleet, not a stalled Blacksmith queue - #25902

ci(dev tip): tip-dmg-build waits on the fleet, not a stalled Blacksmith queue#25902
teamleaderleo merged 1 commit into
mainfrom
tip-dmg-wait-owned

teamleaderleo commented Oct 27, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

cursor Bot commented Oct 27, 2026

Uh oh!

teamleaderleo commented Oct 27, 2026

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

coderabbitai Bot commented Oct 27, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teamleaderleo commented Oct 27, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Risk

Summary by CodeRabbit

Uh oh!

cursor Bot commented Oct 27, 2026

Bugbot needs on-demand usage enabled

Uh oh!

teamleaderleo commented Oct 27, 2026

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

coderabbitai Bot commented Oct 27, 2026

Uh oh!

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

teamleaderleo commented Oct 27, 2026 •

edited by coderabbitai Bot

Loading