[teamleaderleo]

The iOS kit workflow selects the same fixed macOS label on every attempt, so a job can stay queued when that pool starts nothing. This change uses the existing simulator-aware picker to select idle owned capacity first, with Blacksmith macOS as fallback. It accounts for kit runs as one simulator and emits the existing rescue markers. The executing rescue and janitor on main support this workflow through #28431, including its three-minute stall budget and open-PR retry path.

Queue and snapshot reads try the routing token before the job token, with one bounded fallback. Both kit routing-token mint steps now request Actions read access, as the same routing App already does in ci-macos.yml. This reads workflow runs, jobs and capacity-snapshot artifacts through the App quota when the job-token quota is exhausted. No new App, secret, writer scope or fork access is introduced. Production queue improvement is not yet measured.

Regression coverage checks picker names, artifact pins, running and completed simulator accounting, off-fleet placement, quota failures, snapshot fallback, rescue triggers, and the stall budget. Dispatches use the existing routing credential environment.

Performance

metric	before	after	delta	harness or run link
Reported queue wall time	About one hour without a runner	Production improvement not yet verified	Not yet measured	Original incident, attempt 2
Runner-minutes while the job has no runner	0	0	0	Queue time is not runner execution time

No production speedup is claimed. The earlier picker run still waited about 3h48m after quota exhaustion forced Blacksmith placement. The focused simulated-clock regression verifies an owned kit stall budget of at most 180 seconds. Existing bounded rescue watches are reused; no polling process is added.

Warm resources: this changes placement, not build commands or cache keys. It retains the existing Cargo cache. Python routing checks need no Mac compile or DerivedData. Simulator comparisons should reuse warm DerivedData and module caches; this change does not establish additional persistent Swift caches.

Validation: 248 remote Linux runner-pool and cross-branch rescue-source tests passed at 7d4dc54 in run 41150119507. After merging current base, all 25 runner-routing-table tests pass locally. Focused remote checks will be repeated at the final head, including the routing-table tests. The obsolete kit picker_followup entry added by #28600 is removed while retaining its apple-swift-tests classification. Earlier main-rescue tests passed in run 41096022154. The fleet comparison ran the full CmuxMobileChiefTests target on the same simulator class without a crash: merge base 4543835 had 102 passed and 9 failed; c39d28e had 103 passed and 8 failed. Every head failure also failed by name at base. Both refs predate #28337's five-second helper rewrite; the newer picker commits do not change Home sources. Current base is now merged, including #28337’s helper rewrite; the earlier comparison remains historical evidence for the previous head. Fresh review remains pending at bf75881. The original incident run was left untouched.

No capture: CI workflow and picker only

Agent session: agent session

CI red comparison at final head

The automatic cmux-next CI guard group passed at bf75881c3c6ca5016dd2a55718a077b7ec23ad81, covering the runner-pool, owned-pool-source, and routing-table guard group. The tooling-tests job 124403733443 has two base reds: the unwired tests/test_check_objc_selectors_step.py linux-guard registry entry and AppCrateLicenseGate.test_the_ten_first_party_gpl_crates_are_named_exactly with the extra cmux-chats-mirror. Base feat-cmux-next 5f1a9ce9 reproduces both in push run 41159138738, tooling-tests job 124412542698. They are not caused by this PR.

Failure comparison at bf75881

The remaining red jobs are environmental/base reds, with their causes recorded here:

• The iOS simulator build run 41156282579, job 124415476256 failed before checkout in the fleet hook: it waited for another fleet build for 425 seconds, then refused because that build was waiting for the host. The picker itself succeeded in runner job 124415476487 with glaeda-std-xcode-26.6, glaeda-ios-sim, and persistent=true, exactly matching base run 41150994994, runner job 124387387900. This is a fleet admission conflict before any iOS step, not a picker verdict.
• The dependent ios-tests job 124423945800 explicitly reports ios-simulator-build did not pass: failure; it did not run a routing suite and has no independent routing failure. Base's corresponding job 124396711118 passed.
• The kit job 124406107026 failed three simulator tests: TerminalSurfaceFramePacingTests.busyOutputDrawsAtMostOneFramePerDisplayRefresh, TerminalBenchProbeAccountingTests.probeCountsTheResetAndAnIdleEchoShowsAtOnce, and NativeBrowserTabTests.aMacThatForwardsOpensItsLocalhostPageOnThePhone. Each reproduces on base ancestors: frame pacing in base run 41138142557/job 124346936628 and 41147747527/job 124376714398; TerminalBench in 41147747527/job 124376714398; NativeBrowser in 41156807832/job 124408736989, 41153382578/job 124400814436, and 41151767516/job 124390415838. Those heads are ancestors of base 5f1a9ce9; these are established base flakes.
• The fleet Release compile run 41156282526/job 124406297814 received repeated controller 404s (and two 429s) for the compile result and ended no result after its 40-minute wait. Base run 41159138640/job 124415170265 returned attempt 1: pass within seconds. The head failure is a controller result publication outage, not a changed runner-pool path; this PR does not edit the cmux-next compile workflow.

The automatic guard group passed at exact head in job 124403733572; tooling-tests reds are identical on base and documented above. No capture: CI workflow and picker only

Agent session: agent session

[teamleaderleo]

Review (codex-review-box/gpt-6-astra high, lane-place): BLOCK at efb6cfe

BLOCK

1. .github/workflows/cmux-next-ios-kit.yml:42 omits Packages/iOS/CmuxMobileSupport/** from both trigger path lists, even though CmuxMobileKit/Package.swift depends on that package. Changes there will not run the new kit tests, leaving regressions unvalidated.

[blacksmith-sh]

Found 1 test failure on Blacksmith runners:

Failure

Test	View Logs
test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)/ test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review e226ef1: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28145 -f ref=lane/ios-kit-runner-picker -f platform=macos -f backend_mode=local

web and size checks: green

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4122680534501 (b6eb84916c)
• cmux NEXT nightly 1.0.0-nightly.4122811790801 (228b1fb697)
• cmux NEXT nightly 1.0.0-nightly.4122926028501 (e4a99b0dc7)

[teamleaderleo]

Review (codex-review-box/gpt-6-astra high, lane-place): BLOCK at 6a68124

BLOCK

1. ci-owned-pool-rescue.yml:57-65,111-135 does not include the new cmux-next-ios kit tests workflow in workflow_run.workflows or SOURCE_WORKFLOW_PATHS. Consequently, the rescue workflow never wakes on kit runs, so an owned runner job that queues or fails admission can remain stuck until the periodic sweeper runs. The focused rescue tests also fail because these declarations are missing.

[teamleaderleo]

Review (codex-review-box/gpt-6-astra high, lane-place): BLOCK at 9e27cb5

BLOCK

1. tree/scripts/ci/ios_runner_pool.py:517-518 and 536-537 — kit runs are always charged one owned Mac and simulator, even when the picker sent them to Blacksmith. Kit run titles contain no runner field, so the snapshot fallback cannot identify off-fleet runs; after the placement grace period, charged_sim_jobs/charged_jobs should consult Placements like the test-ios path. Otherwise a Blacksmith kit run permanently consumes owned capacity for the duration of the run and can unnecessarily route later jobs away from the owned pool.

2. tree/.github/workflows/ci-owned-pool-rescue.yml:63 — the added workflow name makes the existing workflow-source test fail. Running python3 -m unittest tests.test_ci_owned_pool_rescue.Workflow.test_runs_when_dispatched_or_for_a_screenshots_or_nightly_run reports cmux-next-ios kit tests as an unexpected trigger name; update tests/test_ci_owned_pool_rescue.py:2130-2137 (or the workflow) so the repository test suite passes.

[teamleaderleo]

Review (codex-review-box/gpt-6-astra high, lane-place): BLOCK at d4fa8ae

BLOCK

1. scripts/ci/ios_runner_pool.py:517-518 — charged_sim_jobs() unconditionally charges every cmux-next-ios-kit.yml run one simulator, ignoring placements.off_fleet(). Kit runs routed to Blacksmith (including retries or runs after the owned pool is full) therefore still consume owned simulator capacity in the snapshot fallback, potentially causing subsequent kit runs to overflow unnecessarily. Apply the same placement check used for the test-ios title-based path.

[teamleaderleo]

Review (codex gpt-6.1-sol high fast, router-review): SHIP at fb6e162

The head is a lander base merge (update-branch) of 093a7aa, done to get a compile verdict (runs before #29060 never recover on rerun) and current guards. No correctness findings in the PR's own diff.

[teamleaderleo]

Review (codex gpt-6-astra high, next-review): SHIP at 9c69c48

SHIP at 9c69c48

Visual verdict: no images; Leo tries the build.

Focused routing, rescue, janitor, and picker tests pass. No correctness findings.

Follow-ups: none

[cmux-bot]

cmux-bot approves 9c69c48: an independent review passed this exact head (next-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

[teamleaderleo]

Review (codex-review-box/gpt-6-astra high, lane-place on a fleet lane host; posted by the research overseer because the box host could not write to GitHub): BLOCK at 9c69c48

BLOCK at 9c69c48

Visual verdict: no images; Leo tries the build

1. .github/workflows/cmux-next-ios-kit.yml:83 — [P1][blocking] The picker job is skipped whenever vars.RUNNER_POOLS is empty. Since kit-tests requires runner, the entire kit test workflow is then silently skipped, removing the previous Blacksmith fallback and masking the required check.

2. review/images/README.txt:1 — [P2][follow-up] No after screenshots were supplied for visual review.

Follow-ups:

• [P2][follow-up] No after screenshots were supplied.

[teamleaderleo]

Review (codex-review-box/gpt-6-astra high, lane-place on a fleet lane host; posted by the research overseer because the box host could not write to GitHub): SHIP at e226ef1

SHIP at e226ef1
Visual verdict: no images; Leo tries the build

1. [P2] [follow-up] No after screenshots were supplied for visual comparison.

Follow-ups:

• [P2] [follow-up] No after screenshots were supplied.