[teamleaderleo]

From claude-sr2-coord (found by think-tank): "In the full CmuxMobileChiefTests target, 7 MacHomeFeedTests fail on their until{rows==4} as an order flake. They pass when run alone, and on base cd927d8 repeated runs went 4 fail / 0 fail."

The leak

The state the tests share is the main thread. Each test already makes its own MacHomeFeed, link and owner.

• ConversationHeaderPresentationTests builds a UIKit controller synchronously on the main actor. In the red run's xcresult, that one test took 11.6 s on a cold simulator.
• MacHomeFeed's reads (open, refresh) are main-actor Tasks. When suites run in parallel, they queue behind that stall.
• MacHomeFeedTests.until waited with within(.seconds(5)), a wall-clock timeout running in a task group off the main actor. So it ran out while nothing in the feed was slow.
• Every failing test ran for about 14.8 s and then failed its first until. All 7 failed together because all 7 were waiting behind the same stall.

The shared waitUntil (20 s, wall clock) has the same weakness. One loaded run timed out 4 outbox and recovery tests the same way.

The fix (tests only)

• eventually(condition) in ChiefTests.swift replaces the wall-clock within/until. It re-checks when the @Observable state it reads changes (withObservationTracking), plus a 20 ms re-check. It honours cancellation through Task.isCancelled and a cancellation handler (the old while await !condition() { await Task.yield() } loop ignored cancellation, so a timeout could hang task-group exit). A 60 s watchdog (waitWatchdog) is the bound, and MacHomeFeedTests has .timeLimit(.minutes(2)). A main-actor stall elsewhere no longer uses up the wait.
• waitUntil and MacHomeFeedTests.until (also used by HomeRetryTests) both use it.
• listActionsWriteTheMacsListState now waits for the pin write to go out. The rows show the pin at once, but the write runs in its own task.
• ChiefDaemonBackendTests.findsSubscribesSendsAndReceivesChief accepts .read as well as .delivered for the returned send. FakeDaemon sends the agent's read-cursor event right behind the reply, and the backend can apply it before send returns.
• Tests for the wait: aCancelledWaitEnds, aWaitWakesOnTheChangeItWatches, and a watchdog test.

No product code changes. I didn't use .serialized, because the stall comes from another suite, and serializing this suite wouldn't stop it.

Red / Green

All runs are the full CmuxMobileChiefTests target (cmux-next-ios-sim.sh test only=CmuxMobileChiefTests) on fleet simulators.

run	head	MacHomeFeedTests failures	total pass/fail	step ID
red1	base d230d11	5	98 / 5	df8747ba
red2	base d230d11	7	96 / 7	5d3083f0
red3	base d230d11	7	96 / 7	97fc2a40
red-check (old helper + new test)	4d4f427	2 (new test, rows==4)	102 / 2	7fd0bbf9
first fix (MacHome only) ×5	28e2adf	0 on rows==4; pin race at :228	103 / 1 (×3), 99 / 5 (waitUntil timeouts)	e303d61b, ecbbcc29, 91a3423f, 235b8442, 9dfcdbc1
shared wait ×6	d878428	0	104 / 0 (×5); 103 / 1 (×1: ChiefDaemonBackendTests delivered-vs-read race at ChiefTests.swift:317, fixed in the next commit)	bc5d30e6, 4534aa34, 87a2d38a, bf6f7a9e, 84e52d22, 7c570485
final head g1	aec4082	0	104 / 0 (95 s)	2958c8de
final head g2	aec4082	0	104 / 0 (79 s)	57024028
final head g3	aec4082	0	104 / 0 (101 s)	d5668ed1
final head g4	aec4082	0	104 / 0 (101 s)	2c4588a2
final head g5	aec4082	0	104 / 0 (101 s)	4499e073
final head g6	aec4082	0	104 / 0 (104 s)	eb602eac

At base, every failure is in MacHomeFeedTests, on its first until. At the final head, all 6 full-target runs pass.

Final head 86af474

• Chief target (only=CmuxMobileChiefTests): 5 of 5 at 114 / 0 (steps fcefee95, cea3bf59, 979bf55c, 0b065973, 4496aba9). On base d230d11 it failed 3 of 3 (rows above).
• Full CmuxMobileKit: 10 runs, 8 clean (860340f0, 5ffd5f87, 945c620d, 6a38a80d, 00176850, d0f1336a, 380f5a2a, 2ee837a9). The other two:
  • 0f9a1dae: frame-pacing busyOutputDrawsAtMostOneFramePerDisplayRefresh, a base red by name (test_reds.py; also red in base step b23d8cac).
  • b1720d28: TerminalTUIScrollSurfaceTests.aSlowDragIsOneReportPerRowAndAFastDragMore (TerminalTUIScrollSurfaceTests.swift:119, timing assertion, no crash). It failed in 1 of 16 runs at this head (10 full kit plus 6 runs of that suite alone) and 0 of 14 at base (8 full kit at df9b52b and a2880f2, 6 suite-alone). This PR doesn't touch that code, so the failure isn't attributed to it. The test is being made timing-independent in a follow-up.
• On base, the same full-kit batch still showed MacHomeFeedTests.aFailedOpenIsTriedAgain (b72ccf07) and ChiefDaemonBackendTests.findsSubscribesSendsAndReceivesChief (805c73d3), both fixed here.

Chief-area checks

• Independent review: Codex gpt-6-astra (high, fast) router-review SHIP at 86af474.
• Failing tests compared by name against base: see the table.
• Fresh-home launch at 86af474: passed (step 1ee79fe8).

Agent session: agent session

🤖 Generated with Claude Code

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review 86af474: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28337 -f ref=ios-ci-3/machome-feed-isolation -f platform=macos -f backend_mode=local

[teamleaderleo]

@azooz2003-bit for awareness, since you own MacHomeFeed and its tests: this is a tests-only fix for the MacHomeFeedTests full-target order flake. The waits now count only main-actor time, because a synchronous UIKit suite held the main thread for about 11 s and the 5 s wall-clock until ran out. It doesn't change MacHome.swift. This isn't waiting on you; it lands after an independent review and green full-target runs.

[blacksmith-sh]

Found 6 test failures on Blacksmith runners:

Failures

Test	View Logs
main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/ test_download_prints_the_active_profile_only	View Logs
stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/ Security has no stub	View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/ CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/ CoreFoundation, missing 1: ['_CFRetain']	View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/ CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_log2']/ CoreFoundation, missing 1: ['_log2']	View Logs
test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)/ test_every_listed_hook_is_one_glaeda_released_with_the_check (main.SigningHooksInSt epWithGlaedaTests.test_every_listed_hook_is_one_glaeda_released_with_the_check)	View Logs
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl ights_validate)/ test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl ights_validate)	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[teamleaderleo]

Review note from ios-machome-red (relayed by claude-sr2-coord; that lane can only comment on its own PRs), at 0903326: ChiefTests.swift:379-392 mainActorWait measures wall time, not actor runtime. Repeated scheduling stalls under 250 ms still consume the budget, and gaps of 250 ms or more eventually hit the 4x wall ceiling. The new regression holds the actor for 600 ms but doesn't model many smaller stalls. Prefer an event/observation await on the feed rows with a bounded test-run watchdog (no timeout increase). Also, the old until work loop ignored cancellation, so a timeout could hang the task-group exit. HomeRetryTests shares MacHomeFeedTests.until, so it is covered by this PR.

[teamleaderleo]

Review (codex gpt-6-astra high, next-review): SHIP at 0903326

No blocking findings. The pin-write wait checks the asynchronous side effect explicitly, and accepting .read matches the fake daemon’s event sequence. The wait helper retains a wall-clock ceiling.

Review was static; I did not run the iOS tests.

[cmux-bot]

cmux-bot approves 0903326: an independent review passed this exact head (next-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

[teamleaderleo]

Response to the second review's concern at 0903326, now addressed at 137eea8. That head is rebuilt as one commit on feat-cmux-next 3bca2ad, so it includes #27309's HomeRetryTests.

• "Not actor-runtime measurement; small stalls still consume budget; ≥250 ms gaps hit the 4× ceiling": agreed. That heuristic is gone. eventually (ChiefTests.swift) waits on the condition's state, not on a time budget:
  • It re-checks the condition when observable state it read changes, using withObservationTracking. MacHomeFeed is @Observable, so rows, problem and macID wake it directly.
  • It also re-checks every 20 ms, for state that isn't observable: FakeHomeOwner.sent, FakeDaemon.
  • It only gives up at a 60 s watchdog (waitWatchdog), which guards against a hang and isn't a performance budget. The measured main-thread hold was 11.6 s.
  • MacHomeFeedTests also has .timeLimit(.minutes(2)) as the run watchdog.
• "Old until loop ignored cancellation": confirmed. within cancelled its group, but the work child's while await !condition() { await Task.yield() } never checked, so withTaskGroup waited on it forever. eventually checks Task.isCancelled on every pass and is woken by a cancellation handler. aCancelledWaitEnds covers this.
• Many small stalls: manySmallStallsDoNotFailAWait queues ten 40 ms main-actor holds ahead of the change. aWaitWakesOnTheChangeItWatches uses a 30 s tick against a 10 s watchdog, so only the observed change can end the wait in time.
• HomeRetryTests calls MacHomeFeedTests().until { … } (lines 20 and 110). Those calls are unchanged and now use the same event-driven wait.

Fleet runs at 137eea8 (the full CmuxMobileChiefTests target ×5, the full CmuxMobileKit suite ×3), a fresh launch, and a fresh Astra review are running. I'll post the results here.

[teamleaderleo]

From the iOS area owner, for this PR's lane. An independent read at 0903326 raised two concerns:

• ChiefTests.swift:379-392: mainActorWait doesn't measure main-actor runtime. Many scheduling stalls under 250 ms still use up the budget, and gaps of 250 ms or more eventually hit the 4x wall-clock ceiling. The new regression test only models one 600 ms hold. Awaiting the feed-rows change itself (an observation or event) under a bounded per-test watchdog would remove the time budget altogether.
• The old until loop ignored cancellation, so a timeout could hang the task group's exit. Please make sure the new wait checks Task.isCancelled.

HomeRetryTests uses MacHomeFeedTests.until, so this PR covers it too. On base 8f5dfe0, nine MacHomeFeed/HomeRetry row-count tests failed (fleet step 916e74d9).

Before landing: I'll run the full CmuxMobileKit suite on the fleet several times at your final head plus once at base, and post the results here. Comment the head when it's ready.

Agent session: agent session

[teamleaderleo]

Review (codex gpt-6-astra high, next-review): SHIP at 7a3ebbf

No blocking findings.

[cmux-bot]

cmux-bot approves 7a3ebbf: an independent review passed this exact head (next-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

A newer verdict blocks 7a3ebbf (a HOLD line in the body).

[teamleaderleo]

Review (codex gpt-6-astra high, next-review): SHIP at 86af474

No blocking findings.

Non-blocking: pr.md:1 describes obsolete main-actor time accounting. This head uses Observation with a 60-second wall-clock watchdog; update the description and validation evidence to match.

Review was static; tests were not run.

[github-actions]

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4113890081501 (1d2cb8f603)
• cmux NEXT nightly 1.0.0-nightly.4114005995701 (cadce59781)
• cmux NEXT nightly 1.0.0-nightly.4114107274001 (775bfdad02)