[lawrencecchen]

Summary

P0 for v2.2.10-opt ("I can't even make new workspaces"). The capture shows project import failed 5,311 times in 17 h (06:41Z to 23:52Z on 2026-10-30), on a Mac at 19.9 of 21.5 GB swap and load 13.

Cause: ProjectsImport relayed on every ChatsFeed publish, and a failure fed the next attempt. A failed project.sync left connectionID unset. So the next relay was fresh again: it cleared what was sent, sent project.sync (the daemon rereads every VS Code, Cursor and Zed source) and every harness's complete list again, each with a new idempotency key. Every agent turn changes a chat, so under load each change started one more full import.

Fix: a failed relay now waits a Backoff (2 s, doubling to 5 min, jittered) before it runs again. Index changes during the wait ride on that one retry, which reads the index as it is when it runs. A new daemon connection cancels the wait and relays at once. A success resets the backoff. After N failures, the number of attempts grows with log(time), not with the number of chat changes.

Commits:

1. daf63e3: the regression test and its seams (Target/Sent, an injected clock, indexChanged, inFlight, pendingRetry). This commit also moved the sends off the main actor (deliver became @concurrent). That is a behavior change, and it must not be in a red commit. Commit 3 takes it back. I did not rewrite the pushed history.
2. ac2d9a9: the fix (the backoff).
3. 7768eea: deliver runs on the main actor again, as before commit 1. Moving the sends off main is a separate perf change with its own main-thread measurement (recorded in followups).

The importer now reads the chat feed that start(feed:) receives, not services.chatsFeed. This is a pure seam: the one caller (AppDelegate.swift:317) passes services.chatsFeed, which is the same object. AppServices holds it for its life, so the weak reference does not change behavior.

Clean red ref: fix/p0-project-import-loop-red at 944e4ef (proof only, never merge). It is this head with the fix ac2d9a9 reverted: feat, plus the pure seams, plus the test, with no backoff and no off-main change.

Changelog

Fixed: a project import that fails no longer runs again on every chat change. It waits longer after each failure.

Testing

ProjectsImportRetryTests (CmuxNextAppTests), with a manual clock and a count of attempts. No wall time decides a result.

• aFailingImportWaitsItsBackoffInsteadOfRetryingOnEveryIndexChange: after a failure, 20 index changes add no attempt, and the retry after the wait runs once.
• aNewDaemonConnectionRetriesAtOnce
• aRetryThatSucceedsEndsTheBackoff

Red on 944e4ef (the clean red ref), green on 7768eea: the fleet runs are pending. This PR gets the job ids when they end.

Observed / Not run

• Observed: the capture (errors.list.txt). The prepush gate passed with no new violations (app-size warning only). Compile-check 38bad7a2dba11b559077ace6 passed (state done) at ac2d9a9. Compile-check b3dabc0044a7ac3966092561 of head 7768eea is running.
• Not run yet: the red and green fleet runs of the suite. I did not reproduce the loop in a live app.

No capture: no UI change.

Follow-ups (in followups.md): log the error type as public, so a capture says which request failed; continue past a refused list.

perf claim: none

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review ae6bb77: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28622 -f ref=fix/p0-project-import-loop -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); base red: the same failure on the base c7c9a16; does not block

[blacksmith-sh]

Found 10 test failures on Blacksmith runners:

Failures

Test	View Logs
/tmp/tmp2_bqa6xg/summary-missing.json (live):/summary-missing.json (live):	View Logs
/tmp/tmpq_trz3bt/summary-fail.json (live):/summary-fail.json (live):	View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/ [new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows	View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames	View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/ [tabs] debug.tab_switch timed 0 of 40 switches	View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames	View Logs
main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/ test_download_prints_the_active_profile_only	View Logs
stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/ Security has no stub	View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/ CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/ CoreFoundation, missing 1: ['_CFRetain']	View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/ CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_log2']/ CoreFoundation, missing 1: ['_log2']	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[lawrencecchen]

Review (claude-review-subagent-cw): SHIP at ae6bb77

I found no blocking defect. I read the merge, the two later commits and the net diff over feat 22abf5b. I did not compile or run tests. I used the fleet results that the resolver gave (compile 6ff02b629a0864a6594e1d99, ProjectsImportRetryTests step fa7c719fcaa0b14e1bd7fbce).

Merge f9f44be (remerge-diff)

• The merge keeps the two intents. From feat Chats: acpmux owns the chat list; the sidebar history holds only its window (P0 stall) #28055: the grouped read AcpmuxChatProjects.read(socketPath:) and batches(_:sent:). The feed snapshot, batches(of:) and the feed property are removed. From the PR: Target, Sent.failed, the Backoff retry and deliver on the main actor.
• The conflict resolution adds no other change. The logger is now static (Self.logger), because deliver is static.

Move 867d892 and public import ae6bb77

• The behavior stays the same. AppServices.swift:489-497 gives the same guard as the old relay(): a local connection plus projectList support. It also gives the same socket (chatsFeed?.socketPath) and the same observed read (machines.local.connection).
• The closures capture [weak self], so AppServices and ProjectsImport have no retain cycle. Target(daemon:) is made new for each relay, and only the sending task holds it.
• Re-arm: armConnection() (ProjectsImport.swift:173-183) reads connection() inside withObservationTracking. If AppServices is gone, the read registers nothing, the same as the old guard. The tests pass { nil }, so they arm nothing.
• nonisolated deinit (:158-161) cancels sending and retry.wait.
• Isolation: deliver (:137) is a static member of a @MainActor class, so the sends stay on the main actor, as in 7768eea. target, socketPath and connection are @MainActor closures.
• Layering: CmuxNextAgentLanes already depends on CmuxNextAgentPane, CmuxNextDaemon and CmuxNextWakeups (Package.swift:1004-1007), so the move adds no target edge. public import CmuxNextDaemon is necessary, because Target(daemon:) takes a DaemonConnection and Batch.entriesJSON is [JSONValue].

Read failure (ProjectsImport.swift:209-216), not blocking

• The read failure does not set failed, so it does not back off. This is the same as feat. I do not think that it brings back the P0.
• The P0 was a failed project.sync that kept every relay fresh. Each relay then made the daemon read all editor sources again.
• A read failure on a synced connection sends nothing to the daemon. It makes one coalesced _acpmux/chat_projects request for each index change, with one in flight at a time.
• A read failure on a fresh connection still sends project.sync. If that sync fails, the backoff starts.
• Follow-up, optional: set failed on a read failure too. Then a slow acpmux (5 s deadline under swap) also waits the backoff.

Not blocking: a reconnect during a failed send waits the backoff (PR logic, not the merge)

• Example: the daemon connection changes while a send is in flight. relay() sets sendAgain (:188-190). Then the send fails, and finished clears sendAgain and calls scheduleRetry on the old id (:227-231).
• The new connection then waits up to the current backoff (maximum 300 s), unless an index change comes first. This does not agree with the class doc "only a new daemon connection retries at once" (:15-17).
• A send often fails because the connection goes down, so this race can occur.
• Possible fix: in the failed branch, if target()?.id != id, relay at once and do not schedule a retry. Add a test for it.

Nits

• The PR body is out of date. It says that the importer reads the feed from start(feed:) and that the file is in CmuxNextApp. It also says that the red and green fleet runs are pending. Update it for the merge, the move and the job ids.
• plans/cmux-next/projects.md:47 says "ProjectsImport (CmuxNextApp)". docs/swift-to-rust-audit.md:76 gives the path App/Projects/ProjectsImport.swift. Both are now out of date.

[lawrencecchen]

Conflict wave: merged feat-cmux-next 22abf5b into this branch (merge f9f44be), then two commits.

• Conflict in ProjectsImport.swift: feat's Chats: acpmux owns the chat list; the sidebar history holds only its window (P0 stall) #28055 (acpmux groups the projects; AcpmuxChatProjects.read, no feed snapshot) and this PR's Target/Sent seam with the backoff retry. Both kept. A failed _acpmux/chat_projects read sends nothing, as on feat (no backoff for it).
• 867d892: the merge put CmuxNextApp 86 lines over its arch ratchet, so ProjectsImport moved to CmuxNextAgentLanes with no app types. AppServices passes the local daemon as ProjectsImport.Target(daemon:), the acpmux socket path and the connection to follow. The test builds it with a fake target and no AppServices.
• ae6bb77: public import CmuxNextDaemon (compile job a1d2f943314a refused the public API without it).

Fleet evidence at ae6bb77:

• Compile (cmux-ci compile-check --package CmuxNext --app, job 6ff02b629a0864a6594e1d99): CmuxNext passed (177 s), app passed (41 s).
• ProjectsImportRetryTests (step fa7c719fcaa0b14e1bd7fbce, aws-m4pro-2): 3 tests passed.
• Red ref 944e4ef (step 057c589051f913d339a4d4f6): the suite failed, but by a crash in AppServices.swift:87 (implicitly unwrapped nil while the old test builds AppServices), not by the test's expectation. So this is not yet a clean behavioral red. The moved test no longer builds AppServices.
• Review: SHIP at this head (comment 6552558377).

I did not add ready-to-land: the PR had none, and its red evidence is not clean yet (above). The PR body also still says the fleet runs are pending.