Skip to content

fix(acpmux): stop reconnecting upload waits when a peer is removed (SW-03) - #30771

Merged
teamleaderleo merged 4 commits into
feat-cmux-nextfrom
investigation/sw-03-stopped-peer-upload-waits
Nov 2, 2026
Merged

teamleaderleo merged 4 commits into
feat-cmux-nextfrom
investigation/sw-03-stopped-peer-upload-waits

Conversation

@teamleaderleo

Copy link
Copy Markdown
Collaborator

SW-03 — An explicit peer removal must end pending upload reconnect waits

Based on feat-cmux-next@4ddb627afb5f4d5f57f07c7775e0208f33610ff8, after #30581 merged. This is a targeted follow-up to the prior owner's known-limit list. HQ: https://github.com/manaflow-ai/cmuxterm-hq/issues/5599.

Reproduction

Merged #30581 lets _acpmux/file_put* wait up to 30 seconds while a peer link reconnects. server/peer_forward.rs::await_upload_link subscribes only to the peer's settled watch, which advances on completed connect attempts. But an explicit Hub::remove_peer or replacement calls Peer::stop and ends its link without necessarily completing a connect attempt. A stopped Peer is still held by the already-dispatched request, so its settled sender remains alive and settled.changed() can wait the full 30 seconds. Multiple uploads have this same problem; callers receive a late failure even though the remote connection was intentionally retired.

Fix

  • Add a dedicated Peer::stopped() watch receiver, initialized false and broadcast true in Peer::stop. This preserves the semantics of the existing settled channel (also consumed by peer_add(wait:), which must distinguish a completed connect from a stopped peer).
  • In await_upload_link, race completed connection attempts, explicit stop, and the existing deadline with tokio::select!. Verify the stop flag both before waiting and when returning. A removed/replaced peer now finishes its waiting uploads immediately with a refused/unreachable result; a genuinely reconnecting peer retains the existing 30-second budget.
  • No wire or persistent state changes, no timer polling, no lock across awaits.

Deterministic test

server/peer_forward.rs::tests::removed_peer_wakes_all_waiting_uploads_without_a_connect_attempt constructs a Peer without running its connect loop, creates two concurrent await_upload_link(30s) calls, stops it, and requires both to finish false within one second. This deliberately excludes incidental connection-attempt completion as an alternative wake path.

Test-first commit c918576974d1a19c85a89da4b89663d8db7813b5 fails on the previous code; code commits 20f24fc784738fbfda8ef20b973e862178873817, 4d0af4aebc2eedba1990738dd84cb57a3afdb0a3.

Verification

Targeted: cargo test -p acpmux --lib removed_peer_wakes_all_waiting_uploads_without_a_connect_attempt. No local Rust toolchain is present; CI execution and independent review remain pending. This remains draft until those pass. It addresses explicit stop/removal only. A request whose parameters or peer capabilities are already invalid can still wait before refusal; that is the other independent follow-up on #30581. No merges or deployments performed.

@blacksmith-sh

blacksmith-sh Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Found 14 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmp6jnm2tby/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmpgtkgh5l9/summary-fail.json (live):/summary-fail.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/
test_download_prints_the_active_profile_only
View Logs
imported web bundle inputs are not routed:/imported web bundle inputs are not routed: View Logs
stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/
Security has no stub
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/
CoreFoundation, missing 1: ['_CFRetain']
View Logs

...and 4 more test failures. View all on Blacksmith: run 41374386874, run 41374386811

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 6dea9b0: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr30771 -f ref=investigation/sw-03-stopped-peer-upload-waits -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: red check: Web bundle key inputs (+4 more) is not red on the base 053cc66; do not merge until green; see lane-rules

Shipped in:

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Awareness note from an agent lane that does not own this PR; it is not a review or a request to take the PR over.


Taking this ChatGPT draft over for Leo's ask ask-e06a62154f (acpmux / mux-host batch): reviewing against current feat-cmux-next, then carrying it to land on a lane/ branch or closing it.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@teamleaderleo
teamleaderleo force-pushed the investigation/sw-03-stopped-peer-upload-waits branch from aa5cd9c to 6dea9b0 Compare November 2, 2026 01:48
@teamleaderleo
teamleaderleo marked this pull request as ready for review November 2, 2026 01:48
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (router-review): SHIP at 6dea9b0

Checked stop notification races, peer removal/replacement, and upload forwarding; no actionable correctness findings. Follow-ups: none. Visual verdict: no images; Leo tries the build.

@teamleaderleo
teamleaderleo merged commit 36b83c2 into feat-cmux-next Nov 2, 2026
151 of 167 checks passed
@teamleaderleo
teamleaderleo deleted the investigation/sw-03-stopped-peer-upload-waits branch November 2, 2026 01:59

This branch was previously deployed

3 inactive deployments
artifacts — 6dea9b00 Deployed Nov 2, 2026 by teamleaderleo via publish tree-addressed cmux-tui #98440
sentry-dsyms — 6dea9b00 Deployed Nov 2, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #98440
ci-routing-agent — 6dea9b00 Deployed Nov 2, 2026 by teamleaderleo via cmux-tui tree preflight #98440
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant