Skip to content

acpmux: an ssh peer that connected before the remote wrote its peer token no longer stays served as Web - #24105

Merged
teamleaderleo merged 13 commits into
feat-cmux-nextfrom
cmux/acpmux-peer-send-mac-perms
Oct 26, 2026
Merged

teamleaderleo merged 13 commits into
feat-cmux-nextfrom
cmux/acpmux-peer-send-mac-perms

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

deputy-2 (cosmic-gray), verbatim: "from cosmic-gray, acpmux send air-blue/gate-rules ... is refused with "This chat runs with your Mac's Claude permissions. Start a new chat from this device to control it remotely." The session was imported onto air-blue (a Mac) by acpmux session import and runs bypassPermissions; sending through air-blue's own daemon works. Coordinators on cosmic-gray have to steer lanes on other hosts, so a peer that the host lists in acpmux host (same user, ssh-authenticated) should be allowed to send, or the import should record the session as remotely controllable."

Cause. The policy is already what deputy-2 asks for. A proven peer daemon (Control::Peer) is exempt from the Mac-permissions refusal (web_control_verdict returns early for anything but Control::Web). The link from cosmic-gray to air-blue wasn't proven: acpmux host ls on cosmic-gray showed air-blue … servedAs: remote while cmux16 and huge-blue showed peer. air-blue's daemon had just restarted for the #23880 rollout, and cosmic-gray reconnected inside the window where the new daemon is listening but hasn't yet written run/peer.token; the old daemon had already removed its own. In the air-blue log, the listener came up at 02:22:05 and the token was written at 02:22:08, and cosmic-gray connected at 02:22:10 with an older timestamp still in flight. The ssh token read came back empty, so the connection was served as Web. It stays Web, holding every send to the Web's rules, until the link next drops.

Fix (narrow, no policy change). Two parts, both on the ssh:// path only:

  1. Before connecting: if the token read finds no peer token, the peer reads again after 1 s, 2 s and 4 s, then connects with whatever it has. This covers the window where a starting daemon is listening but hasn't written run/peer.token yet.
  2. Right after initialize, before any watch or attach: if the remote served this link as Web (remote), the peer reads the token once more. If the remote now has a token this connection didn't present (it restarted between the read and the handshake), the link is dropped as an ordinary error. The normal disconnect path then runs (pending requests fail, Disconnected notice, backoff), and the next connect presents the new token. Because nothing has been watched, attached or sent on the link yet, nothing is lost or replayed.

A remote that is too old to make a token, or one that serves this daemon as Web for another reason (it has the same token we presented), stays Web, as before. That costs at most five ssh reads per connection and never loops. peer_auth::is_peer still needs the launch's token and a loopback transport, and the Web refusal (#23792 / D13) is untouched.

I rejected deputy-2's option (b), recording imported sessions as remotely controllable. It would widen the Web rule for every Web client (paired devices and the dashboard), not just peers.

Remote relay analysis (skills/cmux-socket-policy/references/remote-relay-authorization.md)

  • Local command or content execution: none added. The extra token reads are the same read_config_argv ssh command the peer already runs at every connect (fixed argv, the destination parsed by ssh_target, which refuses anything option-shaped). There are at most four extra reads per connection: three while the remote has no token, and one if the remote served the link as Web.
  • Access to unowned objects: none added. A link becomes Peer only by presenting the remote's own launch token over a loopback/ssh transport, which is the existing proof. The fix makes the daemon present that proof sooner; it can't create one.
  • Local-state exposure: none. The token is read over the user's own ssh access, as before. It's never logged or returned, and it never crosses plain ws:// (carries_peer_token is unchanged; this path is ssh-only).
  • Policy tests: the new tests/peer_token_race.rs drives a real remote daemon through a stand-in ssh whose first token read finds none. The existing peer_origin.rs (token plus dashboard token makes Peer; the dashboard token alone stays Web; no RPC returns the token), peer_transport.rs (the token never crosses plain ws:// to a non-loopback host), ssh_peer_injection.rs and remote_guard_modes.rs (a Web connection is still refused on a Mac Claude session) all pass on the head.

Testing

Focused command, run on huge-blue:
cargo test -p acpmux --test peer_token_race. It has three cases, each a real remote daemon behind a stand-in ssh: the remote has no token on the first read (late), the remote shows the previous run's token on the first read (stale), and the remote never has a token (never).

  • Base peer.rs at merge-base 986cfe3, with this test file: FAILED, 3 of 3. late and stale: the peer is not served as peer: {… "servedAs":"remote" …} after 60 s. never: reads of a remote with no token left: 1 right: 5.
  • Head: ok, 3 passed in 12.6 s.

Also run on the head: cargo test -p acpmux --test peer_origin --test peer_transport --test ssh_peer_injection --test remote_guard_modes --test peer_degraded --test peer_lease --test peer_canonical_endpoint (all ok), cargo test -p acpmux --lib peer (57 passed), cargo clippy -p acpmux --all-targets and cargo fmt --check (clean).

Review: Codex at xhigh reviewed 6e48a42 and returned FIX. Its points were that a relink after the handshake could lose or replay queued RPCs and skipped the disconnect notice, and that a token rotated after a non-empty read was never retried. Both are addressed in 0c0b6e1 (above). Re-review is on the final head.

Observed / Not run

  • Observed: the red then green above, and servedAs: remote for air-blue on cosmic-gray's live host ls.
  • Not run: the fix on the live hosts. It reaches air-blue and cosmic-gray once their daemons run a build that includes it. Until then, a reconnect (restarting cosmic-gray's daemon, or the link dropping) clears the stuck link.

Changelog

Fixed: a cmux machine whose peer restarted no longer stays held to remote-device rules, which refused sends to chats running with the Mac's own Claude permissions.

risk:high (peer authentication path).

Agent session: huge-blue

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo teamleaderleo added the risk:high Risk tier (docs/ci/risk-tiers.md) label Oct 22, 2026
@cursor

cursor Bot commented Oct 22, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo
teamleaderleo temporarily deployed to cmux-tui-merge-tree October 22, 2026 02:46 — with GitHub Actions Inactive
@teamleaderleo

teamleaderleo commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owned by acpmux-unify-3 on huge-blue · session

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review f1d18bb: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr24105 -f ref=cmux/acpmux-peer-send-mac-perms -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on de67de9 to compare; do not merge until green; see lane-rules

@blacksmith-sh

blacksmith-sh Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Found 8 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmpfjm0akce/summary-fail.json (live):/summary-fail.json (live): View Logs
/tmp/tmphw5qmd92/summary-missing.json (live):/summary-missing.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
bench OTHER_SWIFT_FLAGS is 'OTHER_SWIFT_FLAGS=$(inherited) -DCMUX_BENCH_HOOKS -DCMUX_BL
OCKING_CHECKS'/
bench OTHER_SWIFT_FLAGS is 'OTHER_SWIFT_FLAGS=$(inherited) -DCMUX_BENCH_HOOKS -DCMUX_BL
OCKING_CHECKS'
View Logs
bench OTHER_SWIFT_FLAGS is 'OTHER_SWIFT_FLAGS=$(inherited) -DCMUX_BENCH_HOOKS -DCMUX_BL
OCKING_CHECKS'/
bench OTHER_SWIFT_FLAGS is 'OTHER_SWIFT_FLAGS=$(inherited) -DCMUX_BENCH_HOOKS -DCMUX_BL
OCKING_CHECKS'
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-3zioy created for this PR by the cmux coordinator.

@github-actions github-actions Bot added risk:normal Risk tier (docs/ci/risk-tiers.md) and removed risk:high Risk tier (docs/ci/risk-tiers.md) labels Oct 22, 2026
…and reconnect when it changed under a Web-served link

Review (Codex xhigh) on 6e48a42: a relink after the handshake could lose or
replay queued RPCs and skipped the disconnect notice; a token rotated after
a non-empty read was never retried. Now the rereads happen before the
connect, and the stale-token check runs right after initialize, before any
watch or attach, and drops the link as an error (normal disconnect path).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ca9-5ac5-7692-adfc-a3ae828256ca&host=huge-blue
@teamleaderleo teamleaderleo changed the title acpmux: a peer that reconnected before the remote wrote its peer token no longer stays served as Web acpmux: an ssh peer that connected before the remote wrote its peer token no longer stays served as Web Oct 22, 2026
… queued meanwhile

Codex re-review on 0c0b6e1: requests queued while the probe ran could go
out on the Web link or replay on the next connection after their callers
got the disconnect error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ca9-5ac5-7692-adfc-a3ae828256ca&host=huge-blue
…l the stale-token check clears it

Codex re-review on affe6c9: draining the queue was no admission barrier (a
request could go out on the Web link while the probe ran, or be enqueued
after the drain), and it dropped accepted notifications such as a remote
session/cancel. Callers now get the unreachable error until the link is
admitted; the handshake's own initialize bypasses the check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ca9-5ac5-7692-adfc-a3ae828256ca&host=huge-blue
@teamleaderleo
teamleaderleo temporarily deployed to cmux-tui-merge-tree October 22, 2026 04:11 — with GitHub Actions Inactive
…queued for the next connection

Codex re-review on 7518ffd: a remote session/cancel sent while the peer was
down returned success and replayed on reconnect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ca9-5ac5-7692-adfc-a3ae828256ca&host=huge-blue
…-peer-send-mac-perms

# Conflicts:
#	cmux-tui/crates/acpmux/src/peer.rs
@cursor

cursor Bot commented Oct 26, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo
teamleaderleo temporarily deployed to cmux-tui-merge-tree October 26, 2026 01:29 — with GitHub Actions Inactive
…started remote writes its new one

stale_token() read the remote once after a Web handshake; a remote that
had removed the old token and not yet written its new one read as None,
so the link stayed Web. While the presented token is gone, reread it
(TOKEN_REREADS) before admission clears. Test: A -> missing -> B.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a175c0-3004-7a40-8069-4149d94915ab&host=huge-blue
@cursor

cursor Bot commented Oct 26, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra xhigh, correctness first): SHIP at c16e334

Diff-only adversarial review on cmux16. First pass at 5ecc7aa flagged that stale-token recovery read the remote once, so a token that vanished before the new one was written left the link served as Web. c16e334 rereads with the existing backoff and adds a test for token A, then none, then B (fails on the old code). Second pass: no blocking findings; a test-coverage note on requests during admission is not blocking.

…-peer-send-mac-perms

# Conflicts:
#	cmux-tui/crates/acpmux/src/peer.rs
…d drops frames queued for a dead link

A connection that presented no token rereads the remote (TOKEN_REREADS)
once admitted and reconnects as a peer when the token appears. Frames
still queued when a link opens were enqueued for the one that dropped
(whose callers already failed), so they are discarded, never replayed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a175c0-3004-7a40-8069-4149d94915ab&host=huge-blue
…e instead of waiting for good

request() checked connected, then registered its pending entry; if run() failed
what waited in between and the next link discarded the queued frame, a prompt
(no deadline) waited forever. The request now notes the link epoch before its
check and fails when the epoch moved by the time it is registered. A
notification on such a link is dropped as before (fire-and-forget).
The late-token rereads keep serving as Web meanwhile, as the link did before for
good; documented at the call. Regression test
a_prompt_admitted_on_a_link_that_went_down_fails_instead_of_waiting fails
without the fix (5 s timeout) and passes with it; peer_token_race 5/5,
peer_long_turn 3/3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a175c0-3004-7a40-8069-4149d94915ab&host=huge-blue
…re admission, and stop interrupts them

- Every outbound frame is tagged with the link epoch it was admitted on;
  serve drops one whose link ended when it dequeues it (replaces the
  one-shot drain at link start, which a blocked sender could slip past).
- The no-token rereads move into stale_token, before admitting clears:
  a live link is never torn down for a late token (late_token removed).
- Reread sleeps select on stop_signal and stop is checked before each
  ssh read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-c920712e011b0d6a1288
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra xhigh, correctness first): SHIP at 1629e90

Diff-only adversarial review after two base merges (#24268 peer prompt no-cap). Earlier passes found and this head fixes: a token that vanished before the new one was written, a token written after the handshake, outbound frames admitted on a dead link going out on the next one (now epoch-tagged and dropped on dequeue), stop() not interrupting rereads, and a prompt admitted on a dead link waiting forever. Each has a regression test (peer_token_race 5/5, peer_long_turn 3/3, link_epoch unit tests). Final pass: no blocking defects; pre-existing (unchanged from base): a configured peer_token is never reread.

@teamleaderleo
teamleaderleo merged commit 419b28c into feat-cmux-next Oct 26, 2026
147 of 155 checks passed
@teamleaderleo
teamleaderleo deleted the cmux/acpmux-peer-send-mac-perms branch October 26, 2026 04:41
@github-actions

Copy link
Copy Markdown
Contributor

Shipped in:

This branch had an error being deployed

2 failed and 2 inactive (1 outdated) deployments
artifacts — 1629e909 Deployed Oct 26, 2026 by teamleaderleo via publish to R2 #54418
sentry-dsyms — 1629e909 Deployed Oct 26, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #54418
cmux-tui-merge-tree — 1629e909 Deployed Oct 26, 2026 by teamleaderleo via publish the merge tree #6057
ci-routing-agent — 0dabe6fa Deployed Oct 22, 2026 by github-actions[bot] via cmux-next macOS placement #30750
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant