Skip to content

cmux daemon: bridge a link share to its hosted room, 3/3: the guest relay (cmux.share/1) - #26849

Merged
teamleaderleo merged 11 commits into
feat-cmux-nextfrom
lane/mp-bridge-relay
Oct 31, 2026
Merged

teamleaderleo merged 11 commits into
feat-cmux-nextfrom
lane/mp-bridge-relay

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

This PR's own diff is its last commit, 87588f32559 (+696/-13). It sits on #26827 (the host socket), which sits on #26825 (provisioning).

Leo (2026-10-28), on Theo's T3 Code "Share with guest": "we want to be having, be able to do this kind of flow for CMux multiplayer ... we can work on the agent chat multiplayer." A guest opens a link in any browser, joins with a name, watches live, and each message waits for the owner.

Tracks manaflow-ai/cmuxterm-hq#4766. Spec: chat-multiplayer.md 4.3, which covers the frame table, recipients, ordering and sizes. The room is manaflow-ai/cmux-platform#6 and #8.

What it does

The bridge starts a relay for each guest whose browser is in the room, from the room's hello or guest_joined. The relay opens a same-user connection to this daemon, a socket pair served by the real connection path (handle_connection_with_permit). Its first line binds it with agent-session-proxy-bind {link_guest}, then it attaches. Every frame is therefore the bound guest's, and the daemon's per-frame checks apply:

  • the share and the guest must still be live;
  • only the guest verbs are allowed;
  • every prompt is held for the owner;
  • a revoke, kick or expiry closes the connection at the daemon first.

What the relay sends the room names only that guest:

From To the room
attach reply snapshot {to, chat {title, harness, status}, records, watermark, has_more, prompts}. It holds the newest records that fit in 512 KiB, has_more when older ones were left out, and the guest's held prompts (at most 16) plus its 16 most recently decided.
each live record records {to: [guest], records}
older {before_seq} agent-session-events → page {to, records, has_more}, also at most 512 KiB
guest_prompt {client_id, text} agent-session-prompt with the client_id (the daemon's deterministic g- prompt id), then prompt_state
acpmux records prompt_state: queued with awaitingOwner → held; queue_approved → approved; queue_declined → declined; queue_removed / prompt_retracted → removed; a refused prompt → refused with its error code
prompt_status {client_ids} one prompt_state for each prompt the relay knows
guest_typing {on} agent-session-typing
  • Size limits: a record over 256 KiB is sent cut, as its seq, dir and kind with truncated: true, full_bytes. A 17th held prompt is refused with share.too_many_held.
  • When a relay ends: a guest_left for the guest's last socket ends it, and so do a kick, a revoke or expiry (the bound connection closes, so the room never hears more of that guest). The kick and end frames to the room are cmux daemon: bridge a link share to its hosted room, 2/2: the host socket (cmux.share/1) #26827's.
  • Reconnects: a guest reconnect (guest_joined again) starts a new relay with a fresh snapshot. A host reconnect does the same for every guest the hello names, after the daemon's guests.
  • Owner presence: the relay sends agent-session-view {visible: true} after its attach, so the owner's presence shows the guest viewing.

Known limits

  • Presence: presence {to} is not relayed. The daemon does not give a bound guest acpmux's presence set; it says who else has the chat open (stream.rs: "Not to a teammate yet"). Relaying it needs that policy changed for guests, with what a guest may see of the others decided first. That belongs in its own PR.
  • prompt_status coverage: a prompt whose last state record is older than the relay's snapshot and later pages gets no prompt_state from prompt_status. The relay knows states only from records it has seen; it does not query acpmux's prompt ledger.
  • Prompt states need acpmux's held prompts: the states come from acpmux's held-prompt records (awaitingOwner, queue_approved, queue_declined), which arrive with the acpmux held-prompts PR. Until then a guest prompt is reported held once acpmux accepts it.
  • Orphan room after a restart (from cmux daemon: bridge a link share to its hosted room, 1/2: provisioning (cmux.share/1) #26825): if the daemon restarts after a create that timed out, that share is revoked locally. A room the PUT made without answering lives until the share's expiry, because the owner token its DELETE would need is never kept.

Risk

risk:normal: daemon-only. It is inert until a share is provisioned with an owner token (the share sheet / op PR), and only the room's development environment is live. The relay reaches the chat only through a bound guest connection, so it adds no new path to the agent or the machine.

Tests

These use the fake room from #26825/#26827 and the existing fake acpmux.

  • Bounded snapshot and Ann's own records and prompts: the fixture's attach page is 200 + 200 + 300 + 200 KiB.
    • Ann's snapshot is at most 512 KiB, keeps seqs 11–13 with has_more, and sends seq 12 cut (truncated, full_bytes, no msg). The watermark is 13.
    • The record acpmux sent during the attach follows the snapshot, to Ann only.
    • Ann's guest_prompt reaches acpmux as hers, with the g- id and approval owner, and she gets prompt_state held. After acpmux's queued {awaitingOwner} and queue_approved records, she gets prompt_state approved (held is not said twice).
    • prompt_status answers for the known prompt only, and older gets a page.
    • Bob, whose browser never joined, gets nothing.
  • Reconnects and kick:
    • Ann rejoining gets a fresh snapshot.
    • After a host reconnect, guests comes first, then a snapshot for each connected guest.
    • Kicking Ann ends her relay: her later prompt never reaches acpmux. Bob's still does and gets his prompt_state.
  • The provisioning and host-socket tests from cmux daemon: bridge a link share to its hosted room, 1/2: provisioning (cmux.share/1) #26825 and cmux daemon: bridge a link share to its hosted room, 2/2: the host socket (cmux.share/1) #26827.

CI receipts are in a comment below.

Changelog

none

Agent session: agent session

Room contract (cmux-platform src/server/ChatShareRoom.ts)

  • Host hello names each guest with an open socket once; with manaflow-ai/cmux-platform#147 it also gives sockets (their count). Each guest socket then reports guest_joined / guest_left. Without a count the daemon ends a guest's relay on its first guest_left (no connection outlives the guest; another open tab reconnects for a fresh snapshot).
  • The room refuses a guest prompt itself past 2/s and 32 KiB, and when no host is ready; the daemon refuses past 16 held or in flight (share.too_many_held) and past 64 asks kept before the relay's attachment answers (share.busy).
  • Each relay connection holds one of the daemon's connection permits (MAX_SERVER_CONNECTIONS).

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Bridges a link share to its hosted room (cmux.share/1, cmux-platform's ChatShareRoom): create provisions the share in the room, each active share keeps one host socket, and each guest whose browser is in the room gets a relay that connects back to this daemon as that guest and feeds the room that guest's chat. The daemon stays the authority — every frame passes its per-frame checks, and revoke, kick, expiry, or a closed chat closes the share at the daemon first.

  • Create provisions the share with the owner's cmux access token (never stored) before answering; the room is CMUX_SHARE_ROOM_URL, else the build's (none for a standalone daemon), and ships inert outside development (a PUT answers 404, share.room_refused; a 401 answers share.needs_sign_in). An unanswered provision returns share.room_unreachable, is retried in the background, then revoked with its room DELETE queued; a provision that answers after the share's expiry never opens the share (it is retired in its place).
  • The queued room DELETE is dropped only once the room acknowledges it — 2xx, or 404/410 as gone; any other answer, 401 and 403 included, is retried with backoff.
  • Each active share keeps one outbound host socket: after every hello it answers guests {active} first, redeem is answered by the registry's own atomic redeem, and kicks are persisted until the room answers kicked. A dropped socket reconnects with backoff; 4000 replaced, 4001 ended, 4003 and a refused upgrade stop it. Closing the agent chat closes its link shares too (end {reason: closed}).
  • Each guest's relay sends a snapshot first: the newest records that fit in 512 KiB (has_more when older ones remain), the watermark, chat title and status, and the guest's held plus most recently decided prompt states; then live records, older pages, prompt_state from the reply and acpmux's records, prompt_status answers, and guest_typing. Room asks before the attach answers are held until it does (past 64 refused, share.busy); daemon records before the snapshot wait for it (past 256 the relay restarts fresh).
  • A relay counts its guest's sockets from hello (sockets), so one of two leaving does not end it; without a count the first guest_left ends it. The 16-held cap counts undecided prompt ids in a per-guest ledger kept across host reconnects (forgotten on a kick or the share's end), so a burst cannot pass it and prompt_status cannot evict it; retrying a decided prompt takes no slot, history pages apply the decisions they name (prompts decided while the guest was away release their slots), and the guest's client ids carry to the next relay so a reconnect's snapshot names those prompts' states. A prompt's state moves forward only, so a refused retry never undoes a decision.
  • Records over 256 KiB go cut as seq, dir, kind with truncated and full_bytes; a 17th held prompt is refused with share.too_many_held. A relay keeps at most 32 calls and 2 history pages in flight; past that older gets an empty page that has more. A relay ends on the guest's last guest_left, a kick, revoke, expiry, or a closed attachment, and while the guest stays connected it restarts after a backoff (250 ms doubling to 30 s) with a fresh snapshot; a guest or host reconnect starts a fresh relay too, and a replaced relay's late frames are dropped. After attach the relay sends agent-session-view {visible: true} so the owner sees the guest viewing.
  • A guest relay's daemon connection claims one of the daemon's connection permits like any client; with none free the guest waits and is tried again on the backoff. An unanswered prompt stays in the ledger and every new relay of the guest sends it again once attached (acpmux runs a prompt id once), so one in flight across a host reconnect is neither lost nor said twice.

Known limits

  • presence {to} is not relayed (the daemon does not give a bound guest the presence set); relaying it needs a policy decision on what a guest may see of others.
  • prompt_status covers only prompts whose state records the relay has seen, not acpmux's prompt ledger.
  • Prompt states come from acpmux's held-prompt records, which land with a separate acpmux change; until then a guest prompt is reported held once accepted.

Risk

  • Daemon-only and inert until a share is provisioned with an owner token; the relay reaches the chat only through a bound guest connection, so it adds no new path to the agent or machine.

Written for commit f81fd0a. Summary will update on new commits.

View guided diff Turn on auto-fix

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 28, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

cmux-ci receipt at head 87588f325593f353ee4c9e7179564c07aa2c2022 (Blacksmith, 32 vCPU), run 40677468138, exit 0:

  • cargo test -p cmux-tui-core --lib proxy::shares::: 19 passed, 0 failed. That is the 11 existing share tests plus 8 room tests, among them a_guest_gets_a_bounded_snapshot_its_own_records_and_its_prompts_states and reconnects_get_fresh_snapshots_and_a_kick_ends_the_guests_relay.
  • cargo clippy -p cmux-tui-core -p cmux-tui --all-targets -- -D warnings: clean.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-p8e2sn created for this PR by the cmux coordinator.

@blacksmith-sh

blacksmith-sh Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
test_a_picked_signing_label_has_a_rerun_alternative (main.RunsOnFallbackTests.test_
a_picked_signing_label_has_a_rerun_alternative)/
test_a_picked_signing_label_has_a_rerun_alternative (main.RunsOnFallbackTests.test_
a_picked_signing_label_has_a_rerun_alternative)
View Logs
test_no_runs_on_chain_ends_on_the_reserved_side_label (main.RunsOnFallbackTests.tes
t_no_runs_on_chain_ends_on_the_reserved_side_label)/
test_no_runs_on_chain_ends_on_the_reserved_side_label (main.RunsOnFallbackTests.tes
t_no_runs_on_chain_ends_on_the_reserved_side_label)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo
teamleaderleo force-pushed the lane/mp-bridge-relay branch 2 times, most recently from 43ea811 to 7453a62 Compare October 28, 2026 11:34
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review f81fd0a: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr26849 -f ref=lane/mp-bridge-relay -f platform=macos -f backend_mode=local

web and size checks: green

Shipped in:

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Rebased on #26827's new head, now at f5b6c6386c9e8c693e3659bf0b90af8a883a5d2e. The PR's own diff is unchanged: +696/-13.

cmux-ci run 40682369154, exit 0: proxy::shares:: 23 passed, 0 failed; agent_chat_notices 12 passed; cargo clippy -p cmux-tui-core -p cmux-tui --all-targets -- -D warnings clean.

@teamleaderleo

teamleaderleo commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · took over from an earlier session · session

For each guest whose browser is in the room (the room's hello or
guest_joined), the bridge opens a same-user connection to this daemon,
served by the real connection path, binds it with
agent-session-proxy-bind {link_guest} on its first line and attaches, so
every frame is the bound guest's and passes the per-frame checks.

To the room, naming that guest only: a snapshot first (the newest
records within 512 KiB, has_more, watermark, chat, the guest's prompt
states), then each live record; older -> page; guest_prompt ->
agent-session-prompt with its client_id, and prompt_state from the reply
and from acpmux's records (queued awaiting the owner -> held,
queue_approved, queue_declined, queue_removed); prompt_status -> one
prompt_state per prompt the relay knows; guest_typing -> typing. A
record over 256 KiB goes cut (truncated, full_bytes); 16 held prompts at
most (share.too_many_held).

guest_left (its last socket), a kick, a revoke or expiry ends the relay;
a guest or host reconnect starts it again with a fresh snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-c3300437b0af7f1bba36
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high fast, review-router; started by the backlog lander): BLOCK at d63a8f1

BLOCK

  1. P2 — agent_session_share_relay.rs:109-115,190-197: Room requests are processed while bind/attach is still pending. An early guest_prompt or older reaches the daemon before an attachment exists, gets agent_session.not_attached, and is lost. Queue guest requests until attach succeeds.

  2. P2 — agent_session_share_room.rs:875-877,917-931; agent_session_share_relay.rs:164-165: Replacing a relay on guest reconnect does not cancel the old task. Its untagged Out::Frame messages can still be written after the new relay’s snapshot, causing duplicate or out-of-order records and prompt states. Frames need generation filtering or task cancellation.

  3. P2 — agent_session_share_room.rs:918-924,933-938,967-978: hello.guests contains one entry per guest, while guest_left is emitted per browser socket. After a host reconnect with multiple sockets for one guest, the relay is initialized with sockets = 1; the first socket leaving removes it, so the remaining browser stops receiving records and cannot send requests. Preserve the actual socket count across host reconnects.

… frames, and outlives an uncounted socket

- The room's asks that come before the relay's attachment answers wait
  for it (up to 64), instead of being refused as not attached and lost.
- Each relay frame carries its guest and generation; the host socket
  drops frames of a relay a reconnect replaced, so no stale record or
  prompt state follows the fresh snapshot.
- A host hello names each guest once, not its sockets: such a relay no
  longer ends on the first guest_left, but with the guest's kick, the
  share's end or the next hello.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-710a4a518c009a6ad126
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high fast, review-router; started by the backlog lander): BLOCK at 81ad50d

BLOCK

  1. P2 — Resource exhaustion bypasses the connection cap — agent_session_share_relay.rs:81 starts each guest connection with handle_connection_with_permit(..., None). This skips the daemon’s MAX_SERVER_CONNECTIONS permit entirely. The configured limits allow up to 16 shares × 8 guests, so public guests can make roughly 128 internal connections, each spawning server and attachment threads, exhausting process/thread resources and starving normal clients.

  2. P2 — Early guest requests are silently dropped — agent_session_share_relay.rs:124-127 discards every request after the 64-entry pre-attach queue fills. In particular, guest_prompt frames receive no refusal or prompt_state, so messages sent while attachment is slow can be permanently lost while the browser believes they were submitted.

…t it cannot keep

- Each guest relay's daemon connection claims one of the daemon's
  connection permits like any client; with none free no relay starts,
  and the guest's next join asks again.
- An ask past the 64 kept before the attachment answers is no longer
  dropped silently: a prompt gets prompt_state refused (share.busy),
  older gets an empty page that still has more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-710a4a518c009a6ad126
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6.1-sol high fast, review-router; started by the backlog lander): BLOCK at 02dbe44

BLOCK

  1. P2 — pre-attach prompt states can precede the required snapshot. In agent_session_share_relay.rs:295-300, early prompts call take; when the held cap is full, take immediately calls note at line 496, which emits prompt_state before attachment. The snapshot is emitted later at lines 427-447. A reconnect with 16 held prompts and another early prompt therefore violates the snapshot-first protocol, and the client may discard the refusal when it initializes from the later snapshot.

  2. P2 — early records have no byte bound. agent_session_share_relay.rs:347-348 stores full records, and lines 398-407 limits only the number of records to 256. The 256 KiB truncation is applied only after attachment at lines 591-595. Large transcript records during a delayed attach can therefore retain hundreds of full multi-megabyte JSON values per guest and exhaust daemon memory.

  3. P2 — ledger eviction loses active held-prompt state. Ledger::remember evicts the oldest client mapping without protecting prompts in open; take invokes it for every new prompt, and prompt_status does so for arbitrary IDs at lines 550-560. After enough decided prompts while 16 prompts remain held, an active prompt can lose its client mapping. Its later decision updates the ledger but frame() returns None, so the guest never receives that state and the prompt disappears from reconnect snapshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6.1-sol high fast, review-router; started by the backlog lander): BLOCK at 19c5585

BLOCK

  1. P2 — Prompt silently lost before attachment. agent_session_share_relay.rs:297-299 calls link.take for pre-attach prompts but ignores its failure. When the 16-prompt cap is full, the prompt is neither forwarded nor answered with prompt_state: refused; it disappears permanently.

  2. P2 — Unbounded relay output queue can exhaust memory. agent_session_share_room.rs:824 creates an unbounded channel, and agent_session_share_relay.rs:386-388 enqueues every record without backpressure. A slow or stalled room socket lets relay tasks accumulate unlimited frames while the host writer processes one at a time; the 512 KiB limit only bounds individual frames.

  3. P2 — prompt_status can evict unanswered prompt mappings. agent_session_share_relay.rs:161-169, 551-558 remembers every requested client ID and evicts entries by state timestamp. Unanswered prompts have no state timestamp, so several requests containing unique unknown IDs can evict their mappings. When acceptance arrives, frame() then has no client ID and emits no state; reconnect snapshots omit the prompt even though it may have run.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Awareness note from an agent lane that does not own this PR; it is not a review or a request to take the PR over.


This PR's branch refuses pushes from the lane carrying it now, so its commits plus the review-fix commit dd57b9a0c83 (early records capped by bytes, non-v1 room frames ignored, counted prompts' client ids kept) continue in #28928, with authorship kept. Please close this one once #28928 lands.

1 similar comment
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Awareness note from an agent lane that does not own this PR; it is not a review or a request to take the PR over.


This PR's branch refuses pushes from the lane carrying it now, so its commits plus the review-fix commit dd57b9a0c83 (early records capped by bytes, non-v1 room frames ignored, counted prompts' client ids kept) continue in #28928, with authorship kept. Please close this one once #28928 lands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6.1-sol high fast, review-router; started by the backlog lander): BLOCK at f81fd0a

BLOCK

  1. P2 — Reconcile outstanding prompts after reconnect. agent_session_share_relay.rs:575: If a guest disconnects with 16 held prompts, the owner decides them, and over 200 newer records arrive, the reconnect snapshot misses those decisions. The ledger still counts all 16 as open and rejects new prompts with share.too_many_held. Neither reconnect nor prompt_status fixes this; the guest must manually page back to the decisions. Automatically reconcile outstanding prompt IDs before enforcing the cap.

  2. P2 — Bound the outbound relay queue. agent_session_share_room.rs:824: Every guest relay sends live records into an unbounded channel, while the host drains it through serial WebSocket writes. If records arrive faster than the room connection uploads them, queued frames grow indefinitely and can exhaust daemon memory. The per-write timeout does not prevent this when individual writes keep completing. Apply a byte budget and restart overflowing relays with fresh snapshots.

This branch was previously deployed

3 inactive deployments
artifacts — f81fd0a3 Deployed Oct 31, 2026 by teamleaderleo via publish tree-addressed cmux-tui #87952
sentry-dsyms — f81fd0a3 Deployed Oct 31, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #87952
ci-routing-agent — f81fd0a3 Deployed Oct 31, 2026 by teamleaderleo via cmux-tui tree preflight #87952
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant