Repository navigation
acpmux: one serialization per streamed entry; agent host 27 -> 13 us/token (perf100) - #23051
lawrencecchen wants to merge 2 commits into
Conversation
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
|
Correctness review (independent subagent, correctness first) and resolutionThe review covered #23052 (purge guard and cleanup) and #23051 (per-token copies).
Checked and fine:
Risk tier: #23052 adds a delete of user data, which is |
|
Bead |
|
|
7bdc4d7 to
bff738e
Compare
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
…nt host and daemon) perf100. The claude-stdio path (what Lawrence's chats use): a recorded-shape 20,000-delta turn from a stand-in Claude Code, at the CLI. Budget guard about 3x the measured debug values; on base 26e79e1 (debug, rbx) the host used 33-35 us and the daemon 129-136 us per delta.
- agent host: each entry is encoded once (EntryFrame borrows it) and kept as its frame bytes; the live send and a resume send those bytes. Before, the entry was serialized to weigh it, deep-copied to send it, and serialized again by the writer. - Message::from_value moves the fields out (it copied params); new Message::into_value; the host builds records with it. The raw Claude line moves into its record (translated first; the raw record still comes first). - Message::to_line serializes from the borrowed fields (to_value copied every param for each client line). - the preview grows in place instead of being copied per chunk. rbx, release, 10 sessions x 4,000 deltas (perf100 bench), two runs each, base 26e79e1: agent host 27-28 -> 13-14 us/token, daemon 89.5-93.5 -> 75.5-85.7 us/token; debug cargo test: host 33-35 -> 23 us, host RSS after the turn 64-89 -> 36-40 MB.
bff738e to
7c1e67d
Compare
|
Rebased onto feat-cmux-next 390c286 (head 7c1e67d) with no conflicts. Note for the captain: since #23203 the hub no longer logs raw |
|
Found 2 test failures on Blacksmith runners: Failures
|
…ing every second (idle acpmux 63% -> 0%) (#26255) * cmux-tui: agent chat notices page the session list and stop reconnecting every second The notices watcher read the `_acpmux/watch` reply through a 4 MiB cap. On cosmic-gray the unpaged reply is 14.7 MB (2786 sessions), so every read was cut short, counted as a success, reset the backoff and reconnected about a second later. Three cmux-tui servers doing that held the daemon at about 54% of a core while idle. - The watcher asks for the list 500 sessions at a time (`limit`, then `_acpmux/sessions` from the cursor) and reads whole lines with no cap. - A cut line, an error reply or a list that never finished is a failure; the backoff resets only after a whole list on a connection that stayed up 60 s. - A connection that ends mid-list keeps what the watch knew of the chats no page reached. - acpmux builds the session page by moving the list in, instead of deep-copying it per reply. (The second copy, `Message::to_line` cloning the result, is what open #23051 removes.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a * cmux-tui: a push before its page wins; an unreadable line or cursor is a failure Review: acpmux turns the watch on before it reads the page, and sends pushes apart from replies, so a push can come before an older page. A chat a push reached now follows its pushes alone for the rest of the connection, and pushes count from the start of the connection. A line that is not JSON or a cursor that is not a string ends the connection as a failure, and the 60 s a connection must stay up count from the end of its list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a * cmux-tui: a push that finds an unknown chat needing nobody reads its notices Review: a chat whose first sign on a connection is a push skips its page row, so the daemon-start read the page did never ran for it. The push now does it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
Closed by the perf lead (agent, cmux-ad order 2026-10-31): stale, with no owner. This is an agent PR (lawrencecchen account). It was opened on 2026-10-21, its last commit is from 2026-10-22, no person pushed or commented in the last 3 days (automated review and sweep notes do not count), no live session names it in the lead status of the last 24 h, and it conflicts with feat-cmux-next (DIRTY). The branch |
…token (takes over #23051) (#28752) * test(acpmux): a streamed Claude turn stays within its CPU budget (agent host and daemon) perf100. The claude-stdio path (what Lawrence's chats use): a recorded-shape 20,000-delta turn from a stand-in Claude Code, at the CLI. Budget guard about 3x the measured debug values; on base 26e79e1 (debug, rbx) the host used 33-35 us and the daemon 129-136 us per delta. Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 * fix(acpmux): one serialization per streamed entry; no per-token copies - agent host: each entry is encoded once (EntryFrame borrows it) and kept as its frame bytes; the live send and a resume send those bytes. Before, the entry was serialized to weigh it, deep-copied to send it, and serialized again by the writer. - Message::from_value moves the fields out (it copied params); new Message::into_value; the host builds records with it. The raw Claude line moves into its record (translated first; the raw record still comes first). - Message::to_line serializes from the borrowed fields (to_value copied every param for each client line). - the preview grows in place instead of being copied per chunk. rbx, release, 10 sessions x 4,000 deltas (perf100 bench), two runs each, base 26e79e1: agent host 27-28 -> 13-14 us/token, daemon 89.5-93.5 -> 75.5-85.7 us/token; debug cargo test: host 33-35 -> 23 us, host RSS after the turn 64-89 -> 36-40 MB. * acpmux: the agent host's encoded-entry buffer keeps base's orphan accounting; an over-limit answer is answered Takes over #23051 onto current feat-cmux-next: - base's orphaned-host end (orphan.rs) counted retained entries a next daemon needs by looking at each Entry; the buffer now keeps encoded frames, so each retained entry records that flag when it is pushed (Retained.needed). - base's claude_stdio::logged_line (workflow snapshots logged as their size) gets an owned form, so the raw Claude line still moves into its record instead of a copy. - an entry over the frame limit that is an answer becomes an error answer to the same id, so its request is not left waiting (the open nit on #23051); anything else is still a stderr note. Red/green: cargo test -p acpmux --lib an_entry_over_the_frame_limit (new), plus the logged_line/workflow/rpc lib tests: 27 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 * acpmux: an agent host entry is never kept or sent as an empty frame Review (Codex, BLOCK at e074c3d): an answer whose id alone is over the frame limit made its error answer over the limit too, and the entry was kept and sent as zero bytes; the controller's reader then waits for a header that never comes, and the orphan count kept the host for an entry no daemon could log. encode_entry now falls back to a one-line stderr note when the replacement does not fit either, takes `needed` from what is kept, and numbers nothing when not even the note fits. encode_frame_within takes the limit so the test runs with a small one. Red/green: cargo test -p acpmux --lib an_entry_whose_replacement_is_over_the_limit_too (new; it does not build without encode_entry), and the agent_host/rpc/logged_line lib tests: 23 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 * acpmux: test that to_line writes the bytes to_value wrote Red on the borrowed to_line: it wrote jsonrpc before id, and server/list_reply.rs compares reply lines byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 * acpmux: to_line writes keys in to_value's sorted order The borrowed to_line wrote jsonrpc first and RpcError in declaration order; to_value().to_string() sorts keys (no preserve_order). Reply lines written by server/list_reply.rs are compared byte for byte with Message::ok(..).to_line(), so keep the bytes the same. cargo test -p acpmux --lib -- rpc:: list_reply agent_host::: red on the previous commit (line_tests and list_reply byte equality), 28 passed here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 * acpmux: cover params-less messages and a bare response in the line test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 * acpmux: add no panic-ratchet hits in the encode-once path The frame encoder and the retained weight added two arithmetic_side_effects hits and one cast_possible_truncation hit to acpmux; use saturating adds and u32::try_from (the body is at most MAX_FRAME). rust_panic_ratchet counts for acpmux are now base's (1494, 222). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-2f016c499153c1bead46 --------- Co-authored-by: lawrencecchen <54008264+lawrencecchen@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: cmux-bot[bot] <348531757+cmux-bot[bot]@users.noreply.github.com>
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
What
Per-token copies removed on the claude-stdio path (what Lawrence's chats use):
Message::from_valuemoves the fields out instead of copyingparams. NewMessage::into_value; the host builds records with it. The raw Claude line moves into its record. It is now translated first, and the raw record still comes first.Message::to_lineserializes from the borrowed fields. Before,to_valuecopied every param for each client line.Numbers (rbx, perf100 bench: 10 sessions x 4,000 deltas, one attached client; base 26e79e1; two runs each)
cargo test)Most of the daemon's per-token cost is still in the hub: two records per token, the log line, the event ring's record copy, and the fan-out. That is the next slice; it needs a profiler on rbx (
perfis blocked for the rbx user).Test
tests/claude_stream_budget.rs: a recorded-shape turn of 20,000 deltas from a stand-in Claude Code (tests/fake_claude_stream.py), run through the CLI. It checks that the reply arrives whole. It also checks that the agent host uses at most 70 µs and the daemon at most 330 µs of CPU per delta, about 3x the measured debug values. It is a budget guard: on base the debug numbers (host 33–35 µs, daemon 129–136 µs) also pass.Review
A correctness review subagent checked this change:
write_frame(&HostFrame::Entry), and the controller decodes them.from_valuegives the same result for every input.write_allandflushper frame.Nits that remain open:
Changelog
none
Observed / Not run
cargo clippy -p acpmux --all-targets -D warnings, where the only errors are the base redscmp_owned(cmux-tui: two base clippy reds (acpmux steer_now_tests, worktree-care store) #22906) andunnecessary_sort_byin cmux-worktree-care. The full suite runs in the landing gate.from_valueandinto_value.Need help on this PR? Tag
@codesmith-botwith what you need.Summary by cubic
Removes per-token copies on the claude-stdio path. The agent host encodes each streamed entry once and keeps its frame bytes, sending the same bytes live and on resume;
Message::from_valueand the newMessage::into_valuemove fields instead of copyingparams;Message::to_lineserializes borrowed fields; and the streaming preview grows in place. Agent host CPU per token drops from 27–28 to 13–14 µs (release, perf100 bench) and RSS after a 20,000-delta turn from 64–89 to 36–40 MB; daemon CPU per token drops from ~90 to ~78 µs.Budget test
tests/claude_stream_budget.rsruns a recorded-shape 20,000-delta turn through a stand-in Claude Code (tests/fake_claude_stream.py) and asserts the reply arrives whole and that host/daemon CPU per delta stay under 3x the measured debug values.Written for commit 7c1e67d. Summary will update on new commits.