Repository navigation
acpmux: ACP session/list goes in pages when the whole list is too large to send - #28161
Conversation
…message limit Red on base: 5250 mirrored rows with tools are one 17 MB reply, refused as message.too_large, as one daemon's 5241 were on 2026-10-30. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
…ge to send session/list answered every session, local and mirrored, with tools, in one reply: one daemon's 5241 rows were 17 MB on 2026-10-30, over the 15 MiB message limit, so it was refused as message.too_large. A list over 14 MiB now goes in pages of at most that, each with the nextCursor ACP's client follows (the session list's keyset cursor, so a row that moves between pages neither repeats nor skips another), and cursor is read. A list that fits is one reply as before, with no nextCursor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
|
Found 4 test failures on Blacksmith runners: Failures
|
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: |
…versized row goes as a stub Review of 669f622: a peer's page was sized before masking, which can lengthen text; a row larger than a page alone was still sent, page and all over the limit; and two rows with one id and updatedAt share a cursor, so a one-row page of the second came forever. A masked page over the budget is cut again at half the budget (down to 64 KiB), a row larger than the page goes as a stub with its size (as an oversized record does, hub/paging.rs), and a cursor the page cannot get past is refused (list.cursor_stuck) instead of answered again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
…CP row's stub keeps no large or masked text byte_page ends a page before rows a cursor cannot tell apart (one id, one updatedAt) or, when they are all it holds, after all of them, and a cursor starts after every such row: each comes once, so list.cursor_stuck is gone. The stub for a row larger than its page keeps only its id and cwd, each at most 4 KiB, and no title, so it stays small whatever field was large and carries no text a peer's reply masks. Tests: byte_pages_read_rows_one_cursor_names_once (four budgets, the pair on and across a page edge) and an_acp_row_larger_than_its_page_goes_as_a_stub (a huge title and cwd); cargo test -p acpmux --lib passes them locally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
…ne/acpmux-sessions-pages Takes #28161's page cutting (stub rows, cursor groups, re-cut of masked pages) under this branch's LIST_PAGE_BYTES name, and moves the re-cut into within_limit, which the _acpmux/sessions pages now use too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
Rows with one id and one updatedAt are told apart by their order: a page that ends on the n-th of them names it <cursor>.<n>, and the next page starts after it. A page no longer has to hold the whole group, which could pass the budget (two 7.5 MiB rows on one page). Test: byte_pages_read_rows_one_cursor_names_once now checks that no page over its budget holds more than one row (red on the group code at budget 5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
A chat here and its copies on peers share an id and an updatedAt. The cursor told them apart only by their order, so a page ending on the second of three skipped the third once the first left. The cursor key now hashes the peer too, and ties sort by peer, so each copy has its own cursor. This daemon's own rows keep the key they had. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c11d09c9f41988fc1d93
|
Review (the owner lane's independent review, at its land request): SHIP at 02632c2 Recorded by lane-gh-broker from this PR's owner lane, which asked to land this exact head after its independent review. next-autoland lands it once every red is the base's; a BLOCK at this head, a HOLD line or a hold label still stops it. |
…-acp-list-pages # Conflicts: # cmux-tui/crates/acpmux/src/server/mod.rs # cmux-tui/crates/acpmux/src/server/requests.rs
|
Red classification at 8c35a0a (merge-base 7deba73). This PR changes only
Review: Sol (gpt-6.1-sol, high) SHIP at 02632c2 (no findings), and SHIP on the base merge at this head (resolutions keep base's |
|
Review (the owner lane's independent review, at its land request): SHIP at 8c35a0a Recorded by lane-gh-broker from this PR's owner lane, which asked to land this exact head after its independent review. next-autoland lands it once every red is the base's; a BLOCK at this head, a HOLD line or a hold label still stops it. |
|
Review (codex gpt-6-astra high, next-review): SHIP at 8c35a0a No blocking findings. |
…hits from #28161) #28161 (4543835) landed while this PR was in review: - hub/session_pages.rs: byte_page's sort key reads each key with get. Every index comes from enumerate, so the order is unchanged. - server/list_reply.rs: sessions and updatedAt are read with get (a missing value is Null, as before), and nextCursor is written via JsonSet::set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-d9e7e9f7fb2b87a57d5f
… add without overflow (panic ratchet) The panic ratchet's tracked arithmetic_side_effects count for acpmux grew past its baseline 1430: three adds in hub/session_pages.rs (4543835, #28161) and two in daemon/unused_exit.rs (55df640, #28353). They saturate now; no value they hold can reach usize::MAX or u64::MAX, so behavior is the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
FYI: hub/session_pages.rs added 3 |
|
Shipped in:
|
…sor tells a chat's peer copies apart) Tests of a cursor a chat and its peer copy share read a daemon from before #28161 (a fake whose cursor names only id and time), as the fleet's daemons are today. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-4388606a06c8d5c111f8
…ing_slice 97 > 94): rewrite the new hits without panics (#28386) * acpmux: inbox, send and outbox status JSON writes without indexing Part of the acpmux panic ratchet base red that came back after #28134 (indexing_slicing 703 > 693, string_slice 97 > 94). These writes go through JsonSet::set instead of IndexMut. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-d9e7e9f7fb2b87a57d5f * acpmux: ls globs, dotted fields, inbox threads and match counts without indexing (10 hits from #28012, #27686, #27791, #27970) - cli/select.rs (911f25f, #28012): field() cuts the dotted path with str::get; glob() compares and tails with slice get. Every index was in bounds, so the output is unchanged. - hub/inbox.rs (cbe8e60, #27686): first_pr reads the number after the marker with str::get. inbox_unsend matches on (index, noticed, notice is none), with the same three outcomes in the same order. - hub/mod.rs (fe3152a, #27791): local_match_counts bumps its slot with get_mut. - hub/queue.rs (46ed428, #27970): the transfer id is read with get. Tests add multibyte names, keys and PR markers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-d9e7e9f7fb2b87a57d5f * acpmux: ACP session/list paging without indexing (5 indexing_slicing hits from #28161) #28161 (4543835) landed while this PR was in review: - hub/session_pages.rs: byte_page's sort key reads each key with get. Every index comes from enumerate, so the order is unchanged. - server/list_reply.rs: sessions and updatedAt are read with get (a missing value is Null, as before), and nextCursor is written via JsonSet::set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-d9e7e9f7fb2b87a57d5f * panic ratchet: lower acpmux to the measured counts (indexing_slicing 731 -> 686, string_slice 97 -> 94, unwrap_used 144 -> 143) #28169 raised the acpmux baseline to the counts base had then, so the hits fixed here could come back unnoticed. Measured at fdb6218 (fast-checks run 41080915558). PR CI now runs the ratchet too (#28169), so a PR that adds a hit is caught before it lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-d9e7e9f7fb2b87a57d5f * acpmux: read ls row tags with get (the hit #28369 added to the panic ratchet) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-d9e7e9f7fb2b87a57d5f --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…back under its baseline (#28418) * fix(acpmux): daemon_memory's start_logged is Linux-only like its one caller #27991 (941b7a4) moved Daemon::start onto start_inner, leaving start_logged used only by the Linux-only retained_session_list_allocation_benchmark, so clippy -D warnings on macOS failed with 'function start_logged is never used'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(acpmux): the session page budget and the unused-exit client count add without overflow (panic ratchet) The panic ratchet's tracked arithmetic_side_effects count for acpmux grew past its baseline 1430: three adds in hub/session_pages.rs (4543835, #28161) and two in daemon/unused_exit.rs (55df640, #28353). They saturate now; no value they hold can reach usize::MAX or u64::MAX, so behavior is the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
ACP's
session/listanswered every session, local and mirrored, with tools, in one reply, and took no parameters. On a daemon that mirrors a fleet that reply passes the 15 MiB message limit: one daemon's 5241 rows were about 17 MB with tools on 2026-10-30 (the same list that broke peer handshakes, #28105), sosession/listthere came back asmessage.too_large, with no way to read less.session/listnow reads ACP's owncursorand answers withnextCursor(both in ACP'ssession/listschema). A list that fits in 14 MiB (a megabyte under the limit, for the frame and masking) is one reply exactly as before, newest first, cached per list build, with nonextCursor. A larger one goes in pages of at most 14 MiB, cut by bytes (hub/session_pages.rsbyte_page, at least one row per page).sessionIdandpeer, not newest first, and the cursor is that hash (h+ 32 hex, 33 bytes whatever the ids). A session that updates or leaves while a client pages moves no other row, so none is skipped or repeated; a row listed twice (one id, one peer) comes once; a page never ends between rows of one hash.FullRowsinhub/list_guard.rs), so it parses and serializes only its own rows. A list whose rows alone are over the budget does not build the whole reply only to drop it._meta.acpmux.omitted {bytes, reason: "too_large"}, keeping only its id andcwd(each at most 4 KiB) and no title, the way a record too large for a history page goes. Rows keep their tools._acpmux/sessionscursors now hash the peer with the id, and ties sort by peer (local first) in both sorts (sort_session_summaries,Rows::finish), so a chat here and its copy on a peer each have their own cursor. Cursors of this daemon's own rows are unchanged; an old cursor for a peer's row falls back to the stamp and may repeat, never skip.Performance
Synthetic fleet: 25 peers × 210 mirrored chats = 5250 rows, each with a 3.3 KB tool description (the incident's mean row size with tools).
session/listreplies to read the whole listmessage.too_large)server/acp_list_tests.rsa_fleet_sized_acp_list_goes_in_pages_under_the_message_limit(red, green)PERF acp_list rows=5250 bytes=20559390 pages=2)session/listof an oversized list)acp_list_tests::a_fleet_sized_acp_list_goes_in_pages_under_the_message_limit, release, 3 runs each, the same runner and test file; before = this PR at 19689e4 (whole list parsed per page), after = 02632c2Test
cargo test -p acpmux --lib acp_list_tests:a_fleet_sized_acp_list_goes_in_pages_under_the_message_limit: the 5250-row state above; followingnextCursorreads every row once, with tools, and no reply line is overMAX_MESSAGE_BYTES.an_acp_list_that_fits_is_one_reply_with_no_cursor: 40 rows, one reply, nonextCursor.byte_pages_read_each_row_once(hub/session_pages.rs): a chat here, its copies on two peers and a row listed twice, at five page budgets: every row comes once (red without the dedupe).byte_pages_hold_their_place_while_sessions_update_and_leave: one row a page; between pages an unread row becomes the newest, then a read and an unread row leave: every remaining row comes once.a_byte_page_cursor_stays_small_for_a_huge_id: rows with 1 MiB ids; the cursor is 33 bytes (red with the ids in the cursor).copies_of_a_chat_on_peers_page_by_their_own_cursorsand the extendedstreamed_rows_preserve_all_fields_and_stable_ties(hub/list_guard.rs): the_acpmux/sessionscursor and both sorts tell a chat's peer copies apart (each red without its change).an_acp_row_larger_than_its_page_goes_as_a_stub(server/list_reply.rs): a row over its budget, and one with a 1 MBcwdand title, go as stubs under 400 B with no title.Red on the test commit 909561f (run 41026359386):
page 0: message too large to send: 20559292 bytes, the limit is 15728640Green on the head 02632c2 (run 41060184119, tested tree b7e0a5a9 = the head's tree): the tests above pass;
cargo clippy -p acpmux --all-targets -- -D warningsis clean; incargo test -p acpmuxeverything passes except three:hermetic_daemon_guard(every_test_binary_runs_in_the_hermetic_environment), which fails the same way on base before acpmux: a bare session name or id prefix reaches the one live session on any host; several are refused with each HOST/NAME command #28000 and at the tip 7f97b84; and two that passed 5 of 5 when rerun alone at the same tree (run 41062160697), in code this PR does not touch:daemon::state_file_tests::managed_unit_start_avoids_a_symlinked_default_homeandheld_queue::agent_messages_joined_to_a_send_whose_agent_cannot_start_are_not_held(load flakes). The job's exit 1 is forced so the log prints.Review
Independent Codex (gpt-6.1-sol, high), merge-base diff: SHIP at 02632c2, no findings, after four BLOCKs fixed in this PR: rows that share a cursor straddling a page edge; a chat's peer copies sharing a cursor; newest-first pages skipping a row that updates between pages; an unbounded cursor and whole-list work per page.
Observed / Not run
Observed: the red and green runs above, on hosted Linux through
cmux-ci test, and the release timings. Read, not run: ACP clients in the wild; an ACP client that ignoresnextCursornow reads the first 14 MiB of a list it could not read at all before.What else grows with the fleet (audit of the other list replies)
_acpmux/sessions/_acpmux/watchwithoutlimitanswer whole: with tools that is the same 17 MB (acpmux ls --tools, a non-linkwatch {withTools: true}); without tools about 2.3 KB per row, so about 6,900 rows at the incident's mix. A filtered_acpmux/sessionswithoutlimit(status: closed,excludeStatus) also answers whole. Paging these needs their clients to follownextCursor; a follow-up._acpmux/waitwith nosessionswaits on every running row and answers full rows twice; about 3,000 running sessions fleet-wide to reach the limit._acpmux/peers,_acpmux/peer_status(problems capped),_acpmux/models(about 1 MB at 26 daemons),_acpmux/chats(MAX_LIMIT5000 × ~700 B), mailbox, team inbox, history pages (MAX_PAGE_BYTES), peer snapshots, workflows, forum, bg tasks, composers. No notification carries a fleet-wide list.Changelog
session/liston a machine that mirrors many peers' chats answers in pages instead of failing as too large.Agent session: agent session
🤖 Generated with Claude Code