[teamleaderleo]

Summary

A linked Mac's acpmux daemon sat at 77-105% of a core with 1.8 GB RSS for 16 h, and acpmux ls there took 6.8 s or timed out. It has 43 peers listing 8,044 rows, plus 450 sessions of its own. Three 10 s samples of it, taken a minute apart, look the same: Hub::locate holds ~4,000 of the 4,441 samples per thread-window, about 0.9 of a core. Under it sit remote_sessions() (a copy of every peer row), the sort of the full list, and serde_json indexing and malloc/memmove churn.

locate answers _acpmux/resolve, and every lifecycle CLI command (acpmux send, last, wait, …) resolves its session name there (cli/resolve.rs control_id). Each resolve built the full list: every peer row copied and all 8.5k sorted. A query the live list missed built it again, and a miss built it up to four times. On a host whose agents and reconcilers run those commands all day, that is the core.

1. locate copies only the peer rows its query could name. Names::may_name is a superset of what by_bead, by_link, by_key_exact and by_key can match: id, prefix, name, short name, bead tag, link session id, and every row of a HOST/… key's host. It is judged on the row's peer and receipt plus two summary fields, so a row costs a few compares, and only the rows it takes are copied and sorted. The full order is a total sort, so restricting it keeps the same relative order. Moved chats' targets are read from the same kind of list (live or all) the matches came from. Close-match names come from a name-only pass, and session_name::candidates is order-independent. A miss reads the closed rows once instead of twice.
2. A peer's notices read only that peer's rows. Every streamed record of a peer's session (session/update, _acpmux/event) found its row with a walk over every peer's rows on the one serial notice loop. So did each ask check, session_changed, purge, _acpmux/lagged, cache save, stale mark, warm round and peers() count. RemoteRows now indexes each peer's rows by key and by the peer's own session id, kept on every write as its (mux.parent, mux.handle) index already is.

Checked and ruled out, from the samples and the code:

• 16 threads blocked in flock are by design: agent_host/wait.rs death watches, one blocking thread per live agent host (deduped by path), waiting on the host's lock. They use no CPU.
• Reconnects to unreachable peers already back off. Each step doubles from 0.5 s to a 60 s cap with equal jitter (peer_link::Backoff), and a peer down for a day parks for an hour (acpmux: preserve SSH failure reasons and slow retries for long-down peers #27002). Dials don't show in the samples.
• Full lists from big peers are read whole only on a handshake. After that a link asks for a delta (watch_delta.rs), and rows arrive one session_changed at a time (fix 2).
• ls latency is the shared list builder's pace (list_guard.rs, ~1/32 of a core): it answers from cached rows and dials no peer. The resolves above held the rows lock and the CPU it waits on.

Performance

metric	before	after	delta	harness or run link
Hub::locate share of daemon samples	~4,000 / 4,441 per thread-window (~0.9 core)	to be measured after relaunch	—	sample PID 10, ×3, build 735d16b
five resolves among 8,000 peer rows (exact, bare, HOST/ prefix, short name, miss)	710.8 ms	69.6 ms	−90%	hub::peer_notice_cost_tests::a_resolve_…, debug build, same CI job (base = fix reverted)
one peer notice among 20,000 peer rows	702.2 µs	30.6 µs	−96%	hub::peer_notice_cost_tests::a_peer_notice_…, debug build, same CI job
daemon CPU (43 peers, 8,044 peer rows)	77-105%	to be measured after relaunch	—	Activity Monitor / ps on the linked Mac
RSS	1.8 GB (peak 2.8 GB)	to be measured after relaunch	—	sample header
peer dials per minute	backoff unchanged (≤ 1/min per down peer, 1/h after a day)	unchanged	0	peer_link::Backoff

A Mac's daemon updates only when its app relaunches, so the fleet "after" figures come from the first relaunch after this lands.

Testing

Red first: e28a2a9 (notice cost) and 26c8c7b (resolve cost) each add a failing test before their fix. Each CI run ran the tests at the head, then reverted the fix in the same job and ran them again:

• cargo test -p acpmux --lib -- hub::peer_notice_cost_tests
  • head: pass (notice 30.6 µs among 20k vs 31.3 µs alone; five resolves 69.6 ms vs one full list 94.5 ms)
  • fix reverted: fail (notice 702.2 µs vs 34.3 µs alone; five resolves 710.8 ms vs one full list 91.2 ms)
• cargo test -p acpmux --lib -- hub::locate hub::own_host hub::peers hub::remote_rows hub::peer_notice_cost_tests hub::peer_snapshot hub::peer_cache hub::permissions hub::purge_tombstones hub::peer_grace hub::key_route cli::resolve session_name: 150 passed.
• cargo test -p acpmux --test pairing -- resolve: passed.
• New: hub::remote_rows::tests (the peer index follows insert, replace, take, retain, mark_peer_stale and take_peer, leaving no empty entry), and a_resolve_still_finds_what_it_named_among_peers (exact, bare across peers, HOST/ exact before prefix, session id, miss).

Observed / Not run

Observed: three samples of the affected daemon and its --json status (taken for this PR); the CI runs above.
Not run: the full acpmux suite and clippy (the PR's CI runs them). The after-relaunch CPU, RSS and ls latency on the linked Mac are pending until it relaunches on a build with this.

Changelog

Fixed: the acpmux daemon no longer spends a core resolving session names when it is linked to peers that list thousands of sessions.

Risk: normal. It adds a read-side filter and an index on a map whose writes already go through one owner type. It changes no wire format, policy or persisted data.

Agent session: agent session

🤖 Generated with Claude Code

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review ca665e6: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr32487 -f ref=lane/acpmux-peer-cpu -f platform=macos -f backend_mode=local

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4150891005901 (2fdcbbae7e)
• cmux NEXT nightly 1.0.0-nightly.4151067571301 (546cb0bf70)
• cmux NEXT nightly 1.0.0-nightly.4151196080201 (8d6a5624a6)
• cmux NEXT nightly 1.0.0-nightly.4151284660301 (22fd89a839)
• cmux NEXT nightly 1.0.0-nightly.4151418814301 (b899f6b4f0)

[blacksmith-sh]

Found 2 test failures on Blacksmith runners:

Failures

Test	View Logs
test_every_compile_job_names_its_warm_sources (main.ThisTree.test_every_compile_job _names_its_warm_sources)/ test_every_compile_job_names_its_warm_sources (main.ThisTree.test_every_compile_job _names_its_warm_sources)	View Logs
the short nightly decide gate must use the pooled Linux route with a Blacksmith fallbac k/ the short nightly decide gate must use the pooled Linux route with a Blacksmith fallbac k	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[teamleaderleo]

Review (codex gpt-6.1-sol high fast): SHIP at 5a0559e

Two rounds: SHIP at 022c697 (the per-peer RemoteRows index: consistent across every write path, Deref read-only, swapped lookups keep peer/mux semantics, sound locking, reasonable cost-test threshold). Then SHIP on the delta 022c697..5a0559e: may_name covers every earlier peer match path, and order, moved-target resolution and dedupe, close matches and error behaviour are unchanged.
Visual verdict: unchanged by this diff (no UI).
Follow-ups: none.

[teamleaderleo]

Review (codex gpt-6-astra high, sweep, via the review router; full diff from merge base 80d716d to this head): SHIP at 5a0559e

No P1/P2 findings.

[teamleaderleo]

Review (codex gpt-6.1-sol high fast): SHIP at ca665e6

The delta since the SHIP at 5a0559e only fixes CI lints: dead code (an unused list builder removed, RemoteRows::retain made test-only) and three panic-ratchet Value index reads made .get()/.pointer(). Behaviour is unchanged. CI at this head: clippy -D warnings clean, 150 targeted tests passed, and no new ratchet hits against this PR reverted (indexing equal, arithmetic −1). The remaining reds (nightly-guards Guard status, release-notary) fail the same way on base.
Visual verdict: unchanged by this diff (no UI).
Follow-ups: none.