[teamleaderleo]

Leo, 2026-11-04: "i feel like the restarted after nightly update does too many layout shifts/flicker renderings and takes too long to reconnect acpmux or whatever". This is the reconnect half (data: manaflow-ai/cmuxterm-hq#6338); restore layout shifts are a separate PR.

What this adds

A relaunch-reconnect scenario in the shared perf harness, beside idle-chats: bench_power.py --scenarios relaunch-reconnect (scripts/cmux-next/bench_relaunch_reconnect.py), run on the fleet through the existing entry:

cmux-ci run --class exclusive --measurement --display --label measure \
  --script scripts/measure/perf-idle-power.sh --ref SHA --arg=scenarios=relaunch-reconnect \
  --arg=config=bench --artifact artifacts/perf-idle-power.json

It opens 6 local agent chats on the app's own acpmux (the test agent), one streaming 10 words a second and shown. Then it quits (Quit and Keep Sessions) and relaunches 3 times in each mode:

• warm: the daemon keeps running.
• update: the bundle is replaced in place while the app is gone, as an update does.
• cold: the daemon is stopped.

Each round records every phase, in ms since the relaunched process started, from process start to the shown chat's first live event.

Marks added for it:

• App launch marks (debug.timings), for the acpmux lookup (start, build homes adopted, status answered, bundled build read, handoff, spawn start/ready, end) and the pane's transport open (start, listener checked, accepted checked, open). A lookup of a daemon the app does not start is marked acpmux.other.*. AgentPaneLaunchTimings now takes marks from any thread, since the lookup runs on the host actor, the way DaemonLaunchTimings already does.
• process_start_epoch_ms in debug.timings, so page marks line up with the launch marks.
• The page's own wall-clock connect path (perf.ts launchTimeline, read by debug.agent_pane perf_stats as launch): first handshake start, handshake answered, connect start, connected, first event of the shown chat, and first live event. A live event is one that arrives within 1 s of when acpmux recorded it, so a replayed backlog does not count.

The daemon log since the quit is kept per round.

What it found

These are medians, ms since the relaunched process started. Same harness, Mac16,11 display worker, at e8e06a6 (3 rounds per mode):

phase	warm	update	cold
app main	1563	4387	1546
did finish launching	1953	4823	1954
acpmux lookup start	2005	4909	2023
daemon status answered	2007	5232	2418
daemon ready (log)	2173	4961	2077
lookup end	2175	4965	2079
pane view created	1916	4784	1915
page handshake start	2920	5793	2911
handshake answered	3087	5959	3066
transport open	3083	5956	3078
page connected	3145	6026	3134
pane painted	3329	6164	3245
first live event	3238	126164 (1 of 3 rounds; 2 never within 120 s)	never within 120 s (0 of 3)

1. The slowest phase by far is on the daemon side. When the daemon is gone before the app relaunches, the adopted streaming turn shows nothing new for about 120 s, or not at all within the bench's 120 s. Its log shows the old daemon stopping on its own when the bundle is replaced (SIGTERM (a restart handoff)) or for the cold mode, and the next one starting about 6 s later. 120 s is acpmux's adoption replay ceiling (hub/hosts.rs REPLAY_CEILING). After that, the next quit's daemon finds no turn to keep it ("nothing used the daemon for 2s: no client, agent, turn or task"). The fix is a separate PR with a red/green test.
2. Warm relaunch: about 1.7 s from process start to the first live event after app main.
   • The app's lookup is quick (status in 2 ms).
   • The longest app-side phase is pane view created to page handshake start: about 1.0 s of page load before the page asks for anything.
   • The handshake itself takes about 170 ms.
   • Connect is about 60 ms.
3. A warm relaunch restarts the daemon. The relaunched app asks the running daemon (same build) to stop ("stopping: a client asked it to") and starts a new one. This is the person-key enroll's handoff (AcpmuxPersonKey.enroll). It costs about 100 ms here, but every adopted agent goes through adoption again. This is a follow-up.
4. Update mode spends about 2.8 s more before main. This is the first launch of a new copy of the bundle, which macOS checks. It is outside the app.

Performance

This PR changes no behavior. Its numbers are the baseline above. Its own cost is a handful of timestamps, each recorded once per launch or once per page, plus one Date.now() compare per event of the shown chat until its first live event.

metric	before	after	delta	harness
relaunch phases measured	0	21 per round	+21	relaunch-reconnect, Mac16,11

Red/green

• bun test test/agent-pane-launch-timeline.test.ts (webviews): at base it fails with TypeError: perf.markLaunch is not a function. Both tests pass at head.
• Bench script: ran end to end at a2e1444 and e8e06a6 on fleet display workers (above).

Risk: low. The Swift adds launch marks (signpost events plus a first-time-per-name mark) on the acpmux lookup and transport open paths. AgentPaneLaunchTimings becomes a Sendable class with a lock, the same shape as DaemonLaunchTimings. The page marks are stored once per page. bench_memory.launch can relaunch into an earlier launch's scratch; its default path is unchanged.

Observed / Not run

• Ran:
  • The bench twice on fleet display workers.
  • CmuxNext compile check at e788a25 and e8e06a6 (passed).
  • The new bun test.
  • scripts/web-lint.sh: its only failures are check:design-tokens (subagents.css) and vp fmt --check (cloudRun/cloudStartProgress), all in files this PR does not touch, red on base 660552b.
  • prepush-gate.py (clean).
• Not run: a NIGHTLY with real agents and an actual Sparkle update. The bench stands in with a bundle swap.

Changelog

• none

Agent session: agent session

🤖 Generated with Claude Code

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review e8e06a6: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr34697 -f ref=lane/acpmux-relaunch-reconnect -f platform=macos -f backend_mode=local

Stale: these images are of e8e06a6; the head is now 7124dfb and a new capture is running.

Gallery before/after for e8e06a6: 16 states rendered, 0 images differ, 3411 more touched states not rendered. Images are in the run's artifact, not on this public PR; the independent review of this head reads them on its own, so the PR needs no hand-made screenshots of these states.

gh run download 41773747935 -R manaflow-ai/cmux -n gallery-advisory-34697-e8e06a63034256682542e024e0751138104015a3 (then read gallery-review.json)

States

State	Theme	Status	Diff	Picked by
agent-pane.chat-width/standard	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.chat-width/standard	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.chat-width/wide	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.chat-width/wide	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.chat-width/wide-blocks	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.chat-width/wide-blocks	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-edge/short-edge	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-edge/short-edge	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-edge/wide-edge	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-edge/wide-edge	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-edge/no-edge	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-edge/no-edge	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-floats/scrolled-to-end	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-floats/scrolled-to-end	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-floats/composer-grown	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-floats/composer-grown	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-rest-gap/gap-40	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-rest-gap/gap-40	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-rest-gap/gap-56	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-rest-gap/gap-56	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.composer-rest-gap/gap-72	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.composer-rest-gap/gap-72	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/shipped	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/shipped	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/two-thirds	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/two-thirds	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/half-tucked	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/half-tucked	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/before	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.continuation-spacing/before	Apple System Colors	unchanged	0%	own App.tsx
agent-pane.copy-controls/output	Apple System Colors Light	unchanged	0%	own App.tsx
agent-pane.copy-controls/output	Apple System Colors	unchanged	0%	own App.tsx

_{Pick other states with a Gallery: entry.id, entry.id/variant line in the PR body.}

Passes: CI passes on 7124dfbb19.

CI passes on 7124dfbb19 (run 41782788057 attempt 1).

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's; yours means the failing file is one this PR changes, also red on main that main's latest full suite fails the same way, seen on other PRs that it failed on another pull request's run lately.

red: cmux-next checks (god files, concurrency, crash safety, l10n); base red: the same failure on the base 2e0b6ac; does not block
red: cmux-next crash ratchet; NEW: green on the base 2e0b6ac; do not merge until green; see lane-rules

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4182499138501 (e0aa712b3b)
• cmux NEXT nightly 1.0.0-nightly.4184087390301 (08052d331e)
• cmux NEXT nightly 1.0.0-nightly.4184224384601 (a3abfc4160)
• cmux NEXT nightly 1.0.0-nightly.4184409870901 (78ee0e108e)

[teamleaderleo]

Review (codex gpt-6-astra high, next-review, since=20261104T032722Z): SHIP at e8e06a6

SHIP at e8e06a6
Visual verdict: unchanged by this diff

1. [P2] [follow-up] [category:correctness] [finding:F1] [outcome:pending] Reestablish a live stream before each measured quit. scripts/cmux-next/bench_relaunch_reconnect.py:244 reuses one streaming turn across every round and mode without checking that it remains active. If an update round loses that turn—as the PR’s daemon logs suggest—subsequent cold rounds measure an idle chat that cannot produce another event, falsely attributing the timeout to cold reconnect. Verify fresh streaming activity before each quit; restore the load outside the measured interval or mark dependent rounds invalid. A mocked run confirmed the loop continues after a timeout without restarting the stream.

Python syntax checks passed. Runtime tests were unavailable here because the harness requires macOS libraries and Bun is absent.

Follow-ups:

• Reestablish and verify the streaming precondition for each round.

[teamleaderleo]

Review (subagent opus-5.5 medium): SHIP at e8e06a6

Visual verdict: unchanged by this diff (no UI).

Checked: Swift 6 isolation of the now thread-safe AgentPaneLaunchTimings (same shape as DaemonLaunchTimings), the direct.ts hot path (one property check per event once the first live event has been seen), the default path of bench_memory.launch (unchanged), the bench's relaunch socket and cleanup, and the perf.ts live-event logic.

Follow-ups:

• Cold mode: SIGTERM to the daemon ends the streaming turn. Either restart the stream after each cold round, or document that cold rounds may never go live.
• Rename launch_to_process_ms: it measures the time from open to process start.
• The docstring says the bench captures the daemon's "stopping" line, but the >= started filter drops the shutdown at quit time. Make the docstring match.

[teamleaderleo]

Review (codex, router-review): SHIP at e8e06a6

Reviewer: codex gpt-6.1-sol xhigh, fast tier.

Follow-ups: 2. Each lands as an issue in this repo that links this receipt; the lane that opened the PR fixes it forward.

SHIP at e8e06a6
Visual verdict: unchanged by this diff

1. [P2] [follow-up] [category:tooling] [finding:F1] [outcome:pending] bench_relaunch_reconnect.py:244: Restore the streaming precondition before each round. The stream starts only once; if an update round loses its turn, subsequent update and cold rounds still run against that dead turn. A mocked adoption failure reproduced later rounds reporting timeouts without any active stream. Recreate or verify and re-arm the load before each quit, then regenerate the baseline.

2. [P2] [follow-up] [category:correctness] [finding:F2] [outcome:pending] perf.ts:237: Daemon recording time cannot distinguish adoption replay from live output. Buffered host entries pass through session_tap → append_logged, which assigns a fresh now_ms() timestamp. Old output arriving during adoption therefore satisfies this check while replay continues. Use source-event freshness or an adoption replay boundary before recording successful live recovery.

Follow-ups:

• Restore the streaming load between benchmark rounds and rerun the baseline.
• Distinguish adoption replay from live output.

[cmux-bot]

cmux-bot approves e8e06a6: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

[teamleaderleo]

Review (codex gpt-6-astra high, router-review range): SHIP at e8e06a6

Visual verdict: no images (range review of a measurement-only change).

Follow-ups:

• [P2] [follow-up] F1: scripts/cmux-next/bench_relaunch_reconnect.py:247-259 filters daemon log timestamps to started, recorded only after the quit or handoff, so the old daemon's stopping/shutdown events are dropped from every cold/update round. Fix: keep events between quit_at and the process start, still tied to the round (issue to follow).

[blacksmith-sh]

Found 6 test failures on Blacksmith runners:

Failures

Test	View Logs
/tmp/tmpd1px566_/summary-missing.json (live):/summary-missing.json (live):	View Logs
/tmp/tmpmhj9etct/summary-fail.json (live):/summary-fail.json (live):	View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/ [new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows	View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames	View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/ [tabs] debug.tab_switch timed 0 of 40 switches	View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

A newer verdict blocks 7124dfb (no independent SHIP at the head).

[teamleaderleo]

Review (codex, router-review): SHIP at 7124dfb

Reviewer: codex gpt-6-astra high, fast tier.

Follow-ups: 1. Each lands as an issue in this repo that links this receipt; the lane that opened the PR fixes it forward.

SHIP at 7124dfb
Visual verdict: unchanged by this diff

1. [P2] [follow-up] [category:tooling] [finding:F1] [outcome:pending] Validate the streaming workload before every round. bench_relaunch_reconnect.py:244 starts streaming only once and continues after reconnect timeouts. If an update loses the turn—as the PR reports—later rounds, including cold mode, measure an idle chat and report misleading reconnect failures. Require fresh streaming progress before each quit; restore the workload between rounds or mark subsequent rounds invalid. A mocked run confirmed four relaunches proceed after stream loss without restarting the workload.

Follow-ups:

• F1: Prevent a lost streaming turn from contaminating subsequent rounds and modes.

[lawrencecchen]

The crash ratchet is red on the feat-cmux-next tip from this PR: swift CmuxNextAgentPane: unchecked 2 -> 3 (AgentPaneLaunchTimings: @unchecked Sendable). #35106 removes @unchecked: the type holds only a Mutex and an OSSignposter, which are both Sendable. It does the same for DaemonLaunchTimings, the source of the copied pattern. mark() does not change.