Skip to content

feat: add React acpmux agent pane - #16042

Merged
teamleaderleo merged 160 commits into
feat-acpmux-chat-panefrom
feat/acpmux-ts-pane
Sep 30, 2026
Merged

teamleaderleo merged 160 commits into
feat-acpmux-chat-panefrom
feat/acpmux-ts-pane

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

The agent session pane now has a React TypeScript renderer that connects directly to the acpmux daemon over its authenticated loopback WebSocket. Native remains the default while the TypeScript pane is validated against the same event stream and seeded transcript path.

What changed

  • Reduced the Swift bridge to a versioned host handshake. Swift starts or finds acpmux and returns the loopback WebSocket endpoint, a daemon bearer token for the web-view launch, and the selected session id. It continues to host WKWebView, customization files, and native-only actions.
  • Added a direct React acpmux client. It initializes, watches sessions, attaches with _acpmux/attach {eventStream: true}, pages older events with beforeSeq, folds mux and ACP events into rows, and sends ACP and _acpmux/* actions directly.
  • Reconciled permission envelopes, optimistic prompt echoes, and stream row state across turns, with focused direct-client tests.
  • Kept the React renderer requested by Leo. Rows are memoized by row id and content version, with Pretext-based named-font measurement, cached markdown blocks, typed-array geometry, binary-search virtualization, latest-message opening, and scroll anchoring.
  • Kept registry-based React row renderers and composer chips. User components can provide a deterministic measure(row, width) function; components without one use anchored post-mount measurement. theme.css, layout.json, and registry.js hot reload from ~/.config/cmux/agent-pane/.
  • Kept the standalone Vite preview harness with Codex and Claude-shaped event recordings, real-speed streaming replay, the 5,000-row transcript, permission and queue fixtures, fixture picker, theme and width controls, and frame-time readout.
  • Native remains the default renderer.

The listener is loopback and already authenticates WebSocket handshakes with the daemon bearer token. WKWebView cannot set an Authorization header, so the host passes the token in the query string accepted by acpmux. The token is only handed to the local web view; Swift does not persist or relay transcript data through the bridge.

The React pivot follows the projections rules in cmuxterm-hq#1051, with the correction that acpmux owns sessions, Swift hosts, and React and native are both clients. This branch includes the merge from origin/main at 2e8363e637c, which Lawrence can pull into #15521 when his branch catches up.

Preview

cd webviews
bun run preview:dev
bun run preview:build

The static output is webviews/dist/acpmux-agent-session-preview/ and uses relative assets for file or static-host use.

Validation

  • bun x tsgo --noEmit
  • bun test src/agent-session/acpmux/model.test.ts src/agent-session/acpmux/direct.test.ts (8 passed)
  • focused oxlint on the direct client and preview
  • bun run preview:build
  • ./scripts/build-agent-session-web.sh
  • python3 scripts/verify-local.py --swift-changed --only swift-syntax --only project --only config-schema --only localization --only test-wiring
  • Swift package tests need macOS CI because this Linux host cannot import the package's existing Darwin dependency.

Performance

Lawrence's native debug.agent_chat.action seed_rows fling measured p50 and p95 at 8.33 ms on a 120 Hz display. The TypeScript pane exposes the same seed and fling stats actions; comparable macOS/fleet numbers are still pending.

Changelog

Changed: React agent pane now connects directly to acpmux and folds its authenticated event stream in TypeScript; Swift hosts the web view and native remains the default.

Latest repair at bd3f0416cc9 preserves live events received during attach, marks rejected optimistic sends failed, and reconciles prompt echoes from older daemons without promptId.

Lifecycle repair at 353b636ac25 adds reconnect and first-session creation, resets per-session sequence state, removes superseded retry rows, and persists React session selection through the host.

The final repair persists a newly created session before its first prompt is sent, including when that prompt is still streaming or rejects.

The final repair drops stale concurrent session selections before they can overwrite the persisted current session.

The final attach guard ignores stale responses from overlapping session selections before they can change the active pane.

The final repair recovers cleanly when acpmux purges the selected session, including while reconnecting after a socket drop.

The final purge repair clears queued and streaming state before attaching a replacement session.

lawrencecchen and others added 30 commits September 27, 2026 21:43
One policy reducer decides the PTY grid for local and Cloud terminals.
The conformance corpus is replayed by the Swift engine and, next, the
cmux-tui Rust twin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TerminalSizingParticipantColor hashes user_id (else participant id) with
FNV-1a 64 into a fixed 10-color palette so Mac and iOS agree.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
TerminalSharingStore is the one Mac-side owner of per-terminal size state
and actions. LocalTerminalSizingHost runs the shared engine for local
terminals with the Mac pane and each phone as participants.
CloudTerminalSizingRelay tracks the Mac and its phones as cmux-tui
participants. Wires CmuxTerminalSizing and CmuxTerminalSharing into the
app and test targets.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Points at manaflow-ai/bonsplit#259 (feat/terminal-size-presence). Land
that PR before merging so the pinned SHA is on bonsplit main.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Local terminals: TerminalController runs LocalTerminalSizingHost per
surface (the Mac pane as mac:<surface>, each phone as mobile:<client_id>)
instead of the smallest-viewport rule, and applies the decided grid
through the existing viewport governor. Mobile RPC gains size_state in
replay, mobile.terminal.size_state and mobile.terminal.detached pushes,
mobile.terminal.reattach, size_policy.set and participant.disconnect. A
disconnected phone's viewport, input and replay are refused until it
reattaches.

Cloud terminals: with shared-sizing-v1 the mirror sends its identity,
forwards each phone as a relay sub-view, relays size-state and routes
detached events. A disconnected-by detach of the Mac stops automatic
reconnection. Daemons without the capability keep the legacy claim path.

Socket: terminal.size_state, size_policy.set, size_to_me,
size_counts.set, participant.disconnect, participants.disconnect_others
route through TerminalSharingStore. Not allowlisted for the remote relay.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tab presence accessory and size context menu (bonsplit), pane bounds
overlay with owner border, hatch, chip, crop pill, HUD and a detached
card, and the SwiftUI size panel. Adds the Size Terminal to My Window
shortcut (ctrl+opt+cmd+=) and command palette actions, all routed
through TerminalSharingStore.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ment

Adds the phone side of docs/shared-terminal-sizing.md: decoders for the
mobile.terminal.size_state and mobile.terminal.detached pushes and the
size_state/self_participant_id replay fields, a per-surface sizing reducer
(generation ordering, network vs disconnected-by detach, reattach), the
presentation facts for the bounds UI, the viewport payload builder with
device_kind/device_name/counts_override, a stand-in owner color matching
the shared FNV-1a rule, and the pure bounds geometry for the letterbox.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Subscribes to the size_state and detached pushes, records size_state from
replay answers, and gates viewport reports, input, paste, mouse, scroll and
replay while another participant has detached this phone from a terminal.
A network detach recovers through the existing replay path. Adds
reattach, size policy, participant disconnect and counts override actions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Draws the owner-color border, hatch and cut-edge fade inside the surface's
letterbox, adds the corner chip, +N cols pill, reconnecting capsule, the
size sheet (mode, participants, counts, disconnect) and the detached card
with Reattach and Reattach as viewer. Strings are localized in the app
catalog for all nine locales.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
All nine app locales. Mac, iPhone and iPad are recorded as brand
literals and the person-device separator as a format literal.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
cmux surface size, size-policy, size-to-me, size-counts, participants,
disconnect-participant and disconnect-others call the terminal.* socket
methods. Session tests cover disconnected-by (no auto-reconnect), network
detach (reconnects) and a phone detach that keeps the mirror attached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Port TerminalSizingEngine to cmux-tui-core/src/sizing_policy.rs with the
same JSON wire shape and replay schemas/terminal-sizing/fixtures.json in
its tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every client view and relay sub-view of a terminal is a participant of
the runtime's sizing engine. Attach, claims and send/send-key input are
activity; an owner that leaves hands the grid to the next owner instead
of freezing it. set-client-sizing maps onto counts overrides.

Add set-size-policy (terminal override or workspace default),
set-size-counts, get-size-state, relay sub-views on resize-attached-view
(view + identity), identity fields on set-client-info, participant ids
and by on detach-client, and reason/by/view on detached. size-state
events and the new attach response fields go only to clients that send
shared-sizing-v1, which identify now advertises. Update the spec,
inventory, SDK schema and generated bindings.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cmux-tui host now takes explicit activity through note-size-activity
(with view for a phone behind this Mac). set-client-sizing is no longer
used as activity because enabled:false means counts false. Activity is
sent only when it would move ownership.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Add note-size-activity {surface, view?}, gated on shared-sizing-v1.
Without view it marks the caller's own view; with view it marks that
relay sub-view, so a Mac mirror forwarding phone input makes the phone
the latest active participant instead of the Mac.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A single Mac with a fixed or smaller grid had no border, hatch or chip,
because sizing chrome appeared only when another viewer was attached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The terminal shows a 1 pt owner-color border at 70% opacity, a faint hatch,
a short cut-edge fade and one caption2 chip at the grid's bottom-trailing
corner ("118×38 · Maya's Mac Studio · 12 cols hidden"), only while this
phone's grid differs. The "+N cols" pill and the top-leading chip are gone.
The chip is a UIKit button owned by the surface so it follows the letterbox
rect; its tap opens the size sheet.

The size sheet is a grouped list: a "118 × 38" header with the owner under
it, one Size menu (Fixed adds a columns × rows field row), participant rows
with an avatar, "Name · Device" and "Sets size" on owners, swipe and context
menu Disconnect, a counts toggle in this phone's context menu, EditButton
reorder in Priority mode, and one confirmed "Disconnect Others" button.

The detached card is "Detached", one line, Reattach and Reattach as viewer.

Model: showsChip now follows viewportDiffers only; adds isOwner(_:) and
ownerLabel, which drops the person name when the device name already
contains it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The size panel is now 280pt, opaque and padded (16/12pt): the grid and
its owner on one line, a Size mode menu (with a cols × rows pair in
Fixed), participant rows with a hover menu (Counts toward size,
Disconnect) and drag handles only in Priority, and Disconnect Others with
an inline confirm. The size map, help text, scope select, switches,
eject icons and Size to My Window button are gone from the panel.

TerminalSizePanelPresenter owns the one popover and anchors it to the
tab accessory, else the tab item, for every entrypoint. The accessory
click toggles it; the mouse-down that closes a transient popover is
recorded so its mouse-up does not reopen it.

The pane overlay keeps a 1pt owner-color border that animates to a new
grid, a fainter hatch, a 16pt fade on a cut edge, and one AXButton chip
(118×38 · Maya's Mac [· 12 cols hidden]) that opens the panel. The HUD,
flash and +N cols pill are removed. The detached card is shorter.

Label rules move to TerminalSharingPresentation in CmuxTerminalSharing
with injected strings and tests. Unused strings are removed; new ones
carry all nine macOS locales. Bonsplit moves to the fork branch head
with the avatar-only accessory and popoverAnchorView(for:).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@blacksmith-sh

This comment has been minimized.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

CI failure attribution

CI stopped on 353b636ac2 (run 36757370934 attempt 1): 1 code, 3 unknown.

Job Verdict Why
suite-coverage unknown no known signature; failed step: Require the suite that judges this diff
web / agent-session-web-resources unknown no known signature; failed step: Build and test Agent Session web resources
guards / workflow-guard-tests / preflight code a Python test failed
guards / workflow-guard-tests / release-ios unknown no known signature; failed step: Validate iOS package conventions for this change
Matched log lines
guards / workflow-guard-tests / preflight: FAIL: test_every_advertised_path_is_supported (__main__.ConfigurationReviewPathsTests.test_every_advertised_path_is_supported)

Not re-run automatically: suite-coverage, web / agent-session-web-resources, guards / workflow-guard-tests / preflight, guards / workflow-guard-tests / release-ios are not machine failures.

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Dogfood tours of 353b636a

sidebar-and-chrome-tour at 353b636a: not run

skipped: CI left no app build for this head (its compile failed or was cancelled)

Tours are picked by the paths globs in dogfood/scenarios/*.json; a Dogfood-tours: a, b line in the description picks them instead (none turns this off). Look at every frame before merging: a green tour only means no step failed.

Move transcript projection and ACP actions into the React client. Keep Swift as the WKWebView host and pass an authenticated loopback endpoint through the versioned handshake.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review at 3ba6b383b1f: the direction change was checked against the acpmux listener and schema. The listener serves WebSocket text frames on the dashboard port and accepts its bearer token in the query string, which lets WKWebView connect without an Authorization header.

Fixed: Swift no longer projects transcript rows or handles chat mutations for the web pane; the host bridge returns only the authenticated endpoint, launch token, and selected session. React now initializes, watches, attaches with eventStream, folds mux and ACP events, pages with beforeSeq, and sends ACP actions directly. Preview recordings use the same event-stream shape.

Left: macOS Swift package tests, CI, and a live 5,000-row macOS fling measurement remain pending. Native remains the default.

teamleaderleo and others added 3 commits September 30, 2026 09:30
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Handle permission envelopes, settle optimistic prompts, and reset streaming rows between turns. Install the webview lockfile before bundling Pretext resources in CI.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Merge in-flight events with attach pages, reconcile legacy prompt echoes, and mark rejected sends failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@blacksmith-sh

This comment has been minimized.

teamleaderleo and others added 7 commits September 30, 2026 10:46
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review at 353b636: PASS. The fresh review exercised purge reset, replacement attach, reconnect after purge, overlapping attach guards, superseded streams, and packaged asset parity; 8 focused tests passed.
Fixed: direct client lifecycle races, first-session creation and persistence, stale attach isolation, superseded retry rows, purge recovery, and the generated resource bundle.
Left: Swift package tests and comparable macOS fling numbers remain CI or fleet validation.

@teamleaderleo
teamleaderleo marked this pull request as ready for review September 30, 2026 18:25
@teamleaderleo
teamleaderleo merged commit 86e251b into feat-acpmux-chat-pane Sep 30, 2026
77 of 115 checks passed
@teamleaderleo
teamleaderleo deleted the feat/acpmux-ts-pane branch September 30, 2026 18:40
lawrencecchen added a commit that referenced this pull request Oct 1, 2026
… renderer switch

PR #16042 merged code that uses bonsplit APIs added after the branch's pin
b32f48b (TabContextAction.sizeMode*, BonsplitContrastPalette, TabPresence,
BonsplitController.popoverAnchorView), so the app failed with "type
'TabContextAction' has no member 'sizeToMyWindow'" and related errors. Pin
vendor/bonsplit to 351bfa7, the commit cmux main pins (on bonsplit main;
b32f48b is its ancestor). AgentSessionPanelView applied .id and .frame to an
if/else, which failed with "instance member 'id' cannot be used on type
'View'"; the renderer switch is now inside a Group. Native stays the default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
teamleaderleo added a commit that referenced this pull request Oct 1, 2026
Ports #16042's customization hot reload. AgentPaneCustomizationWatcher
watches theme.css, layout.json and registry.js next to cmux.json (so
CMUX_NEXT_CONFIG_FILE moves them) with ConfigFileWatcher, reads them off
the main actor, drops unchanged results, and sets the value on every
agent pane; new panes get the current value. It runs while any agent tab
has a view.

AgentPaneView pushes a change at once, and replays it after a load and
when the page asks for the handshake (its bridge and registry exist by
then). registry.js is evaluated as its own host script, since the bundled
CSP has no unsafe-eval, so a broken registry no longer stops the theme;
then applyCustomization({themeCSS, layout}) runs. A missing file is
skipped instead of failing the push, a deleted theme.css sends "" and
clears the style, and layout.json must be a JSON object. The page hands
layout to the registry's configure. The bundled page is rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
teamleaderleo added a commit that referenced this pull request Oct 1, 2026
Each replay of registry.js now runs it inside a fresh function, as
#16042's indirect eval did, so top-level const, let and class
declarations don't collide with the previous run. The README says the
file is replayed and should be idempotent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD
teamleaderleo added a commit that referenced this pull request Oct 1, 2026
…cmux/agent-pane hot reload (#16433)

* cmux-next: failing tests for the agent pane dev server and customization

The pane's page now comes from an AgentPaneSource (the bundled file, or a
loopback Vite dev server), and AgentPaneCustomization carries the
~/.config/cmux/agent-pane/ files to the page. Both are stubs here: the
dev-server override is ignored, its origin is never trusted, and the
customization reads and pushes nothing, so the new suites fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

* cmux-next: load the agent pane from a Vite dev server in Debug builds

A Debug or tagged build launched with
CMUX_NEXT_AGENT_PANE_DEV_URL=http://127.0.0.1:<port>/ loads the pane from
that loopback dev server (bun run dev:agent-pane, port 4176) instead of
the bundled page, so TypeScript edits hot-reload in the running app. The
handshake still comes from Swift, so the dev page talks to the real
acpmux daemon.

AgentPaneSource.resolve takes allowsDevServer, which the App sets only
under #if DEBUG: Release loads the bundled file whatever the environment
says. The override must be http on 127.0.0.1 or localhost with an
explicit port and no credentials; anything else keeps the bundled page.
Navigation and the handshake trust only that exact origin. reload.sh
forwards the variable into the tagged launch environment, and the pane
README documents the loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

* cmux-next: hot-reload ~/.config/cmux/agent-pane/ into the agent pane

Ports #16042's customization hot reload. AgentPaneCustomizationWatcher
watches theme.css, layout.json and registry.js next to cmux.json (so
CMUX_NEXT_CONFIG_FILE moves them) with ConfigFileWatcher, reads them off
the main actor, drops unchanged results, and sets the value on every
agent pane; new panes get the current value. It runs while any agent tab
has a view.

AgentPaneView pushes a change at once, and replays it after a load and
when the page asks for the handshake (its bridge and registry exist by
then). registry.js is evaluated as its own host script, since the bundled
CSP has no unsafe-eval, so a broken registry no longer stops the theme;
then applyCustomization({themeCSS, layout}) runs. A missing file is
skipped instead of failing the push, a deleted theme.css sends "" and
clears the style, and layout.json must be a JSON object. The page hands
layout to the registry's configure. The bundled page is rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

* cmux-next: failing test for replaying a registry.js with top-level declarations

The pane re-evaluates registry.js on every load, handshake and file
change. As a global classic script, a second run of a file with a
top-level const, let or class throws a SyntaxError (duplicate variable)
before registering anything. Checked in the system jsc: the raw script
registers once and then throws.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

* cmux-next: run registry.js in its own function scope

Each replay of registry.js now runs it inside a fresh function, as
#16042's indirect eval did, so top-level const, let and class
declarations don't collide with the previous run. The README says the
file is replayed and should be idempotent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RYQHfug1ZVQDp4eWgwVUtD

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants