Route preview automation through live owner streams

[juliusmarminge]

Summary

Replaces preview automation's replicated owner bookkeeping with live, environment-scoped desktop hosts. A host exists only while its request stream is connected; thread selection travels with each request instead of controlling host availability.

Provider sessions are pinned to one desktop runtime so multi-step browser work cannot jump between independent Electron cookie and DOM state. Pinning survives focus changes and fails over only when that host connection disappears.

Also adds durable, per-tab browser viewport sizing for users and agents, including a complete device-toolbar interaction: the preview menu reveals a Chrome-style toolbar and resize rails with responsive sizing, exact dimensions, named device presets, rotation, and a one-click return to Fill panel.

Why

Preview automation could randomly stop accepting navigation or snapshot requests after pairing, reconnecting, switching threads, or moving focus between windows. The development app was still running, but the server had either lost the valid automation host or retained one without a usable transport.

The old design also coupled automation availability to whichever thread a desktop was currently displaying. That meant a phone-originated chat could not use an otherwise healthy Mac Mini Electron runtime unless the Mini happened to have the same thread open. Thread focus is UI state, not host capability, so it should never have been an eligibility requirement.

Browser layout testing had a separate source-of-truth problem: the guest webview always inherited the preview panel's dimensions. Users could not hold a mobile or desktop breakpoint constant while resizing the app, agents had no protocol operation for selecting or verifying a viewport, and the earlier sizing UI did not provide a coherent toolbar/rail workflow. Zoom, cursor overlays, and recordings also assumed that panel bounds and browser-content bounds were identical.

What was broken before

• The web owner used React.useId(), which is not globally unique across Electron windows or independent mounts.
• Owner snapshots and live request streams were separate broker state and could disagree.
• Independent asynchronous reportOwner and clearOwner calls allowed stale mounts to erase or recreate newer ownership.
• Client timestamps controlled focus ordering and could arrive out of order.
• Requests and responses were not fully bound to the connection lease that received them.
• Routing, pending-request insertion, stream registration, and cleanup had interleaving race windows.
• Broker provisioning was implicit enough that MCP and different WebSocket sessions were not obviously guaranteed to share one state instance.
• The renderer mounted an owner inside ChatView for only activeThreadRef, and the broker filtered by that thread. A capable desktop became unavailable merely because its UI was showing another conversation.
• Multiple connected desktops were reselected per request, allowing one agent sequence to jump between physically separate browser runtimes.
• Viewport size was ephemeral panel layout rather than part of the preview session, so it could not be controlled remotely, persisted per tab, or reported to automation.
• Content positioning, agent cursors, and recording dimensions treated the outer panel as the page viewport, which becomes incorrect as soon as a fixed viewport is centered or scrollable.
• Rapid UI resize commits could resolve out of order, allowing an older drag or dimension edit to overwrite a newer one.
• The old corner-only resize affordance was hard to grab and did not expose a complete device emulation workflow.

How host routing works now

1. Register the desktop, not the visible thread. Each Electron renderer creates one cryptographically random host identity per environment and keeps the host mounted at the renderer root. Web and iOS clients do not register automation hosts.
2. Acquire capability through a live stream. Connecting creates one broker connection and a server-issued cryptographic connectionId lease. Stream lifetime is the source of truth for availability.
3. Carry thread scope on the request. Every automation request includes its threadId. The selected desktop synchronizes that thread's preview sessions and can create a hidden Electron webview even when another thread is visible.
4. Pin provider sessions to a desktop. The first request chooses the best compatible host for the environment. Later requests from that provider session remain on the same connection regardless of focus changes. The assignment is removed on disconnect, replacement, or credential expiry.
5. Use focus only as initial preference. Among equally capable hosts, a focused desktop wins initial selection; otherwise the most recently focused or connected host wins. An unfocused desktop remains fully eligible.
6. Route and reserve atomically. Host selection, affinity assignment, and pending-request insertion happen in one synchronized broker state transition.
7. Validate every response. Responses must match the exact client, connection lease, and pending request. Stale streams cannot complete newer work.
8. Let stream finalizers own cleanup. Finalization removes only the exact lease and its pending work. An old stream cannot clear its replacement.
9. Share one broker explicitly. MCP and every WebSocket session receive the same broker instance at server route assembly.

How the device-toolbar flow works now

1. Hidden by default. The preview's More menu contains Show device toolbar. In Fill mode the guest continues to track the available panel exactly, with no extra toolbar or rails taking space.
2. Show without a layout jump. Enabling the toolbar snapshots the guest's currently available CSS-pixel dimensions into Responsive mode, then reveals the host-side toolbar and its resize frame.
3. Resize from real rails. Forty-pixel left, right, and bottom rails plus both bottom corners provide large pointer and keyboard targets. Drag math tracks the grabbed edge across centered, clipped, zoomed, and panel-boundary states.
4. Choose or type a size. The toolbar selector groups common desktop, laptop, iPad, iPhone, Pixel, and Galaxy presets. Width and height fields allow exact freeform values; edits commit on Enter or when leaving the control.
5. Rotate in place. Rotate swaps width and height while preserving the selected preset identity, so portrait and landscape remain the same device selection.
6. Return to Fill. The toolbar's close button or Hide device toolbar switches the tab back to Fill panel and removes the toolbar and rails.

The toolbar is derived from the tab's viewport setting, not from a separate visibility flag. Therefore an agent calling preview_resize with a preset or freeform size exposes the same UI, while selecting Fill hides it. UI commits are serialized per tab so fast drags, keyboard changes, exact-input edits, and MCP updates cannot settle out of order.

Fixed Responsive and preset viewports remain unchanged when the right panel is resized; the outer area only changes centering or scrolling. The Electron host converts CSS-pixel targets to rendered element bounds using the tab's current zoom factor. Agent cursors, scrolling, hidden tabs, and recording canvases use the actual content rectangle rather than the outer panel rectangle.

The selected mode and dimensions live in the server's per-thread, per-tab preview snapshot. Navigation and status updates preserve them, and a resized event synchronizes every connected client without activating a background tab. Browser tabs and device sizes therefore remain scoped to their own thread even when another thread is visible.

The preview_resize MCP tool accepts fill, freeform, or preset input. It waits until the renderer has applied the requested mode and the guest reports the expected window.innerWidth/window.innerHeight; preview_status reports both the setting and measured CSS viewport.

Device presets intentionally model viewport dimensions only. They do not spoof a mobile user agent, device-pixel ratio, touch input, or other device capabilities.

Remote behavior

• A chat sent from iOS can use a connected Mac Mini or MacBook desktop host for that environment, including creating a hidden webview for a thread the desktop is not displaying.
• The desktop does not need to display the target thread, keep the preview panel open, or hold window focus.
• Browser tabs and viewport settings remain scoped per thread/tab. Opening or resizing a browser in thread B never makes thread B's preview appear in thread A's right panel.
• If both Mac Mini and MacBook are eligible and equally capable, focus influences only the initial choice. The provider session then remains pinned to that machine.
• If the pinned desktop disconnects, the next request selects a remaining compatible host. It does not silently switch machines while the original host is alive.

Resulting invariants

• No automation host exists without a live request stream.
• Host capability is environment-scoped and independent of routed thread UI.
• A stale connection cannot focus, clear, or respond for its replacement.
• A provider session cannot jump between live desktop runtimes.
• Pending requests cannot cross connection or server-lifetime boundaries.
• All MCP and WebSocket routing observes the same broker state.
• Every preview tab has one server-owned viewport setting; missing legacy data deterministically means Fill panel.
• Fixed viewport dimensions do not change when the outer panel changes.
• Toolbar visibility, resize rails, UI actions, and MCP resizing all derive from that same per-tab setting.
• A successful agent resize result reflects the measured guest CSS viewport, not assumed DOM bounds.

Testing

• Preview broker, MCP, renderer-root, identity, stream-consumer, cross-WebSocket, resize persistence, compatibility routing, and sticky-assignment coverage remains in place.
• Focused viewport layout, zoom, rail geometry, responsive capture, serialized action, and client-state suites: 12 tests passed across 3 files.
• vp check — passed with 0 errors (20 existing repository-wide warnings).
• vp run typecheck — passed across all 15 packages.
• React Doctor — no new device-toolbar findings; remaining findings are existing compiler/manual-memoization diagnostics outside this interaction.

---

Note

High Risk
Large cross-cutting change to MCP/WebSocket preview routing, session affinity, and Electron CDP input; breaking error-tag changes for automation callers.

Overview
Preview automation no longer uses replicated reportOwner / clearOwner state. Desktops connect as environment-scoped hosts over a live stream (connected + request events) with a server-issued connectionId; focusHost only affects tie-breaking. The broker pins each provider session to one host, routes threadId on each request, validates responses against the pending lease, and is provisioned once for MCP and all WebSocket sessions. Public errors shift toward PreviewAutomationNoAvailableHostError (legacy focused-owner / host-not-connected tags drop out of the union).

Viewport sizing becomes server-owned per tab (fill, freeform, presets) with preview_resize / WS previewResize, resized events, and status that reports setting plus measured CSS size. The web renderer adds PreviewAutomationHosts, a device toolbar and resize rails on HostedBrowserWebview, and content vs panel rects so zoom, cursors, and recordings track the guest viewport.

Desktop preview gains zoom sync across navigation/webview replacement, main-frame load failures that persist until a new load, CDP key sequences (PreviewKeyboard) with focus emulation and cleanup on interrupt, and typing via in-page execCommand (plus PreviewAutomationTargetNotEditableError). Dev refreshes the Mac launcher script env as fallback-only exports and retries transient t3code-dev:// renderer load failures.

^{Reviewed by Cursor Bugbot for commit ba9d8e5. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Route preview automation through live owner streams with connection-aware host tracking

• Replaces the owner-based automation model with a stream-based host model: PreviewAutomationBroker now returns a Stream<PreviewAutomationStreamEvent> (with connected and request events) per connected host, and tracks connections via generated connectionId values with sticky routing, lease expiry, and capability negotiation.
• Adds a resize automation operation end-to-end: new RPC (preview.resize), PreviewAutomationResizeInput/Result schemas, server-side PreviewManager.resize handler, MCP tool, and a device toolbar UI (BrowserDeviceToolbar) with interactive resize handles (BrowserViewportResizeHandles) in the web client.
• Introduces focusHost (replacing reportOwner/clearOwner) for focus updates, scoped per (environmentId, clientId, connectionId) with latest-wins concurrency.
• The desktop PreviewManager now types into pages via page runtime (avoiding Input.insertText), dispatches Chromium-correct key events with focus emulation and cleanup, and emits a typed PreviewAutomationTargetNotEditableError when a target is not editable.
• Adds HostedBrowserWebview viewport layout, zoom-aware scaling, scroll tracking, and content geometry publishing via a new browserSurfaceStore presentContent method.
• Risk: PreviewAutomationHostNotConnectedError and PreviewAutomationNoFocusedOwnerError are removed from the public error union; callers must handle PreviewAutomationNoAvailableHostError and PreviewAutomationTargetNotEditableError instead.

^{Macroscope summarized ba9d8e5.}