fix(dashboard): survive browser disconnects in /chat PTY

[cpmidnite]

fix(dashboard): survive browser disconnects in /chat PTY

Problem

The dashboard /chat tab opens a WebSocket to /api/pty, which spawns a hermes --tui child behind a POSIX pseudo-terminal. When the browser disconnects — laptop sleep, tab loses focus long enough for the WS to time out, network blip, etc. — the WS handler hits WebSocketDisconnect, finally calls bridge.close(), and the TUI child gets killed.

On reconnect, the front-end opens a fresh WS, the server spawns a brand-new hermes --tui, and the user's conversation appears lost even though the underlying session is still on disk. Hitting "Resume session" from the sidebar gets back to the session, but xterm.js scrollback is also gone because the front-end always generates a new channel id per mount, forcing a clean re-mount.

This is a recurring papercut for anyone using the in-browser dashboard chat across naturally bursty browsing patterns.

Approach

Three coordinated changes — server-side PTY survival + scrollback ring buffer + stable client-side channel id:

1. Server: channel-keyed PTY registry with grace-period eviction

• New _PTY_BY_CHANNEL: dict[str, _PtySession] keyed by the same channel id pty_ws already accepts via query param.
• Browser disconnects no longer call bridge.close() immediately. Instead they mark the session detached, record last_attached_at, and schedule a one-shot asyncio.Task (_evict_after_grace) that closes the bridge after HERMES_PTY_RECONNECT_GRACE_SECONDS (default 300s, capped at 3600s) if no reattach happens.
• On WS connect: look up the channel; if a live bridge exists and isn't currently attached, cancel any pending eviction and reuse it. If the existing entry's bridge has already died (e.g. user typed /quit), evict and spawn fresh.
• Concurrent attach attempts on the same channel are rejected with WS close code 4409 + a one-line ANSI error frame, so a second tab can't hijack the active session.
• New @app.on_event("shutdown") closes all live bridges so hermes dashboard exit doesn't leak children. (We're aware FastAPI deprecates on_event in favor of lifespan; the rest of web_server.py still uses on_event so this stays consistent — happy to migrate the whole file in a follow-up PR if maintainers prefer.)

2. Server: PtyBridge scrollback ring buffer

• PtyBridge.__init__ takes new scrollback_bytes (default 256 * 1024).
• Every non-empty chunk returned from read() is appended via _append_scrollback() which trims FIFO to the configured limit.
• snapshot_scrollback() returns the buffer as bytes (safe to call from the event loop thread after the executor-side read completes; single-writer single-reader on the buffer).
• On reattach (not on fresh spawn), pty_ws sends \x1b[2J\x1b[H (clear + home), the snapshot bytes, then a grey [reconnected] marker before resuming the live pump. Fresh spawns are unchanged — TUI paints its own opening frame.

3. Client: stable channel id + last-session memory

• ChatPage.tsx now persists channel id in localStorage["hermes.chat.channel"] so a tab refresh reuses the same channel (and thus the same surviving PTY).
• Channel id is regenerated only when ?resume= changes (tracked via hermes.chat.channel_resume) — so jumping to a different session lineage still gets a fresh PTY.
• Last session id is persisted in localStorage["hermes.chat.last_session_id"]; if the user lands on bare /chat (no ?resume=), we set the URL param from storage so reload-without-bookmark also resumes.
• All localStorage access is wrapped in try/catch — Safari private browsing falls back to ephemeral ids without crashing.
• Channel id validated against the same [A-Za-z0-9._-]{1,128} regex _VALID_CHANNEL_RE enforces server-side; malformed stored values are discarded.

What's intentionally NOT in this PR

• No client-side WebSocket auto-reconnect loop. Survival comes from the next natural mount (page refresh, navigation back, tab refocus that triggers React re-mount) reattaching successfully. An auto-reconnect-on-onclose loop is a separate feature with its own retry/backoff/auth-refresh edge cases.
• No persistence across hermes dashboard process restart. Registry is in-process only. Restart still loses live PTYs (TUI children die with the parent).
• No changes to tui_gateway / Ink / the TUI itself. This PR is purely the PTY plumbing.
• No changes to /api/ws JSON-RPC sidebar. That endpoint sits right next to pty_ws and was left untouched.

Testing

Added TestScrollback (3 cases) and TestPtySurvival (4 cases):

• test_scrollback_collects_recent_bytes / test_scrollback_trims_to_limit / test_scrollback_empty_after_spawn
• test_pty_survival_reattach_same_channel_reuses_pty — connects, disconnects, reconnects with same channel, asserts identical bridge PID.
• test_pty_grace_evicts_after_timeout — disconnect + sleep past grace, asserts channel evicted and bridge dead. Grace overridden to 0.2s via env var to keep test fast.
• test_pty_reattach_replays_scrollback — asserts the [reconnected] marker plus echoed bytes arrive on reattach.
• test_pty_second_concurrent_attach_rejected — second concurrent attach gets WS close 4409.

Tests use TestClient with _resolve_chat_argv monkeypatched to a trivial /bin/sh echo loop, so they don't depend on a real hermes --tui build. All 4 TestPtySurvival tests pass; all 3 TestScrollback tests pass.

Verification run:

$ /home/apple/.hermes/hermes-agent/venv/bin/python -m pytest \
    tests/hermes_cli/test_pty_bridge.py \
    tests/hermes_cli/test_web_server.py -q
...
169 passed in ~17s

$ ruff check hermes_cli/web_server.py hermes_cli/pty_bridge.py \
    tests/hermes_cli/test_pty_bridge.py tests/hermes_cli/test_web_server.py
All checks passed!

$ cd web && npm run build
...build OK

Manual smoke-test against the real ASGI app via TestClient:

pid1: 919329
attached after disconnect: False
scrollback bytes captured: 30
pid2 (after reattach): 919329
SAME PID: True

Files touched

File	LOC
hermes_cli/web_server.py	+169 / -20
hermes_cli/pty_bridge.py	+22 / -2
tests/hermes_cli/test_pty_bridge.py	+34 / -0
tests/hermes_cli/test_web_server.py	+158 / -1
web/src/pages/ChatPage.tsx	+80 / -10
Total	+461 / -34

Compatibility / config knobs

• HERMES_PTY_RECONNECT_GRACE_SECONDS — float seconds, default 300, max 3600, invalid values fall back to default.
• localStorage keys introduced: hermes.chat.channel, hermes.chat.channel_resume, hermes.chat.last_session_id. None collide with existing dashboard storage.
• Default behavior change for existing users: closing the browser tab no longer kills the dashboard chat's TUI process for 5 minutes. Memory cost per orphaned session: ~256 KiB scrollback + the running hermes --tui itself. Shutdown handler reaps everything on hermes dashboard exit so it's bounded.

Notes for reviewers

• I'm a downstream user of the dashboard, not a Nous maintainer — happy to revise any of the design decisions (eviction grace default, scrollback size, channel-id storage strategy, on_event → lifespan migration) if you'd prefer a different shape.
• The deprecated @app.on_event("shutdown") is consistent with existing usages in the same file; flagged in case a sweep makes sense as a follow-up.
• The 4409 WS close code is a stretched-meaning use of HTTP 409 Conflict in the WS code-space (4000-4999 is the application-defined range, RFC 6455). Open to a different code if maintainers have a convention.

[teknium1]

Closing — the keep-alive/reattach approach for dashboard /chat PTYs landed on main in #60515 (salvage of #50084 by @TinkerOfThings, who filed the same fix later with a token-keyed registry + ring-buffer replay + explicit close-code contract). You were the earliest to attack this problem (May 28) — thanks for pushing on it; credit noted in the salvage PR body. The shipped design covers your channel-keyed survival + scrollback-replay goals.