feat(tui_gateway): async-side shared heavy-read bound for session.list (seam a)

[Kyzcreig]

Phase 2 of docs/desktop/2026-07-05-session-list-loop-starvation-PRD.md (D-4/D-5/INV-4/INV-6). Shared per-loop async SessionDB heavy-read gate (default 2, config dashboard.heavy_read_max_concurrency, lazy/loop-bound NOT import-frozen). REST + WS list_sessions_rich (and siblings) acquire the SAME semaphore async-side BEFORE asyncio.to_thread dispatch (seam a — not inside the worker pool, so cheap ws ops never starve). Saturation: bounded wait + queue_wait log + shed stats + retryable backend_busy (503/JSON-RPC). Dashboard status exposes session_db_heavy_reads stats. 349 gateway tests green (Apollo re-ran per-file runner). Reviewed by Apollo. Live K=8 incident-regime certify = Apollo post-merge.

[greptile-apps]

Greptile Summary

This PR adds a shared async gate for heavy SessionDB reads in the gateway. The main changes are:

• A loop-local SessionDB heavy-read semaphore with busy responses and stats.
• REST heavy reads routed through the shared gate.
• Websocket SessionDB-heavy methods routed through the same gate before worker dispatch.
• A dashboard config default and status field for heavy-read concurrency.
• Tests for shared REST/WS bounding, shedding, and queued websocket behavior.

Confidence Score: 4/5

The shared gate needs fixes around semaphore replacement and websocket executor isolation before merging.

• A config change can let old and new semaphores admit heavy reads at the same time.
• Gated websocket long handlers now run in the default executor instead of the gateway RPC pool.
• Status polling can add synchronous config reads to the event loop.

hermes_cli/session_db_heavy_gate.py, tui_gateway/ws.py, tui_gateway/server.py

Important Files Changed

Filename	Overview
hermes_cli/config.py	Adds the default dashboard heavy-read concurrency setting.
hermes_cli/session_db_heavy_gate.py	Adds the shared gate, busy error, stats, and test reset helper; config reload and status stats need attention.
hermes_cli/web_server.py	Routes REST heavy SessionDB reads through the new gate and exposes gate stats in status.
tui_gateway/server.py	Adds heavy-method classification, busy JSON-RPC errors, and a bound synchronous request helper.
tui_gateway/ws.py	Routes selected websocket SessionDB-heavy methods through the shared gate before running handlers.
tests/test_web_server_sessiondb_eventloop.py	Adds tests for shared bounding, queue shedding, lazy sizing, and websocket non-starvation.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
  participant REST as REST endpoint
  participant WS as WS endpoint
  participant Gate as Shared heavy-read gate
  participant Exec as Worker thread
  participant DB as SessionDB

  REST->>Gate: request heavy read slot
  WS->>Gate: request heavy read slot
  alt slot acquired
    Gate->>Exec: run blocking read
    Exec->>DB: scan sessions
    DB-->>Exec: result
    Exec-->>Gate: return
    Gate-->>REST: response
    Gate-->>WS: JSON-RPC response
  else wait timeout
    Gate-->>REST: 503 backend_busy
    Gate-->>WS: JSON-RPC backend_busy
  end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
  participant REST as REST endpoint
  participant WS as WS endpoint
  participant Gate as Shared heavy-read gate
  participant Exec as Worker thread
  participant DB as SessionDB

  REST->>Gate: request heavy read slot
  WS->>Gate: request heavy read slot
  alt slot acquired
    Gate->>Exec: run blocking read
    Exec->>DB: scan sessions
    DB-->>Exec: result
    Exec-->>Gate: return
    Gate-->>REST: response
    Gate-->>WS: JSON-RPC response
  else wait timeout
    Gate-->>REST: 503 backend_busy
    Gate-->>WS: JSON-RPC backend_busy
  end

Loading

_{Reviews (1): Last reviewed commit: "feat(tui_gateway): bound heavy session r..." | Re-trigger Greptile}

[greptile-apps]

Queued Waiters Cross Gate Swap

When the config value changes while tasks are already waiting on the old semaphore, _value can still look fully drained because queued acquire() calls have not decremented it yet. This replaces the loop entry with a new semaphore, so new requests can enter through the new gate while old waiters later enter through the old gate, temporarily exceeding the configured heavy-read bound.

[greptile-apps]

Status Polls Load Config

session_db_heavy_read_stats() now runs inside the async status endpoint, but it calls _configured_max_concurrency(), which synchronously loads dashboard config. Frequent status polling or a slow config filesystem can block the event loop and delay unrelated REST or websocket work.

[greptile-apps]

Heavy Handlers Leave RPC Pool

For heavy methods that are also long handlers, this path bypasses server.dispatch() and runs the full handler in asyncio.to_thread() instead of the gateway RPC pool. With a small or shared default executor, two slow session.list or projects.tree scans can occupy the default executor and delay otherwise cheap websocket offloads.

[greptile-apps]

Handler Errors Become Dispatch Crashes

handle_request_bound() calls handle_request() without the exception mapping that dispatch() applies for long-handler workers. If a gated websocket handler raises, clients now receive a generic -32603 internal error from handle_ws instead of the previous handler error response, and telemetry counts it as a dispatch crash.