fix(desktop): probe half-open gateway socket on wake and reconnect - #90769
fix(desktop): probe half-open gateway socket on wake and reconnect#90769Owen-narcissus wants to merge 1 commit into
Conversation
macOS sleep/wake (or a silent network drop) can leave the renderer's WebSocket half-open: no close event fires, so connectionState stays 'open' while every RPC hangs until its per-call timeout. prompt.submit's timeout is 30 minutes, so the user's next message reads as "enter does nothing until I restart the app". - Add a minimal ping RPC (tui_gateway/server.py) answered synchronously on the WS reader thread. - On wake signals, reconnectNow now probes the open-looking socket with a 5s-bounded ping and force-closes it on failure, letting the existing reconnect machinery (backoff, tile rebinding, session refresh) take over. A pre-ping backend answering -32601 is treated as healthy. - Tests: half-open socket force-reconnects; healthy socket untouched; method-not-found backend untouched; backend ping envelope contract.
|
Thanks for the triage note linking #89092. Since both PRs target the same half-open-socket class after wake, here's how this one differs: #89092 (force-redial on recovery): on power resume / network online / gateway reconnect, unconditionally closes the socket when it still reports This PR (#90769, probe-then-act): adds a minimal Both approaches fix the reported "enter does nothing until I restart the app" symptom; the difference is whether recovery is unconditional (redial on every wake) or verified (redial only when the probe proves the socket is dead). The two are compatible — a probe-then-redial path could sit on top of #89092's force-redial wiring if maintainers prefer that direction. Tests: half-open socket force-reconnects; healthy socket untouched; method-not-found backend untouched; backend ping envelope contract. |
|
Additional data point: Windows + WSL2 + remote gateway (same half-open socket symptom) Same root cause confirmed on a Windows 11 host with Hermes Desktop connected to a WSL2 backend via remote gateway ( Symptom: Desktop intermittently loses the connection; the user must click "Save & Reconnect" in Gateway settings. Typing after a silent drop appears to do nothing until the connection is rebuilt. Backend log evidence (gui.log,
Version: backend v0.20.4 (includes #55545 loopback ping fix; note the keepalive ping stays active on non-loopback binds, so a GIL stall can still false-positive there — but in this case no stalls were logged, the drops are client-side). This confirms the half-open socket scenario also occurs on Windows/WSL2 remote-gateway setups, not just macOS local. A |
|
Merged via #93694 (rebase-merge, your commit preserved with full authorship — thank you!). We kept your ping RPC + probe exactly as designed, and added two things on top: the backend-update success path now nudges the same reconnect (fixes the 'force-quit after remote update' report), and the old blind gateway.close() on every wake signal was removed in favor of your probe — your own healthy-socket tests showed the blind close was churning good connections. Closing as landed. |
Summary
Fix a desktop hang where, after sleep/wake or a silent network drop, typing a new prompt and pressing Enter appears to do nothing until the app is restarted.
Problem
macOS sleep (or a network switch) can leave the renderer's WebSocket half-open: the TCP connection dies without a
closeevent, soconnectionStatestill reads'open'while every RPC hangs until its per-call timeout. The wake nudges inuseGatewayBoot(reconnectNow) skipped reconnect whenever the socket reported open, so the dead socket was never rebuilt.The first thing the user types after waking hits
prompt.submit, whose request timeout is 30 minutes (PROMPT_SUBMIT_REQUEST_TIMEOUT_MS— deliberately long because turn ACKs can legitimately take minutes, see #55024). The message silently hangs for that whole window. Restarting the app rebuilds the socket, which is why only a restart appeared to fix it.On the backend, the disconnected-but-idle session then gets reaped by the WS-orphan reaper 20s after the socket drops, compounding the client/server state mismatch.
Changes
pingJSON-RPC method (tui_gateway/server.py) answered synchronously on the WS reader thread — the round-trip measures socket health, not backend load, so it works even mid-turn.reconnectNownow probes the open-looking socket with a 5-second-boundedpingand force-closes it on failure, letting the existing reconnect machinery (backoff,resetTileRuntimeBindings,refreshSessions) take over. A pre-ping backend answering-32601(method not found) proves the socket is alive and is deliberately not reconnected, so version-skewed backends can't spin the reconnect loop.-32601backend → untouched (16/16 inuse-gateway-boot.test.tsx).pingenvelope contract (tests/tui_gateway/test_ping_probe.py).Verification
npx vitest run src/app/gateway/hooks/use-gateway-boot.test.tsx— 16/16 passed.tsc --noEmitclean;npm run packsucceeded for macOS arm64.Scope
This does not change the 30-minute
prompt.submittimeout (that is intentional for long turns); it ensures a dead connection fails fast before a submit ever hangs against it. Unrelated working-tree files are not included.