Skip to content

fix(tui): stop recovery hanging after gateway restart - #63437

Open
morluto wants to merge 1 commit into
NousResearch:mainfrom
morluto:fix/tui-rearm-events-after-gateway-restart
Open

fix(tui): stop recovery hanging after gateway restart#63437
morluto wants to merge 1 commit into
NousResearch:mainfrom
morluto:fix/tui-rearm-events-after-gateway-restart

Conversation

@morluto

@morluto morluto commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes a TUI recovery hang after the Python gateway crashes or disconnects.

The TUI already attempts to recover automatically:

  1. It detects that the gateway exited.
  2. It starts a replacement gateway.
  3. The replacement sends gateway.ready.
  4. The TUI should receive that event and resume the user's session.

Step 4 is currently broken. Restarting the gateway resets GatewayClient to an unsubscribed state. The replacement gateway's events are buffered, but the React subscription effect only runs when the TUI first mounts—it does not run again after a gateway restart.

As a result, the replacement process starts successfully, but its gateway.ready event is never delivered. The TUI remains stuck on:

gateway exited · recovering session…

The user must restart the entire TUI to recover.

This PR makes GatewayClient remember that the TUI consumer already subscribed. After replacing a gateway transport, it automatically re-arms the existing deferred event drain. The replacement gateway's gateway.ready event can then reach the recovery handler and resume the persisted session.

The initial startup behavior from #36658 remains unchanged: events received before React mounts are still buffered and delivered on a later microtask.

Related Issue

Fixes #63434.

Related history:

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Separated the lifetime of the mounted TUI consumer from the lifetime of an individual gateway transport.
  • Re-armed deferred event delivery whenever GatewayClient resets for a replacement transport.
  • Preserved the generation guard that prevents stale drain microtasks from delivering old events.
  • Added a regression that replaces a live gateway transport and verifies that:
    • the original gateway.ready is delivered
    • the replacement gateway.ready is delivered
    • the following session.info arrives once and in FIFO order

How to Test

From ui-tui:

npm run build --prefix packages/hermes-ink

npm test -- --run \
  src/__tests__/gatewayClient.test.ts \
  src/__tests__/gatewayRecovery.test.ts \
  src/__tests__/createGatewayEventHandler.test.ts

npm run typecheck
npx eslint src/gatewayClient.ts src/__tests__/gatewayClient.test.ts
npx prettier --check src/gatewayClient.ts src/__tests__/gatewayClient.test.ts

Results:

3 test files passed
96 tests passed
TypeScript typecheck passed
ESLint passed
Prettier passed

The new regression fails on current main:

expected [ 'gateway.ready' ] to have a length of 3 but got 1

Only the original gateway event is delivered. The replacement gateway's events remain buffered.

With this change, the observed order is:

gateway.ready
gateway.ready
session.info

This proves that event delivery survives the transport replacement and preserves FIFO ordering.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature
  • I've run all affected tests and checks; they pass
  • I've added tests for my changes
  • I've tested on Linux 6.8 with Node.js

Documentation & Housekeeping

  • Documentation — N/A; no user-facing interface changed
  • cli-config.yaml.example — N/A; no configuration changed
  • CONTRIBUTING.md or AGENTS.md — N/A; no architecture or workflow changed
  • Cross-platform impact considered; the change uses platform-independent event and microtask behavior
  • Tool descriptions and schemas — N/A; no tool behavior changed

Screenshots / Logs

Not applicable. This transport-lifecycle failure is covered by a deterministic fake-WebSocket regression.

@alt-glitch alt-glitch added type/bug Something isn't working comp/tui Terminal UI (ui-tui/ + tui_gateway/) sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state P2 Medium — degraded but workaround exists labels Jul 12, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused recovery regression. The premise remains present on current main: GatewayClient.resetStartupState() clears subscribed at ui-tui/src/gatewayClient.ts:221, so later gateway.ready frames buffer through publish() at ui-tui/src/gatewayClient.ts:170-175. The mounted consumer calls gw.drain() only in its effect at ui-tui/src/app/useMainApp.ts:824-826, while the recovery handler restarts the gateway at ui-tui/src/app/useMainApp.ts:810-815.

The patch preserves the existing generation guard and deferred FIFO drain behavior from ui-tui/src/gatewayClient.ts:618-662, while making that mounted-consumer state survive replacement transport startup. Current main has no subsequent changes to either touched file since the PR base, so this should be a clean salvage.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: TUI recovery hangs after gateway restart

3 participants