fix(tui): keep long slash commands (/hatch) alive with worker heartbeats

[liuhao1024]

What does this PR do?

/hatch and other minutes-long TUI slash commands were killed by the gateway's 45s slash-worker timeout while the worker subprocess kept running orphaned, burning image credits with no way to deliver the result. On top of that, the TUI's blanket slash.exec fallback retried command.dispatch on any rejection, so the user saw the misleading not a quick/plugin/bundle/skill command: hatch instead of the real timeout error (#99831).

This PR implements the issue's minimal-mitigation direction:

• Worker heartbeats (tui_gateway/slash_worker.py): while a command executes, the worker emits {"id": rid, "heartbeat": true} lines at half the configured slash timeout (capped at 30s; HERMES_SLASH_HEARTBEAT_S overrides). Protocol writes are guarded by a module lock so heartbeat and result lines can never interleave into corrupt JSON.
• Gateway treats heartbeats as liveness (tui_gateway/server.py): _SlashWorker.run() restarts its timeout window on a heartbeat instead of failing, so a live-but-slow command is never declared timed out. A wedged worker — one whose heartbeat thread is dead too — still hits the timeout, preserving the watchdog semantics.
• The TUI surfaces real worker errors (ui-tui/src/app/createSlashHandler.ts): worker-infrastructure failures (slash worker timed out, exited, closed pipe, start failed, failed) no longer retry command.dispatch; they surface verbatim. Route signals (use command.dispatch) and other failures still fall through to the dispatch retry, exactly as before.

Related Issue

Fixes #99831

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• tui_gateway/slash_worker.py: added _resolve_heartbeat_s() + _emit_heartbeats() and a _stdout_lock; the command loop now runs the heartbeat thread while _run() executes and writes its result line under the lock.
• tui_gateway/server.py: _SlashWorker.run() skips heartbeat messages so they reset the wait window instead of tripping if not msg.get("ok").
• ui-tui/src/app/createSlashHandler.ts: the slash.exec catch only falls through to command.dispatch when the error is not a worker-infrastructure failure.
• Tests: tests/test_slash_worker_watchdog.py (heartbeat interval derivation, emitter stop semantics, broken-pipe quietness), tests/test_tui_gateway_server.py (heartbeat resets the window + stale-response skipping; silent worker still times out), ui-tui/src/__tests__/createSlashHandler.test.ts (timeout surfaces verbatim, dispatch retry must not fire).

How to Test

1. pytest tests/test_slash_worker_watchdog.py -q — Observed result: 6 passed.
2. pytest tests/test_tui_gateway_server.py -q -k "slash" — Observed result: 25 passed (incl. the 2 new heartbeat tests).
3. pytest tests/tui_gateway/ -q — Observed result: 915 passed, 2 pre-existing order-dependent failures (test_gui_surface_toolsets.py::test_holds_exactly_the_gui_affordances, test_stale_provider_resume_live.py::test_renamed_provider_heals_to_new_identity) that reproduce identically on a clean upstream/main checkout of this worktree (verified via stash-diff baseline).
4. cd ui-tui && ../node_modules/.bin/vitest run src/__tests__/createSlashHandler.test.ts — Observed result: 82 passed.
5. Manual end-to-end: run hermes --tui, invoke /hatch <description>, and confirm the job no longer aborts at 45s with the misleading error; the worker keeps heartbeating and the result is delivered when generation completes.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 26 (arm64)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Not applicable (backend behavior + unit tests; the issue's repro log lines are quoted in #99831).

[Enough1122]

AI code review — automated review for reference; please use your judgment.

The fix correctly addresses the watchdog false-positive (#99831): the worker now emits {"id": rid, "heartbeat": true} lines at half the configured slash timeout (capped at 30s, floor 0.5s, HERMES_SLASH_HEARTBEAT_S override), and the gateway's _SlashWorker.run treats a heartbeat as liveness — continue before the ok check (tui_gateway/server.py) so live-but-slow commands like /hatch are no longer killed, while a silent worker still hits the timeout. The rid guard means stale heartbeats from a prior command are ignored, and the shared _stdout_lock prevents heartbeat/result lines from interleaving on the JSON-line protocol.

The worker-side lifecycle is careful: hb_done is set and the heartbeat thread joined (with a safety margin) before the result is written, in both the success and exception paths, and a broken pipe ends the heartbeat thread quietly rather than raising. The UI change is also sound — routing slash worker (timed out|exited|closed pipe|start failed|failed) errors straight to the error surface instead of falling through to command.dispatch, which would mask the real failure with a "not a command" answer; the new test pins both the surfaced error and the absence of the dispatch retry.

Non-blocking: heartbeats prove the worker process's heartbeat thread is alive, not that the command is making progress — a worker whose main _run loop deadlocks while the heartbeat thread stays healthy would now keep the gateway waiting indefinitely instead of timing out. This is an acknowledged trade-off of the fix, but worth a heartbeat "stall" counter or a cap on total wait. Also, if stdout is ever a full/blocked pipe, the heartbeat thread can hold _stdout_lock while blocked in write(), and the main thread's subsequent result write would block on that lock forever — pre-existing blocking risk, but the new lock makes the failure mode a hang rather than a dropped line.

Verdict: LGTM

[liuhao1024]

Thanks for the review! The thread-vs-progress trade-off is acknowledged — a stall counter is a reasonable follow-up but beyond this fix's scope; the blocked-pipe lock risk is pre-existing, as noted.