Skip to content

Fix TUI slash worker zombie subprocess leak (#38095) - #38114

Closed
Elshayib wants to merge 2 commits into
NousResearch:mainfrom
Elshayib:fix/tui-slash-worker-zombies
Closed

Fix TUI slash worker zombie subprocess leak (#38095)#38114
Elshayib wants to merge 2 commits into
NousResearch:mainfrom
Elshayib:fix/tui-slash-worker-zombies

Conversation

@Elshayib

@Elshayib Elshayib commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #38095 — TUI slash worker subprocesses leak after session close, accumulating zombie processes.

Three related fixes in tui_gateway/server.py:

  1. _finalize_session() now closes the slash worker. Previously, any code path that called _finalize_session() without separately handling the worker would leak the subprocess. The session.close handler and _shutdown_sessions() had redundant separate close logic, but _finalize_session() itself did not.

  2. _SlashWorker.close() is more robust. Increased terminate→wait timeout from 1s to 5s, added explicit stdout/stderr/stdin close to unblock reader threads, and a final wait(2) to reap the zombie. The old 1s timeout was too short for slow machines under load.

  3. _restart_slash_worker() clears the old reference. Sets session["slash_worker"] = None after closing the old worker, preventing a window where both old and new workers are simultaneously live (observed as duplicate session-key workers in the wild).

Tests

Added 5 new tests in tests/test_tui_gateway_server.py:

  • test_finalize_session_closes_slash_worker
  • test_finalize_session_handles_missing_worker
  • test_session_close_jsonrpc_closes_worker
  • test_restart_slash_worker_clears_old_reference
  • test_slash_worker_close_reaps_process

All 5 tests pass.

islam666 and others added 2 commits June 3, 2026 09:58
The main branch reported 0.15.1 / 2026.5.29 while the latest published
release tag v2026.5.29.2 reports 0.15.2 / 2026.5.29.2. This caused
source installs tracking origin/main to appear stale/downgraded.

Update pyproject.toml, hermes_cli/__init__.py, and acp_registry/agent.json
to match the release tag metadata.

Fixes #38093
Three related fixes in tui_gateway/server.py:

1. _finalize_session() now closes the slash worker subprocess before
   returning. Previously, any code path that called _finalize_session()
   without separately handling the worker would leak the subprocess.

2. _SlashWorker.close() is more robust: increased terminate->wait timeout
   from 1s to 5s, added explicit stdout/stderr/stdin close to drain
   reader threads, and a final wait(2) to reap the zombie. The old 1s
   timeout was too short for slow machines under load.

3. _restart_slash_worker() now clears session[\"slash_worker\"] = None
   after closing the old worker, preventing a window where both old and
   new workers are simultaneously live (observed as duplicate
   session-key workers in the wild).

Co-Authored-By: OWL <noreply@zoo-ai.com>
@Elshayib
Elshayib requested a review from a team June 3, 2026 10:22
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/tui Terminal UI (ui-tui/ + tui_gateway/) labels Jun 3, 2026
@teknium1

teknium1 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Superseded by #42132 (merged), which closes the slash_worker subprocess leak via two guards: process-group kill on PTY teardown + a cross-platform parent-death watchdog in the worker. Closing as resolved — thanks for tackling this; the merged fix salvaged the process-group-kill and watchdog approaches with contributor authorship preserved.

@teknium1 teknium1 closed this Jun 8, 2026
@Elshayib
Elshayib deleted the fix/tui-slash-worker-zombies branch June 8, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: TUI slash worker subprocesses leak after session close, accumulating zombie processes

3 participants