Skip to content

fix(tui): reap orphaned slash_worker processes on gateway startup (Fixes #48643) - #48656

Open
Nigmat-future wants to merge 3 commits into
NousResearch:mainfrom
Nigmat-future:fix/slash-worker-orphan-reap-48643
Open

fix(tui): reap orphaned slash_worker processes on gateway startup (Fixes #48643)#48656
Nigmat-future wants to merge 3 commits into
NousResearch:mainfrom
Nigmat-future:fix/slash-worker-orphan-reap-48643

Conversation

@Nigmat-future

Copy link
Copy Markdown

Summary

  • Reap orphaned tui_gateway.slash_worker processes on gateway startup when they no longer descend from a live tui_gateway process
  • Bind newly spawned slash workers to a Windows kill-on-close job object so they die with the gateway even on hard kill
  • Treat zombie/dead parents as orphaned in the slash_worker parent-death watchdog

Fixes #48643

Test plan

  • pytest tests/tui_gateway/test_slash_worker_lifecycle.py tests/test_slash_worker_watchdog.py -q (11 passed)
  • pytest tests/test_tui_gateway_server.py::test_slash_worker_close_reaps_zombie_and_closes_fds tests/test_tui_gateway_server.py::test_restart_slash_worker_closes_orphan_when_session_reaped -q
  • Manual (Windows): kill gateway mid-compaction, restart, confirm no stale slash_worker PIDs remain and Desktop reconnects without taskkill

Orphaned tui_gateway.slash_worker subprocesses survived gateway restarts
and blocked Desktop reconnects. Reap unowned workers at startup, bind new
workers to a Windows kill-on-close job object, and treat zombie parents as
orphaned in the in-worker watchdog.

Fixes NousResearch#48643
@alt-glitch alt-glitch added type/bug Something isn't working comp/tui Terminal UI (ui-tui/ + tui_gateway/) comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists labels Jun 18, 2026
psutil 7.x exposes ppid as a method; attribute access returned a bound
method and crashed the orphan reaper on real process scans.
Use CREATE_BREAKAWAY_FROM_JOB when spawning slash workers on Windows so
AssignProcessToJobObject succeeds under Desktop/Electron parent jobs.
Add Windows e2e tests for job teardown and orphan reaping.
supplefrog added a commit to supplefrog/hermes-agent that referenced this pull request Jun 23, 2026
Resume Desktop chats lazily so opening a stored session can paint history before slow agent construction completes. Avoid starting the lazy agent build when staging image attachments; prompt submission will consume the queued attachment when the agent is actually needed.

Also harden the Electron backend shutdown path on Windows by terminating the backend process tree. Slash-worker lifecycle cleanup is intentionally left to NousResearch#48656.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing a real cross-process slash-worker lifecycle gap.

Problems

  • tui_gateway/server.py:262 needs current-main reconciliation. Current main's same Popen call already passes creationflags=windows_hide_flags() at tui_gateway/server.py:323; applying the PR's separate creationflags value would either conflict or discard the existing no-console-window behavior. Compose the flags when salvaging.
  • The PR's startup reaper is useful, but it does not eliminate the startup parent-identity race: current tui_gateway/slash_worker.py:30-31 imports cli before the PPID snapshot/watchdog at :147-152. #64534 contains focused coverage for that remaining window and is complementary.

Suggested changes

  • Retain the startup reaper/zombie-parent work, but integrate the Windows creation flag with windows_hide_flags() and evaluate it alongside #64534's immutable parent-PID handoff.

Automated hermes-sweeper review.

Comment thread tui_gateway/server.py
@@ -256,7 +262,11 @@ def __init__(self, session_key: str, model: str):
bufsize=1,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current main already passes creationflags=windows_hide_flags() in this same Popen call (tui_gateway/server.py:323). Salvage this by composing the breakaway bit with that helper; adding a second creationflags kwarg conflicts, while replacing it loses the existing no-console-window behavior.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tui_gateway.slash_worker processes are not reaped when their parent session is interrupted mid-compaction

3 participants