Skip to content

fix(gateway): handle CREATE_BREAKAWAY_FROM_JOB rejection on Windows /restart - #49043

Closed
deepbluetseng wants to merge 1 commit into
NousResearch:mainfrom
deepbluetseng:fix/restart-win32-scheduled-task-breakaway
Closed

fix(gateway): handle CREATE_BREAKAWAY_FROM_JOB rejection on Windows /restart#49043
deepbluetseng wants to merge 1 commit into
NousResearch:mainfrom
deepbluetseng:fix/restart-win32-scheduled-task-breakaway

Conversation

@deepbluetseng

Copy link
Copy Markdown

Problem

When the Hermes gateway is run via Windows Scheduled Task, running /restart on Telegram causes the gateway to exit cleanly with no auto-restart — the bot goes silent until the watchdog cycle (up to 5 minutes) detects the outage and respawns the gateway.

The root cause is in _launch_detached_restart_command() (gateway/run.py): it spawns a Python watcher subprocess using windows_detach_popen_kwargs(), which includes the CREATE_BREAKAWAY_FROM_JOB (0x01000000) creation flag.

When the gateway runs inside a job object that does not have JOB_OBJECT_LIMIT_BREAKAWAY_OK set — which is the case for Windows Scheduled Tasks — CreateProcess returns ERROR_ACCESS_DENIED (WinError 5). The watcher never starts, the gateway exits, and nothing respawns it until the watchdog kicks in minutes later.

Fix

Wrap the watcher subprocess.Popen in a try/except OSError block. On failure:

  1. Retry using windows_detach_flags_without_breakaway() — which omits CREATE_BREAKAWAY_FROM_JOB but retains DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW.
  2. DETACHED_PROCESS alone is sufficient to survive the parent console closure in the Scheduled Task / pythonw.exe scenario — BREAKAWAY is only needed when the parent is in an Electron/Tauri job object that tears down children on exit.

Changes

  • gateway/run.py_launch_detached_restart_command():
    • Import windows_detach_flags_without_breakaway alongside windows_detach_popen_kwargs
    • Wrap the watcher subprocess.Popen in try/except OSError with a BREAKAWAY-less fallback

Testing

  • Manual verification: on a Windows machine with gateway running via Scheduled Task, /restart on Telegram now successfully restarts the gateway within seconds instead of going silent for 5 minutes. Confirmed by gateway log: the watcher PID appears after the shutdown sequence.

Related

The same pattern (try BREAKAWAY first, fall back without) already exists in hermes_cli/gateway.py and gateway_windows.py:_spawn_detached — this fix brings _launch_detached_restart_command in line with the rest of the codebase.

…restart

When Hermes gateway runs under a Windows Scheduled Task, the process
is inside a job object without JOB_OBJECT_LIMIT_BREAKAWAY_OK set.
CREATE_BREAKAWAY_FROM_JOB (0x01000000) fails with ERROR_ACCESS_DENIED
(WinError 5), and the restart watcher never starts — the bot goes
silent until the watchdog respawns it minutes later.

Fix: wrap the watcher subprocess.Popen in try/except OSError and retry
with windows_detach_flags_without_breakaway() as fallback, matching
the pattern already used in hermes_cli/gateway.py.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jun 19, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #42242 — both wrap the _launch_detached_restart_command() watcher subprocess.Popen in a try/except OSError block and retry with windows_detach_flags_without_breakaway() when the parent job object rejects CREATE_BREAKAWAY_FROM_JOB (WinError 5). Same mechanism at the same call site in gateway/run.py; #42242 is the earlier still-open PR. Note: the merged #40956 added the windows_detach_flags_without_breakaway() helper but did not wire the OSError fallback into this restart-watcher call site, so the bug is still live on main — this fix is legitimate, just a duplicate of the open #42242. Related: #42116 (issue), #42993 (opposite mechanism — adds breakaway).

@deepbluetseng

Copy link
Copy Markdown
Author

Thanks for the review — understood. This PR duplicates #42242, so I'll close it and defer to the earlier PR. Appreciate the confirmation that the underlying bug is still present on main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants