Skip to content

fix(windows): self-heal post-update gateway relaunch via Scheduled Task - #115495

Open
sgustjd2 wants to merge 1 commit into
NousResearch:mainfrom
sgustjd2:fix/windows-update-selfheal-scheduled-task
Open

sgustjd2 wants to merge 1 commit into
NousResearch:mainfrom
sgustjd2:fix/windows-update-selfheal-scheduled-task

Conversation

@sgustjd2

Copy link
Copy Markdown

Problem

On Windows, hermes update leaves the gateway down and exits with:

RuntimeError: Windows gateway relaunch after update was not verified alive

The post-update relaunch (_cold_start_windows_gateway_after_update, _verify_relaunched_gateways_alive) spawns the gateway as a detached child of the updater process. When the updater runs inside a Windows Job Object that denies CREATE_BREAKAWAY_FROM_JOB, that child is reaped on updater teardown (#48820 / #91675). The CLI already detects this and prints the manual recovery schtasks /Run /TN Hermes_Gateway — but the update itself never recovers, so the gateway stays down until a human intervenes.

Reproduced on Windows 11 with the gateway installed as a Scheduled Task (Hermes_Gateway): both the cold-start and the paused-relaunch paths spawn a child that dies within seconds of the updater exiting, on two consecutive update runs.

Fix

Add gateway_windows.run_scheduled_task_and_wait(): run the registered Scheduled Task (which starts the gateway under Task Scheduler — outside any caller Job Object) and wait for liveness. Wire it as a fallback at both relaunch sites, fired only when the normal detached spawn was reaped. This is the same mechanism as the recovery hint the CLI already prints, now applied automatically. Start attestations are labeled (scheduled task) so honest reporting is preserved.

  • No behavior change on the happy path (detached spawn survives) or on non-Windows.
  • Falls back to the existing error + hermes gateway restart hint when no Scheduled Task is registered or schtasks refuses.
  • Targets the current profile's task only; a fully-down multi-profile fleet recovers the active profile here (noted in a comment for future per-home plumbing).

Test

  • Branch coverage for run_scheduled_task_and_wait (unregistered task → []; schtasks refused → []; task ran + gateway up → live pids; task ran + still dead → []).
  • py_compile on both changed modules.
  • Manual: schtasks /Run /TN Hermes_Gateway reliably brings the gateway up on the same box where the detached relaunch is reaped.

🤖 Generated with Claude Code

The post-update relaunch (_cold_start_windows_gateway_after_update and
_verify_relaunched_gateways_alive) spawns the gateway as a detached child of
the updater. When the updater runs inside a Job Object that denies
CREATE_BREAKAWAY_FROM_JOB, that child is reaped on updater teardown
(NousResearch#48820 / NousResearch#91675), so every `hermes update` leaves the gateway down and raises
"Windows gateway relaunch after update was not verified alive".

Add gateway_windows.run_scheduled_task_and_wait(): run the registered Scheduled
Task (which starts the gateway under Task Scheduler, outside any caller Job
Object) and wait for liveness. Use it as a fallback at both relaunch sites, only
when the detached spawn was reaped -- the same `schtasks /Run /TN Hermes_Gateway`
recovery the CLI already prints, now applied automatically. Attestations are
labeled "(scheduled task)" so honest post-update reporting is preserved.

No change on the happy path or on non-Windows; falls back to the existing error
and `hermes gateway restart` hint when no task is registered or schtasks refuses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery platform/windows area/install-update Installer, updater, packaging, wheels, doctor sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Sep 19, 2026
@ehz0ah

ehz0ah commented Sep 21, 2026

Copy link
Copy Markdown

I found three blocking correctness issues on head 9db0a09af6d585b2de7e6cabe7caa45f832acf39.

  1. P1: Multi-profile recovery can report success while sibling profiles remain down. update_cmd_windows.py:1210 calls run_scheduled_task_and_wait(all_profiles=True), but the helper runs only get_task_name() for the current profile. all_profiles=True only widens the PID poll. One active-profile gateway can therefore make recovery appear successful for the full paused fleet. The helper must relaunch and verify each paused profile by explicit task/profile identity.

  2. P1: The reported post-exit Job Object failure is still not handled. The Scheduled Task fallback runs only when the in-process readiness poll returns no PID. A non-breakaway child can remain alive during that poll and then die when the updater exits. In that path, the fallback is skipped and the update reports success for a gateway that will be terminated during updater teardown. Recovery must use a launch path whose survival is established independently of the updater process.

  3. P1: Existing cross-platform tests regress. On current upstream/main (afc3b7c6f397c6d21fd2c129ca17b18b544dd8dc), the focused suite passes: 42 passed, 0 failed, 5 skipped. With this PR composed onto that main, it reports 39 passed, 3 failed, 5 skipped. The exact PR head reports 29 passed, 3 failed, 4 skipped. All three failures raise RuntimeError: gateway_windows is Windows-only because mocked readiness failures now reach get_task_name() on macOS. Please preserve the existing cross-platform test seam and add focused tests for the new fallback behavior.

Additional evidence: ruff and git diff --check pass. GitHub reports no hosted checks for this head. I did not run a native Windows runner, so this review does not claim Windows verification.

PR #84409 overlaps this same Scheduled Task recovery path. Its current implementation snapshots existing PIDs before /Run and accepts explicit task/profile identity. Please reconcile with that active carrier to avoid duplicate or conflicting fixes. This helper should also reject pre-existing gateway PIDs as proof that /Run launched a new durable process.

sgustjd2 added a commit to sgustjd2/aztomz that referenced this pull request Sep 23, 2026
- 환경 메모: 모델 gemini-2.5-pro → claude-sonnet-5 (Anthropic Claude Code 구독 OAuth,
  effort medium, 2026-09-23) 갱신. Gemini 구성은 이력으로 보존.
- hermes-update.ps1 래퍼(Windows 업데이트 2버그 우회)·Hermes_Gateway_Watchdog
  스케줄작업 문서화. 업스트림 PR NousResearch/hermes-agent#115495 참조.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alt-glitch alt-glitch added os/windows Native Windows behaviour or breakage: PowerShell install, Windows paths, cmd.exe, WSL-vs-native diff and removed platform/windows labels Oct 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install-update Installer, updater, packaging, wheels, doctor comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery os/windows Native Windows behaviour or breakage: PowerShell install, Windows paths, cmd.exe, WSL-vs-native diff P1 High — major feature broken, no workaround sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants