fix(update): escape parent job via schtasks for Windows gateway post-update spawn

[Halldrix]

What does this PR do?

Escape the parent job object for the post-update Windows gateway spawn by routing through the Task Scheduler when the gateway's Scheduled Task exists.

This is the second half of the fix for #84185 — the companion to #84212 (which addresses only the silent/false success report). Where #84212 makes the updater honest ("did the process actually survive?"), this PR makes the spawn actually succeed in the first place by leaving the parent job object via schtasks /Run /tn <Hermes_Gateway>.

Related Issue

Refs: #84185
Refs: #84212

This PR does not close #84185 on its own — #84212 fixes the silent lie, this PR fixes the silent death.

Evidence (validated against a real Windows 11 host)

Spawn method	Flags	Survives parent-job teardown?
subprocess.Popen	+CREATE_BREAKAWAY_FROM_JOB	No — CreateProcess succeeds, no OSError, but the child is silently kept inside the job and killed with it
subprocess.Popen	fallback without breakaway	No (control)
schtasks /Run /tn <task>	via Task Scheduler	Yes — the scheduler runs the gateway outside any job containing the updater

Crucially, breakaway_error=None on the test host: no failure signal, no detection path. The current ✓ is doubly invisible.

Type of Change

• Bug fix (non-breaking change which fixes an issue)

Changes Made

• hermes_cli/gateway_windows.py — new helper _spawn_via_scheduled_task(timeout_s=6.0): registers + runs the gateway's own Scheduled Task via schtasks /Run, then polls _wait_for_gateway_ready() until a new gateway process appears. Before triggering, the task scripts are regenerated (_write_task_script() + _install_scheduled_task() delete+create) so the spawn never replays a stale Python path from task-creation time. The post-trigger poll snapshots existing gateway PIDs (pre_pids) and only returns True when a genuinely new PID appears (set(ready) - pre_pids) — a pre-update gateway still draining does not satisfy the check. Returns False when no task is registered, script write fails, registration fails, the trigger failed, or no new process showed up in time.
• hermes_cli/update_cmd.py — _cold_start_windows_gateway_after_update(): (1) prefer _spawn_via_scheduled_task() and print ✓ ... (via Scheduled Task); (2) fall back to _spawn_detached() when no task exists; (3) gate the fallback ✓ behind _wait_for_gateway_ready() and print an explicit ✗ ... did not survive + hermes gateway start recovery hint when the spawned process never comes up.
• hermes_cli/gateway.py — _spawn_gateway_restart_watcher(): inject the same Scheduled-Task escape into the inline watcher script so unmapped gateways (Scheduled Task / vbs / manually-launched gateway run) also respawn through the Task Scheduler when registered. The watcher refreshes task scripts before triggering (_write_task_script() + _install_scheduled_task()) and snapshots pre-existing PIDs before the poll (_pre_pids), counting only new processes (set(_fgp()) - _pre_pids). Falls back to the current subprocess.Popen(breakaway) → Popen(no breakaway) chain only when the task route fails — covers the "no spawn at all" variant of [Bug]: Windows: gateway cold-started after 'hermes update' dies silently (no logs, no PID file, no exit record) — offline until manual restart #84185 (see issue thread).
• tests/hermes_cli/test_update_gateway_schtasks_escape.py — 11 new tests covering: (a) schtasks preferred + ✓ when task exists; (b) fallback to direct spawn when no task; (c) survival-check reporting (✓ vs ✗ + recovery hint); (d) helper's full contract: no task registered, schtasks /Run failure, no new PID appears, only pre-existing gateway (must NOT pass), script write failure, task registration failure, task triggered + new PID (must pass); (e) watcher source-level assertions confirming task refresh + PID snapshot are present.
• tests/hermes_cli/test_update_cold_start_gateway_liveness.py — 2 existing tests updated to assert the new output strings (✓ Starting Windows gateway after update / ✗ ... did not survive) and mock _spawn_via_scheduled_task to force the fallback path.

How to Test

scripts/run_tests.sh tests/hermes_cli/test_update_gateway_schtasks_escape.py -v
scripts/run_tests.sh tests/hermes_cli/test_update_cold_start_gateway_liveness.py -v
# plus the neighboring update-path suites:
scripts/run_tests.sh tests/hermes_cli/test_gateway_windows.py tests/hermes_cli/test_update_venv_health.py tests/hermes_cli/test_update_orphan_backend_reap.py

Sabotage-run: with the fix stashed, all 11 new tests fail (error/failure); with the fix restored they all pass. The 2 modified liveness tests also fail without the updated assertions. Verified locally before opening.

Real-Windows validation: the job-object escape was reproduced end-to-end on a real Windows 11 host via a minimal two-process harness (parent creates a KILL_ON_JOB_CLOSE job, spawns the "updater" inside it, closes the handle). subprocess.Popen+breakaway → child killed; schtasks /Run → child survives. See issue #84185 thread for the full evidence table.

Review follow-up (review by @Enough1122): Points 2 and 3 were implemented after review — see the reply comment for the full rationale. Point 1 (deduplication of schtasks logic between watcher and helper) is deferred to a follow-up issue due to the watcher's embedded python -c constraint. Point 4 (fallback port race) is a known limitation with no demonstrated repro.

Checklist

• I have read the Contributing Guide (CONTRIBUTING.md)
• My commit messages follow Conventional Commits (fix(update): escape parent job via schtasks for Windows gateway post-update spawn)
• I have searched for existing PRs and confirmed none cover this fix (fix(update): verify Windows gateway cold-start survives before reporting success #84212 is complementary and explicitly out-of-scope on the job-object escape)
• I have tested this on a fresh install or via the test suite (sabotage-run verified: tests fail without the fix, pass with it)
• I have added tests for the new behavior
• N/A — No docs update needed (no new user-facing config / behavior change)
• N/A — No changelog entry (bug fix)

---

🛠️ Dev: Halldrix
🤖 Sidekick: Hermes Agent v0.20.0
🔍 Discipline: Test-first (tests/hermes_cli/test_update_gateway_schtasks_escape.py), sabotage-run verified (11/11 red without fix), real-Windows prototype validated on hardware affected by #84185, review-bot points 2-3 implemented post-review

[linfeng961]

Reporter of #84185 here. This PR is exactly the follow-up @teknium1 asked for when closing #84185 — the honest-reporting fix (#86687) makes the silent death visible, but the gateway still dies after every update until a manual hermes gateway start. The schtasks escape is the missing half.

I reproduced the cold-start death 4x on my Windows 10 + Scheduled Task + Telegram setup. Happy to test this PR against my Scheduled Task environment if it helps the review. Requesting maintainer attention so this doesn't sit unmerged: @teknium1

[Enough1122]

AI code review — automated review for reference, author can ignore or act on any point.

PR: fix(update): escape parent job via schtasks for Windows gateway post-update spawn

1. Duplicate schtasks implementations — hermes_cli/gateway.py::_spawn_gateway_restart_watcher inlines its own schtasks /Run + 6s poll, while hermes_cli/gateway_windows.py::_spawn_via_scheduled_task implements the same logic (via _exec_schtasks + _wait_for_gateway_ready). Two copies of the same escape mechanism will drift (timeouts, error handling). Reusing the helper from both spawn points keeps the behavior — and future fixes — in one place.
2. find_gateway_pids() matches any gateway, not the one the task spawned — in the watcher, _started_via_task becomes True if a gateway pid appears within 6s. If a different gateway instance is already running (e.g. the pre-update process still draining), the direct spawn of run_argv is skipped and the update's new command never launches — the respawn is silently lost. The check confirms "a gateway is up", not "the NEW gateway is up".
3. The Scheduled Task runs a possibly stale command — the task executes whatever command was registered at task-creation time, which may not match the update's run_argv (new install path, _respawn_env_overlay env). When a task is registered, the direct spawn of run_argv is skipped entirely; if the task's command is stale, the update never takes effect. Consider only preferring the task path when the task command matches run_argv (or verifying the launched process is the new one).
4. Fallback race on slow task start — schtasks /Run returns before the gateway is up; if startup exceeds the 6s poll, the watcher falls back to a direct spawn and two gateways may race for the same port. A longer poll or a "task trigger acknowledged" return (rather than process-visible) would narrow the window.
5. The honest-survival reporting is a real improvement — turning the "spawned but died" case into an explicit ✗ with hermes gateway start guidance beats the previous false ✓.

[Halldrix]

@Enough1122 thanks for the detailed review. We've evaluated each point:

Points 2 and 3 are now implemented — they are the core of the #84185 bug class, not speculative hardening:

Point 2 (PID matching): We now snapshot the set of existing gateway PIDs before triggering the Scheduled Task, and the post-trigger poll only counts PIDs that did not exist before (set(ready) - pre_pids). A pre-update gateway that is still draining no longer satisfies the check on its own — only a genuinely new process spawned by the task does. This is applied in both _spawn_via_scheduled_task() (cold-start path) and the embedded watcher script (_spawn_gateway_restart_watcher).

Point 3 (stale task command): The Scheduled Task scripts are now regenerated and re-registered before every /Run trigger. _write_task_script() rebuilds gateway.cmd + gateway.vbs with the current get_python_path(), HERMES_HOME, and profile arg, and _install_scheduled_task() atomically deletes + re-creates the task — identical to a fresh install procedure. The spawn never replays a stale Python path from task-creation time.

Point 1 is legitimate debt, but the watcher runs as an embedded python -c string in a detached subprocess — it cannot import _spawn_via_scheduled_task() directly without serializing the helper into the watcher argv. Refactoring this requires a larger restructuring of how the watcher is spawned; we'd rather keep this PR focused and open a follow-up issue for the deduplication.

Point 4 is a theoretical race (task takes >6s and the fallback direct spawn competes for the port). The adaptive-poll fix would introduce an arbitrary long wait in the happy path. We're marking this as a known limitation — if anyone can demonstrate the contention in a reproducible environment, we'll treat it as a prioritized follow-up.

Point 5 confirmed, no action needed.