Skip to content

fix(gateway): exclude service-managed PIDs from orphan reaping - #85743

Closed
JonthanaHanh wants to merge 1 commit into
NousResearch:mainfrom
JonthanaHanh:fix/reap-orphan-exclude-service-pids
Closed

JonthanaHanh wants to merge 1 commit into
NousResearch:mainfrom
JonthanaHanh:fix/reap-orphan-exclude-service-pids

Conversation

@JonthanaHanh

Copy link
Copy Markdown
Contributor

Summary

_reap_unsupervised_gateway_orphans() kills every gateway PID found by find_gateway_pids() on hosts without systemd (macOS launchd, Windows Scheduled Task). This includes service-managed gateways that are NOT orphans — they are supervised by launchd/systemd and should never be killed during a stale-process sweep.

The function's own docstring says exclude_pids is for "service-managed PIDs that should not be killed during a stale-process sweep", but it only excludes its own PID — not service-managed ones.

Root Cause

_reap_unsupervised_gateway_orphans() calls find_gateway_pids(exclude_pids=own) where own only contains {os.getpid()} plus extra_exclude. find_gateway_pids() includes _get_service_pids() in its results (line 634), but those service-managed PIDs are NOT in the exclusion set — so they get reaped.

Fix

Add own |= _get_service_pids() to the exclusion set before scanning, wrapped in try/except for robustness. This preserves the #77276 orphan protection while excluding launchd/systemd-supervised gateways.

True orphans (reparented leftovers not present in launchctl list / systemctl) are still found and reaped.

Changes

  • hermes_cli/gateway.py: Add 5 lines in _reap_unsupervised_gateway_orphans() to exclude _get_service_pids() from the reap target set.

Test Plan

  • Syntax check passes
  • Module imports correctly
  • On macOS with launchd gateway: verify desktop serve startup no longer kills the gateway
  • On Windows with Scheduled Task gateway: verify desktop serve startup no longer kills the gateway
  • Verify true orphans (no service supervisor) are still reaped

Fixes #85344
Fixes #85044
Fixes #84855
Fixes #85368

_reap_unsupervised_gateway_orphans() kills every gateway PID found by
find_gateway_pids() on hosts without systemd (macOS launchd, Windows
Scheduled Task). This includes service-managed gateways that are NOT
orphans — they are supervised by launchd/systemd and should never be
killed during a stale-process sweep.

Add own |= _get_service_pids() to the exclusion set before scanning,
so launchd/systemd-supervised gateways are preserved. True orphans
(reparented leftovers not present in launchctl/systemctl) are still
found and reaped, preserving the NousResearch#77276 protection.

Fixes NousResearch#85344 (macOS launchd gateway killed by desktop serve startup)
Fixes NousResearch#85044 (Windows Scheduled Task gateway killed by desktop serve)
Fixes NousResearch#84855 (Permission denied to kill orphaned gateway PID)
Fixes NousResearch#85368 (gateway process repeatedly killed, messaging offline)
@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery comp/desktop Electron desktop app (apps/desktop/*) platform/windows Native Windows-specific behavior or breakage P1 High — major feature broken, no workaround sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows labels Aug 14, 2026
teknium1 added a commit that referenced this pull request Aug 15, 2026
…r to all platforms

Compose the service-PID exclusion (#85743, RelaxJonh) and the recorded-PID +
parent-chain exemption (#86100, arccat-114) into one cross-platform rule:

- _get_service_pids() exclusion now runs unconditionally, not only under
  is_macos() — it is the authoritative "supervised" signal for launchd and
  any systemd unit visible on a host that got past the systemd gate.
- The recorded-healthy-gateway (get_running_pid()) + parent-chain exemption
  now runs on every platform, not only Windows. A recorded, liveness-verified
  gateway is by definition not an orphan "the pidfile/runtime record can't
  see", so the reaper must never target it — this covers Windows Scheduled
  Task / Startup VBS supervision, standalone launcher-started gateways
  (the case #85743 alone would miss), and macOS/WSL equivalents.

True orphans (no service registration, no valid runtime record) are still
found and reaped, preserving the #51325/#75936 duplicate-port protection.

Existing macOS regression tests updated to pin get_running_pid to None for
their scenario; Windows regression tests from #86100 carry over unchanged.

Bug class: #83683 (root), #86287, #86098, #85738, #85368, #85344, #85044,
#84855, #84824, #84200.
@teknium1

Copy link
Copy Markdown
Collaborator

Thanks @JonthanaHanh — the service-managed PID exclusion you built here was the right instinct, and the recorded-registry angle in your approach informed the final design. This landed via #86658, which consolidated the supervised-gateway detection across all three platforms (systemd, Windows Scheduled Task, macOS launchd) plus the recorded-healthy guard, so one fix covers every reap path. Credit noted in the salvage PR body. Closing as superseded by #86658 (merged, 169ff2d).

@teknium1 teknium1 closed this Aug 15, 2026
bobaba76 pushed a commit to bobaba76/hermes-agent that referenced this pull request Aug 27, 2026
…r to all platforms

Compose the service-PID exclusion (NousResearch#85743, RelaxJonh) and the recorded-PID +
parent-chain exemption (NousResearch#86100, arccat-114) into one cross-platform rule:

- _get_service_pids() exclusion now runs unconditionally, not only under
  is_macos() — it is the authoritative "supervised" signal for launchd and
  any systemd unit visible on a host that got past the systemd gate.
- The recorded-healthy-gateway (get_running_pid()) + parent-chain exemption
  now runs on every platform, not only Windows. A recorded, liveness-verified
  gateway is by definition not an orphan "the pidfile/runtime record can't
  see", so the reaper must never target it — this covers Windows Scheduled
  Task / Startup VBS supervision, standalone launcher-started gateways
  (the case NousResearch#85743 alone would miss), and macOS/WSL equivalents.

True orphans (no service registration, no valid runtime record) are still
found and reaped, preserving the NousResearch#51325/NousResearch#75936 duplicate-port protection.

Existing macOS regression tests updated to pin get_running_pid to None for
their scenario; Windows regression tests from NousResearch#86100 carry over unchanged.

Bug class: NousResearch#83683 (root), NousResearch#86287, NousResearch#86098, NousResearch#85738, NousResearch#85368, NousResearch#85344, NousResearch#85044,
NousResearch#84855, NousResearch#84824, NousResearch#84200.
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
…r to all platforms

Compose the service-PID exclusion (NousResearch#85743, RelaxJonh) and the recorded-PID +
parent-chain exemption (NousResearch#86100, arccat-114) into one cross-platform rule:

- _get_service_pids() exclusion now runs unconditionally, not only under
  is_macos() — it is the authoritative "supervised" signal for launchd and
  any systemd unit visible on a host that got past the systemd gate.
- The recorded-healthy-gateway (get_running_pid()) + parent-chain exemption
  now runs on every platform, not only Windows. A recorded, liveness-verified
  gateway is by definition not an orphan "the pidfile/runtime record can't
  see", so the reaper must never target it — this covers Windows Scheduled
  Task / Startup VBS supervision, standalone launcher-started gateways
  (the case NousResearch#85743 alone would miss), and macOS/WSL equivalents.

True orphans (no service registration, no valid runtime record) are still
found and reaped, preserving the NousResearch#51325/NousResearch#75936 duplicate-port protection.

Existing macOS regression tests updated to pin get_running_pid to None for
their scenario; Windows regression tests from NousResearch#86100 carry over unchanged.

Bug class: NousResearch#83683 (root), NousResearch#86287, NousResearch#86098, NousResearch#85738, NousResearch#85368, NousResearch#85344, NousResearch#85044,
NousResearch#84855, NousResearch#84824, NousResearch#84200.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/desktop Electron desktop app (apps/desktop/*) comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround platform/windows Native Windows-specific behavior or breakage sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

4 participants