Skip to content

fix: hermes update kills freshly-restarted gateway service - #5409

Closed
kshitijk4poor wants to merge 2 commits into
NousResearch:mainfrom
kshitijk4poor:fix/gateway-restart-kills-service-pid
Closed

fix: hermes update kills freshly-restarted gateway service#5409
kshitijk4poor wants to merge 2 commits into
NousResearch:mainfrom
kshitijk4poor:fix/gateway-restart-kills-service-pid

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Problem

After every hermes update, the gateway service restarts and immediately stops:

✓ Update complete!
✓ Restarted hermes-gateway
→ Stopped 1 manual gateway process(es)
  Restart manually: hermes gateway run

The update path restarts the service via systemd/launchd, then sweeps for stale manual gateway processes using find_gateway_pids() which scans ps aux and returns ALL matching PIDs — including the one just spawned by the service manager. The sweep kills it immediately.

Introduced in 84a875c — the commit correctly added profile-scoped stop, but the find_gateway_pids() sweep in the update path has no service-PID exclusion.

Fix

  • Add _get_service_pids() helper that queries systemd MainPID and launchd PID for active gateway services
  • Add exclude_pids parameter to find_gateway_pids() and kill_gateway_processes()
  • After restarting services in the update path, call _get_service_pids() and pass the result to find_gateway_pids(exclude_pids=...) so freshly-restarted service PIDs are skipped

Tests

9 new tests:

  • TestGetServicePids: systemd MainPID, launchd PID, empty, zero-PID exclusion
  • TestFindGatewayPidsExclude: exclude_pids filtering
  • TestServicePidExclusion: integration tests proving service PID survives after launchd/systemd restart, manual PIDs still killed

Note: TestCmdUpdateLaunchdRestart::test_update_with_systemd_still_restarts_via_systemd was already failing on current main (pre-existing).

After restarting a service-managed gateway (systemd/launchd), the
stale-process sweep calls find_gateway_pids() which returns ALL gateway
PIDs via ps aux — including the one just spawned by the service manager.
The sweep kills it, leaving the user with a stopped gateway and a
confusing 'Restart manually' message.

Fix: add _get_service_pids() to query systemd MainPID and launchd PID
for active gateway services, then exclude those PIDs from the sweep.
Also add exclude_pids parameter to find_gateway_pids() and
kill_gateway_processes() so callers can skip known service-managed PIDs.

Adds 9 targeted tests covering:
- _get_service_pids() for systemd, launchd, empty, and zero-PID cases
- find_gateway_pids() exclude_pids filtering
- cmd_update integration: service PID not killed after restart
- cmd_update integration: manual PID killed while service PID preserved
…f-import, harden launchd parsing

- Loop over user/system scope args instead of duplicating the systemd block
- Call get_launchd_label() directly instead of self-importing from hermes_cli.gateway
- Validate launchd output by checking parts[2] matches expected label (skip header)
- Add race-condition assumption docstring
@teknium1

teknium1 commented Apr 6, 2026

Copy link
Copy Markdown
Contributor

Merged via PR #5448. Your commits were cherry-picked onto current main with your authorship preserved in git log. Thanks for the fix, @kshitijk4poor!

@teknium1 teknium1 closed this Apr 6, 2026
@kshitijk4poor
kshitijk4poor deleted the fix/gateway-restart-kills-service-pid branch August 5, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants