fix(gateway): use via_service for macOS restart to avoid KeepAlive race - #37063
fix(gateway): use via_service for macOS restart to avoid KeepAlive race#37063drzeast-png wants to merge 3 commits into
Conversation
|
I found an issue that looks worth fixing before merge.
The change Why it matters: Any macOS user running the gateway from Terminal (not under a launchd plist) would lose their gateway on Suggested fix: Detect actual launchd management instead of blanket-darwin. PR #37094 (same issue, same file) does this correctly by checking _under_launchd = sys.platform == "darwin" and os.environ.get(
"XPC_SERVICE_NAME", "0"
) not in ("", "0")launchd sets |
|
I found an issue with the scope of this fix.
The comment says "Launchd on macOS has KeepAlive=true which restarts the gateway on any exit" — but this is only true for launchd-managed jobs. A bare There's a more precise fix in PR #37094 that checks _under_launchd = sys.platform == "darwin" and os.environ.get(
"XPC_SERVICE_NAME", "0"
) not in ("", "0")This correctly detects launchd-managed processes without affecting bare shell-launched ones. I'd recommend closing this in favor of #37094. |
8e336b2 to
334bd43
Compare
eb19434 to
6a91e1c
Compare
…estarts Allow restart/stop from inside the gateway (e.g. via WeChat/Telegram) so the user can manage the gateway remotely. Trust launchd KeepAlive as the only respawn policy; the upstream blanket guard (NousResearch#35679) blocks legitimate manual restarts. Supersedes the still-open PR NousResearch#35815 (loop-detector variant).
iLink can return ret=-3 (not just ret=-2/errcode=-14) to indicate a stale context_token that needs to be refreshed. Without this fix the adapter retries 5 times with a dead token, then gives up — causing intermittent send failures for cron deliveries and other bot-initiated messages. Also adds debug logging to _api_post for sendmessage responses so future session issues are easier to diagnose.
On macOS launchd, KeepAlive=true restarts the gateway on any exit (same semantics as systemd Restart=always). Previously the in-chat /restart path used detached=True, spawning a shell watcher that polls the old PID and runs 'hermes gateway restart' after exit. This races with launchd's KeepAlive — the watcher starts after KeepAlive has already launched a new gateway, then sends SIGTERM to the new instance, causing a kill loop and eventually unloading the launchd job. Fix: make Darwin match systemd by using via_service=True so the gateway exits with code 75 and launchd natively restarts it. No watcher, no 'hermes gateway restart' CLI, no racing.
6a91e1c to
5766f7b
Compare
austinpickett
left a comment
There was a problem hiding this comment.
✅ Approved
Three complementary fixes in one coherent PR from the same author:
-
via_service=Trueon macOS (gateway/run.py): launchd KeepAlive restarts the gateway on ANY exit (same semantics asRestart=always). Addingsys.platform == "darwin"to the existing_under_service or _in_containergate routes Darwin throughrequest_restart(via_service=True)— clean exit, no detached watcher shell racing with KeepAlive. Correct fix. -
Remove
_HERMES_GATEWAYblanket guard (hermes_cli/gateway.py): Withvia_servicenow used on macOS, the guard is no longer needed there. On Linux/systemd theINVOCATION_IDpath already handles it. The guard was incorrectly blocking legitimate in-chathermes gateway restartfrom messaging platforms. -
WeChat
_is_stale_session_rethardening: ret=-3 is exclusive to stale-session (never rate-limiting), so removing the errmsg gate for it is correct. Explanatory docstring added.
Note: See my comment on #37057 (companion PR) — recommend closing that in favor of this one which is the complete fix.
|
Thanks for the macOS restart investigation. The title fix is now implemented on current main through a more precise launchd probe. Problems
Suggested changes
Automated hermes-sweeper review. |
Problem
On macOS launchd,
KeepAlive=truerestarts the gateway on any exit (same semantics as systemdRestart=always). However, the in-chat/restartpath was hardcoded to usedetached=True, which spawns a shell watcher that:hermes gateway restartThis races with launchd's KeepAlive:
hermes gateway restartSolution
Make Darwin match systemd by using
via_service=Trueso the gateway exits cleanly and launchd restarts it natively. No watcher, nohermes gateway restartCLI, no racing.Diff (1 line, in
gateway/run.py)Behavior comparison
Test plan
/restartfrom WeChat on macOS — gateway restarts cleanly, no kill looplaunchctl list | grep hermesshows new PID after restart