Skip to content

fix(cli): add missing subprocess.run() timeouts in gateway - #3732

Closed
dlkakbs wants to merge 1 commit into
NousResearch:mainfrom
dlkakbs:fix/gateway-subprocess-timeouts
Closed

fix(cli): add missing subprocess.run() timeouts in gateway#3732
dlkakbs wants to merge 1 commit into
NousResearch:mainfrom
dlkakbs:fix/gateway-subprocess-timeouts

Conversation

@dlkakbs

@dlkakbs dlkakbs commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Summary

All subprocess.run() calls in hermes_cli/gateway.py lacked a timeout parameter. If systemctl, launchctl, loginctl, wmic, or ps blocks — e.g. D-Bus socket unavailable, WMI service stuck on Windows, or launchd momentarily unresponsive — hermes gateway start/stop/restart/status/install/uninstall hangs indefinitely with no feedback or escape other than Ctrl+C.

Same class of bug as #3469 (context_references subprocess calls) and #3693 doctor/status subprocess calls). gateway.py was not covered by either.

Root cause

subprocess.run() defaults to timeout=None — waits forever. The rest of the CLI already uses timeouts consistently (clipboard.py, banner.py, doctor.py, status.py). gateway.py was the remaining gap.

Fix

Timeout values applied across all 32 call sites:

Call type timeout
Lifecycle: start, stop, restart, enable, disable, daemon-reload, launchctl load/unload 30s
Query: is-active, systemctl status, launchctl list, loginctl show-user, journalctl, tail, ps aux, wmic 5–10s
loginctl enable-linger 10s

For _is_service_running() and launchd_status(), TimeoutExpired is caught explicitly and treated as not-running — consistent with how non-zero return codes are already handled in those functions.

All other call sites are either already inside try/except Exception blocks (find_gateway_pids, get_systemd_linger_status, _enable_systemd_linger) or propagate TimeoutExpired as a clear exception instead of hanging forever.

Verification

AST-based check confirms zero remaining subprocess.run() calls without
timeout= in gateway.py:

python3 -c "
import ast, sys
source = open('hermes_cli/gateway.py').read()
tree = ast.parse(source)
missing = [n.lineno for n in ast.walk(tree)
           if isinstance(n, ast.Call)
           and isinstance(n.func, ast.Attribute)
           and n.func.attr == 'run'
           and isinstance(n.func.value, ast.Name)
           and n.func.value.id == 'subprocess'
           and not any(kw.arg == 'timeout' for kw in n.keywords)]
assert not missing, f'missing timeout at lines: {missing}'
print('ok')
"

All subprocess.run() calls in hermes_cli/gateway.py lacked a timeout
parameter. If systemctl, launchctl, loginctl, wmic, or ps blocks
(e.g. D-Bus unavailable, WMI service stuck, launchd unresponsive),
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback to the user.

Timeout values applied:
- Lifecycle commands (start/stop/restart/enable/disable/daemon-reload,
  launchctl load/unload): timeout=30
- Status/query commands (is-active, loginctl show-user, launchctl list,
  systemctl status, journalctl, tail, ps aux, wmic): timeout=5-10
- loginctl enable-linger: timeout=10

For _is_service_running() and launchd_status(), TimeoutExpired is caught
explicitly and treated as not-running, matching how non-zero return codes
are already handled. All other call sites are either inside existing
try/except Exception blocks (find_gateway_pids, _enable_systemd_linger,
get_systemd_linger_status) or raise TimeoutExpired as a clear error
instead of hanging forever.

Same class of fix as NousResearch#3469 (context_references) and NousResearch#3693 (doctor/status).
teknium1 added a commit that referenced this pull request Apr 6, 2026
All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR #3732 (dlkakbs) and issue #4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
teknium1 added a commit that referenced this pull request Apr 6, 2026
All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR #3732 (dlkakbs) and issue #4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
@teknium1

teknium1 commented Apr 6, 2026

Copy link
Copy Markdown
Contributor

Merged via #5424. Your approach — tiered timeouts across all subprocess.run() calls in gateway.py — was the right one. We reimplemented on current main since the launchctl section was rewritten (bootout/bootstrap/kickstart replaced load/unload/start/stop), and added a third tier (90s) for stop/restart commands to respect our TimeoutStopSec=60 in the systemd unit. Thanks for the contribution, @dlkakbs!

@teknium1 teknium1 closed this Apr 6, 2026
Tommyeds pushed a commit to Tommyeds/hermes-agent that referenced this pull request Apr 12, 2026
…esearch#5424)

All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR NousResearch#3732 (dlkakbs) and issue NousResearch#4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…esearch#5424)

All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR NousResearch#3732 (dlkakbs) and issue NousResearch#4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 28, 2026
All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR NousResearch#3732 (dlkakbs) and issue NousResearch#4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…esearch#5424)

All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR NousResearch#3732 (dlkakbs) and issue NousResearch#4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…esearch#5424)

All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR NousResearch#3732 (dlkakbs) and issue NousResearch#4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…esearch#5424)

All 35 subprocess.run() calls in hermes_cli/gateway.py lacked timeout
parameters. If systemctl, launchctl, loginctl, wmic, or ps blocks,
hermes gateway start/stop/restart/status/install/uninstall hangs
indefinitely with no feedback.

Timeouts tiered by operation type:
- 10s: instant queries (is-active, status, list, ps, tail, journalctl)
- 30s: fast lifecycle (daemon-reload, enable, start, bootstrap, kickstart)
- 90s: graceful shutdown (stop, restart, bootout, kickstart -k) — exceeds
  our TimeoutStopSec=60 to avoid premature timeout during shutdown

Special handling: _is_service_running() and launchd_status() catch
TimeoutExpired and treat it as not-running/not-loaded, consistent with
how non-zero return codes are already handled.

Inspired by PR NousResearch#3732 (dlkakbs) and issue NousResearch#4057 (SHL0MS).
Reimplemented on current main which has significantly changed launchctl
handling (bootout/bootstrap/kickstart vs legacy load/unload/start/stop).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants