Skip to content

feat(dashboard): reliable Restart Hermes via SIGUSR1 gateway restart - #75845

Open
MistakerXI wants to merge 1 commit into
NousResearch:mainfrom
MistakerXI:feature/restart-hermes
Open

feat(dashboard): reliable Restart Hermes via SIGUSR1 gateway restart#75845
MistakerXI wants to merge 1 commit into
NousResearch:mainfrom
MistakerXI:feature/restart-hermes

Conversation

@MistakerXI

Copy link
Copy Markdown

Problem

The dashboard "Restart Hermes" button was unreliable on systemd installs:

  1. Gateway PID never changed. The endpoint spawned a subprocess that slept 1s then ran systemctl restart hermes-dashboard. systemd kills the entire cgroup on restart, including that child, so the gateway restart never happened.
  2. Bogus "Action failed (exit ?)" toast. The frontend polled for a spawned action record, but the endpoint was fire-and-forget, so it read exit_code: null and showed a red error even though the restart succeeded.

Fix

  • Backend (hermes_cli/web_server.py): POST /api/system/restart-hermes now sends SIGUSR1 directly to the running gateway process (same graceful, drain-aware path hermes gateway restart uses). The gateway is a user service with Restart=always in its own cgroup, so it survives the dashboard restart. Falls back to the old spawn path when no PID file is found or on platforms without SIGUSR1.
  • Frontend (web/src/contexts/SystemActions.tsx): the hermes branch no longer calls setActiveAction/polls — the dashboard reload itself is the real feedback, eliminating the false error toast.
  • Web bundle rebuilt from source.

Evidence

  • Tests: test_web_server.py 100/100 pass
  • E2E on Pi: POST /api/system/restart-hermes → 200 {ok:true, via:sigusr1}, gateway PID 30928→31115, dashboard back ~10s, service active

- POST /api/system/restart-hermes now signals the gateway directly (SIGUSR1)
  instead of spawning a subprocess that systemd cgroup-kills with the dashboard
- Web UI no longer polls for a spawned action record on restart-hermes
  (fire-and-forget endpoint) — dashboard reload is the real feedback, so the
  bogus "Action failed (exit ?)" toast is gone
- Web bundle rebuilt from source so the feature survives `hermes update`

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the dashboard/gateway restart interaction. The current main implementation does not already provide this endpoint, but the service-management path needs revision before this can be safely salvaged.

Problems

  • hermes_cli/web_server.py:4059 probes only system scope and :4113 uses sudo systemctl. Main's existing managed-dashboard logic probes systemctl --user first and preserves that scope (hermes_cli/main.py:7283-7335), so the normal user unit is missed.
  • hermes_cli/web_server.py:4174 signals any live gateway PID. SIGUSR1 requests a service restart (gateway/run.py:25364-25365) and exits with code 75 (gateway/run.py:12372-12393); a manually run gateway has no supervisor guaranteed to relaunch it.
  • web/src/contexts/SystemActions.tsx:88 polls dashboard-restart, but the endpoint creates no action record. The status endpoint reports running=false, exit_code=null for an unrecorded action (hermes_cli/web_server.py:4652-4657).

Suggested changes

  • Route dashboard restart through the existing scope-aware service logic, gate SIGUSR1 on confirmed supervision, and treat dashboard restart as structured fire-and-forget unless it has a real action record.
  • Add focused backend and frontend regression tests for these paths.

Automated hermes-sweeper review.

Comment thread hermes_cli/web_server.py
# appears where the command we would run actually exists.
try:
out = subprocess.run(
["systemctl", "list-unit-files", "hermes-dashboard.service"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probes only the system manager. Current main deliberately probes systemctl --user first and keeps that scope for restart because Hermes installs Linux services in user scope by default (hermes_cli/main.py:7283-7335). Please reuse or extract that scope-aware logic; otherwise the standard hermes-dashboard.service is not detected here.

Comment thread hermes_cli/web_server.py
if not pid:
return None
try:
os.kill(pid, signal.SIGUSR1) # POSIX-only, guarded above

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_running_pid() proves this is a live gateway, not that it has a supervisor. SIGUSR1 sets via_service=True (gateway/run.py:25364-25365) and the gateway exits with code 75 (gateway/run.py:12372-12393), so a manually launched gateway can be stopped without being relaunched. Gate this path on confirmed service supervision and otherwise use the existing restart fallback.

});
return;
}
setActiveAction(action);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This activates the shared action-status poller, but restart_dashboard() does not register _ACTION_PROCS or a successful _ACTION_RESULTS entry. GET /api/actions/dashboard-restart/status therefore returns running: false, exit_code: null (hermes_cli/web_server.py:4652-4657) and produces the same synthetic failure toast if the page remains available. Treat this as fire-and-forget or create a real pollable action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants