Skip to content

fix(docker): preserve running gateways across restart - #33619

Closed
LeonSGP43 wants to merge 1 commit into
NousResearch:mainfrom
LeonSGP43:fix-33597-docker-restart-preserve-running-state
Closed

LeonSGP43 wants to merge 1 commit into
NousResearch:mainfrom
LeonSGP43:fix-33597-docker-restart-preserve-running-state

Conversation

@LeonSGP43

Copy link
Copy Markdown

Closes #33597

Summary

  • preserve running in gateway_state.json when the gateway is torn down by an unexpected signal such as container restart
  • write planned-stop markers for s6 stop and restart so explicit user actions still persist stopped
  • cover the new shutdown-state and s6 marker behavior with targeted regression tests

Testing

  • uv run --frozen pytest -o addopts= tests/gateway/test_clean_shutdown_marker.py tests/hermes_cli/test_service_manager.py\n- uv run --frozen ruff check gateway/run.py gateway/status.py hermes_cli/service_manager.py tests/gateway/test_clean_shutdown_marker.py tests/hermes_cli/test_service_manager.py\n- git diff --check

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery area/docker Docker image, Compose, packaging P2 Medium — degraded but workaround exists labels May 28, 2026
@benbarclay

Copy link
Copy Markdown

Superseded by #43236 (merged, 5cf6e28), which implements both of this PR's core changes:

  1. Preserve running on unexpected-signal teardown — _stop_impl in gateway/run.py now persists gateway_state=running (not stopped) when the shutdown was signal-initiated (container/s6 SIGTERM, OOM, bare kill), so container_boot auto-starts on the next boot.
  2. Planned-stop marker on the s6 stop path — S6ServiceManager.stop() now writes the planned-stop marker (resolved via s6-svstat) before s6-svc -d, so an explicit hermes gateway stop is classified as intentional and correctly persists stopped.

#43236 uses the existing planned-stop marker primitive (rather than inferring intent from the signal) and is verified end-to-end against a real docker restart — fails on a pre-fix image (prior_state=stopped action=registered), passes on the fixed one (prior_state=running action=started). Closes the same underlying issue (#42675 / your #33597).

Thanks for the fix here — the approach matched; it just landed via #43236 first. One residual your PR also touched: writing the marker on the s6 restart path (not just stop). I'm tracking that as a follow-up to fold into the restart path.

@benbarclay benbarclay closed this Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docker Docker image, Compose, packaging comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Docker restart persists gateway as stopped, preventing s6 auto-start on next boot

3 participants