fix(gateway): guard stale /restart redelivery when dedup marker is missing - #56107
Merged
Merged
Conversation
…ssing When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue #18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
alt-glitch
added
type/bug
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
Jasper6439
pushed a commit
to Jasper6439/hermes-agent
that referenced
this pull request
Jul 5, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
melon-xf
added a commit
to melon-xf/hermes-agent
that referenced
this pull request
Sep 3, 2026
…ssing (NousResearch#56107) When .restart_last_processed.json goes missing, a redelivered /restart from Telegram polling can no longer be caught by the update_id comparison, so it re-restarts the gateway forever (issue NousResearch#18528, reported by @dontcallmejames who hit it in production — gateway restarting every ~2min, zero messages processed). Fallback: on marker-missing, suppress the /restart only when we can confirm we just came out of a restart cycle (_booted_from_restart, captured at startup from .restart_notify.json before it is unlinked) AND the process is still within a 60s post-boot window. Consumed one-shot. This closes the loop without swallowing a genuine first /restart on a fresh boot — the flaw in the original bare-uptime approach. Credit to @dontcallmejames for the diagnosis and original patch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A missing
.restart_last_processed.jsondedup marker no longer lets a redelivered Telegram/restartloop the gateway forever.Root cause: when the marker goes missing (manual cleanup / filesystem loss / a failed write in the prior cycle), the
update_idcomparison in_is_stale_restart_redelivery()can't run, so a/restartre-delivered by PTB polling (after its graceful-shutdown ACK fails) sails through and re-restarts the gateway on every boot — an infinite loop. Reporter hit this in production: gateway restarting every ~2 min all day, zero messages processed.Changes
gateway/run.py: on the marker-missing path, suppress the/restartonly when we can independently confirm we just came out of a restart cycle —_booted_from_restart(captured at startup from.restart_notify.jsonbefore it's unlinked) and the process is still within a 60s post-boot window. Flag is consumed one-shot.tests/gateway/test_restart_redelivery_dedup.py: +3 tests (redelivery suppressed, fresh-boot honored, late-restart honored).Why not the originally-proposed bare 60s uptime guard: the marker is missing on every fresh install / first-ever restart, so a blanket "uptime < 60s → stale" would silently swallow a legitimate first
/restarta user sends right after boot. Gating on_booted_from_restartcloses the loop without regressing that workflow.Validation
scripts/run_tests.sh tests/gateway/test_restart_redelivery_dedup.py→ 12 passed. Sibling restart/startup suites green.Credit to @dontcallmejames (#18528) for the diagnosis and original patch — the guard was reworked to key on a real restart-cycle signal instead of a bare timer.
Infographic