fix(gateway): recover resume_pending sessions instead of sending a blank turn - #56262
Merged
Conversation
alt-glitch
added
type/bug
Collaborator
teknium1
force-pushed
the
hermes/hermes-5363b8db
branch
3 times, most recently
from
July 1, 2026 10:51
0278dbc to
9b1375a
Compare
…ank turn A session interrupted by a gateway restart is flagged resume_pending and auto-continued on startup via _schedule_resume_pending_sessions(), which dispatches an empty-text internal MessageEvent. The recovery system note that should fill that empty turn is gated, in _run_agent(), on _interruption_is_fresh — the age of the LAST PERSISTED TRANSCRIPT ROW. For an active thread returned to after >1h of silence, that transcript clock is stale even though the interruption (last_resume_marked_at) is seconds old. The gate evaluates False, the note is not prepended, and the model receives a genuinely blank user turn — replying with confused 'that message came through blank' noise. Fix (two parts, both default-on, behavior unchanged for healthy turns): 1. resume_pending freshness now also considers last_resume_marked_at (the restart watchdog's own stamp). The branch fires when EITHER the transcript clock OR the resume mark is fresh, so the startup scheduler's freshness decision and the per-turn injection agree. 2. Empty-turn safety net: if the user turn is still blank after all injections AND the session is resume_pending, backfill a recovery note so a blank turn can never reach the model. Scoped to resume_pending so ordinary empty turns (e.g. uncaptioned image) are untouched. Adds 3 regression tests; the two core ones fail on the pre-fix logic.
…rding Follow-up on the salvaged resume_pending fix: the empty-turn safety net now emits the same reason-aware recovery note as the _is_resume_pending branch (reason phrase + 'session restored' guidance + no-re-execute instruction) instead of a second, differently-worded note. Also adds the AUTHOR_MAP entry for the salvaged commit.
teknium1
force-pushed
the
hermes/hermes-5363b8db
branch
from
July 1, 2026 10:52
9b1375a to
979a343
Compare
teknium1
enabled auto-merge (rebase)
July 1, 2026 10:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
After a gateway restart, an auto-resumed thread now always gets its recovery system note instead of occasionally receiving a genuinely blank user turn (which made the model reply with confused "that message came through blank" noise).
Root cause:
_schedule_resume_pending_sessionsdispatches an empty-text turn expecting the injector to fill it with a recovery note, but the scheduler judges freshness offlast_resume_marked_at(interrupt time) while the injector gates_is_resume_pendingon the transcript clock (_last_transcript_timestamp). For a thread quiet >1h before the crash, those disagree →_is_resume_pendinggoes False → the note is never injected → the model gets a blank user turn.Changes
gateway/run.py: (1) dual freshness signal —_is_resume_pendingnow fires when the transcript clock orlast_resume_marked_atis within the window, so the scheduler's decision and the per-turn injection agree. (2) empty-turn safety net — ifmessageis still blank AND the session isresume_pending, backfill the recovery note; scoped toresume_pendingso a legitimately empty user turn (uncaptioned image) on a normal session is untouched. The safety-net note reuses the canonical reason-aware recovery wording from the_is_resume_pendingbranch (follow-up on the salvage).scripts/release.py: AUTHOR_MAP entry for the salvaged commit.Validation
_is_resume_pending=False→ blank turnresume_pending(branch missed)tests/gateway/test_restart_resume_pending.pyE2E-verified against the real
_is_fresh_gateway_interruption/_last_transcript_timestamphelpers: the exact stale-transcript bug case flips from False (old single-signal) to True (dual signal).Salvaged from #46997 by @abchiaravalle (first-time contributor); authorship preserved via rebase-merge.
Infographic
Nous Research