fix: salvage 4 stream/loop stability fixes (#7901, #7693, #7738, #7747) - #7910
Merged
Conversation
When API routers rewrite finish_reason from "length" to "tool_calls", truncated JSON arguments bypassed the length handler and wasted 3 retry attempts in the generic JSON validation loop. Now detects truncation patterns in tool call arguments regardless of finish_reason. Fixes #7680 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Models that do not use <think> tags (e.g. GLM-4.7 on NVIDIA Build, minimax) may return content=None or empty string when truncated. The previous _thinking_exhausted check treated any None/empty content as thinking-budget exhaustion, causing these models to always show the 'Thinking Budget Exhausted' error instead of attempting continuation. Fix: gate the exhaustion check on _has_think_tags — only trigger the exhaustion path when the model actually produced reasoning blocks (<think>, <thinking>, <reasoning>, <REASONING_SCRATCHPAD>). Models without think tags now fall through to the normal continuation retry logic (up to 3 attempts). Fixes #7729
Cherry-picked from PR #7747 with follow-up fixes: - Narrowed suspend_all_active() to suspend_recently_active() — only suspends sessions updated within the last 2 minutes (likely in-flight), not all sessions which would unnecessarily reset idle users - /stop with no running agent no longer suspends the session; only actual force-stops mark the session for reset
The test expected content=None to immediately trigger thinking-exhaustion, but PR #7738 correctly gates that check on _has_think_tags. Without think tags, the agent falls through to normal continuation retry (3 attempts).
This was referenced Apr 11, 2026
Codename-11
added a commit
to Codename-11/hermes-agent
that referenced
this pull request
Apr 12, 2026
The /api/sessions/{id}/chat/stream SSE loop lacked the periodic
keepalive heartbeat that the /v1/chat/completions endpoint already
has (added in NousResearch#7910). During agent initialization and long LLM API
calls (especially with extended thinking), the stream goes silent
for 15-60+ seconds. Clients and intermediary proxies interpret the
silence as a stalled connection and drop it, resulting in 0-byte
responses and the user seeing typing indicators but no reply.
Adds the same keepalive pattern: track last_activity with
time.monotonic(), emit SSE comment keepalives every 30 seconds
(CHAT_COMPLETIONS_SSE_KEEPALIVE_SECONDS), and reset the timer on
each data frame write.
Codename-11
added a commit
to Codename-11/hermes-agent
that referenced
this pull request
Apr 13, 2026
The /api/sessions/{id}/chat/stream SSE loop lacked the periodic
keepalive heartbeat that the /v1/chat/completions endpoint already
has (added in NousResearch#7910). During agent initialization and long LLM API
calls (especially with extended thinking), the stream goes silent
for 15-60+ seconds. Clients and intermediary proxies interpret the
silence as a stalled connection and drop it, resulting in 0-byte
responses and the user seeing typing indicators but no reply.
Adds the same keepalive pattern: track last_activity with
time.monotonic(), emit SSE comment keepalives every 30 seconds
(CHAT_COMPLETIONS_SSE_KEEPALIVE_SECONDS), and reset the timer on
each data frame write.
luigileap
added a commit
to luigileap/hermes-agent
that referenced
this pull request
Apr 16, 2026
…mpressor The `_make_compressor` helper uses `ContextCompressor.__new__` to bypass `__init__`, so every attribute referenced by the method under test must be set explicitly. PR NousResearch#7910 (fix(agent): route compression aux through live session runtime) added a `main_runtime` dict inside `_generate_summary` that reads `self.model`, `self.provider`, `self.base_url`, `self.api_key`, and `self.api_mode`. The focus-topic regression test bypassed `__init__`, so those attributes were missing and `_generate_summary` raised `AttributeError: 'ContextCompressor' object has no attribute 'model'`, making the test fail with `result is None` (the method returned None via its broad-exception cooldown path) and `KeyError: 'messages'` on the captured-prompt assertion. Initialise the attributes in `_make_compressor` so the test exercises the real code path. Same pattern as the prior CLI-interrupt fix that added missing attributes for a `SomeClass.__new__(SomeClass)` stub.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvages four PRs that fix agent stream/loop stability issues — stopping mid-stream, false early termination, truncated tool calls, and stuck session resume loops.
Cherry-picked PRs
PR #7901 — fix(api-server): keep chat-completions SSE alive during long tool runs (helix4u)
/v1/chat/completionsSSE stream went silent during tool work, causing frontends to think the run diedX-Accel-Buffering: noheader, matching the existing pattern on the/v1/runsendpointPR #7693 — fix: detect truncated tool_calls when finish_reason is not length (tomqiaozc)
finish_reasonfrom"length"to"tool_calls", truncated JSON args bypassed the length handler and wasted 3 retries before giving up}or]) to short-circuit early, mirroring the streaming pathPR #7738 — fix(agent): prevent false thinking-exhaustion for non-reasoning models (ygd58)
_thinking_exhaustedfired for any model returningcontent=Noneon truncation, not just reasoning models_has_think_tagsregexPR #7747 — fix(gateway): break stuck session resume loops on restart (konsisumer)
suspendedflag on SessionEntry,/stopmarks session suspended, gateway startup suspends recently-active sessionsFollow-up fixes applied during salvage
suspend_all_active()→suspend_recently_active()— only suspends sessions updated within the last 2 minutes (likely in-flight), not all sessions which would unnecessarily reset idle users/stopwith no running agent no longer suspends the session; preserves original "No active task to stop." behaviortest_length_empty_content_detected_as_thinking_exhaustedto expect continuation retry behavior (3 attempts) instead of immediate thinking-exhaustion, matching the new think-tag gatingTest results
tests/gateway/test_api_server.py: 107 passedtests/gateway/test_sse_agent_cancel.py: 6 passedtests/run_agent/: 720 passed, 1 pre-existing skip (anthropic pkg)tests/gateway/test_session.py: 60 passedtests/gateway/test_session_reset_notify.py: 13 passedtests/gateway/test_gateway_inactivity_timeout.py: 8 passed