Skip to content

fix(agent): recover Anthropic stale streams without OPENAI_API_KEY errors - #55245

Closed
gislihaukur wants to merge 1 commit into
NousResearch:mainfrom
gislihaukur:fix/anthropic-stale-stream
Closed

gislihaukur wants to merge 1 commit into
NousResearch:mainfrom
gislihaukur:fix/anthropic-stale-stream

Conversation

@gislihaukur

Copy link
Copy Markdown

Summary

  • Re-resolve Anthropic OAuth credentials when rebuilding the client after a stale stream kill, instead of falling back to an empty OPENAI_API_KEY
  • Close the in-flight Anthropic streaming client before rebuild so the worker thread unblocks
  • Abort after repeated stale-stream reconnect attempts instead of spinning forever

Context

Observed on Telegram profiles using Anthropic OAuth: stale SSE streams triggered client rebuild, which then failed with AuthenticationError: Missing OPENAI_API_KEY because rebuild reused stale/empty credentials.

Test plan

  • scripts/run_tests.sh tests/run_agent/test_run_agent.py -q (new tests for rebuild key resolution + stale-stream abort path)
  • Manual: trigger stale stream on Anthropic OAuth profile and confirm reconnect succeeds without OPENAI_API_KEY error

Made with Cursor

…rors

Re-resolve Anthropic OAuth credentials on stale-stream client rebuild and close the in-flight Anthropic stream so the worker thread can exit instead of spinning forever.

Co-authored-by: Cursor <cursoragent@cursor.com>
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/anthropic Anthropic native Messages API labels Jun 29, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

The core fix here — routing Anthropic stale-stream cleanup to the Anthropic client instead of the OpenAI rebuild — has already landed on main via the merged #53926 (the stale-stream cleanup site now branches on api_mode == "anthropic_messages" and calls _rebuild_anthropic_client()). This PR re-resolves Anthropic credentials before still calling the OpenAI rebuild, which is superseded by that merge. The net-new piece worth reviewing on its own is the stale-stream retry-abort cap (HERMES_STREAM_RETRIES). Related: merged #53926, cluster #28161 / #51844.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Good fix for Anthropic stale stream recovery. The key improvement is closing the Anthropic client before rebuilding — this unblocks the for event in stream loop that would otherwise spin forever. The retry count tracking with configurable HERMES_STREAM_RETRIES is a nice touch for observability.

Correctness: The Anthropic-specific path correctly closes the client before rebuild, matching the non-streaming/interrupt paths.
Code Quality: Clean separation of Anthropic vs non-Anthropic recovery paths.


Reviewed by Hermes Agent

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Fixes stale stream recovery for Anthropic. Re-resolves OAuth credentials when rebuilding the client, closes in-flight Anthropic client to unblock the worker thread, and adds retry logic with configurable max retries.

  • Clean fix for a real production issue
  • Good error handling and retry logic
  • Well-scoped: 3 files, 178 additions

Reviewed 3 files, 178 additions. Approved.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the stale-stream recovery work. This is already implemented on current main; this automated hermes-sweeper review is closing the superseded PR.

  • a0b9663c7cfed007acede4f1261212811aa623ef implemented the Anthropic-native cleanup path and shipped in v2026.7.1.
  • agent/chat_completion_helpers.py:3080 now closes and rebuilds the Anthropic client for anthropic_messages; the OpenAI-client rebuild remains only in the non-Anthropic branch at :3087.
  • tests/run_agent/test_28161_anthropic_stream_pool_cleanup.py:109 covers the stale-stream path and asserts that the OpenAI rebuild is not called.
  • Current main also has a stale-stream circuit breaker at agent/chat_completion_helpers.py:2093 and :3075, addressing repeated unresponsive-provider attempts.

This matches the prior #53926 discussion identifying the Anthropic cleanup routing as already landed.

@teknium1 teknium1 closed this Jul 15, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants