Skip to content

fix(agent): route Anthropic stream cleanup through _anthropic_client, not OpenAI rebuild - #34203

Closed
liuhao1024 wants to merge 2 commits into
NousResearch:mainfrom
liuhao1024:fix/anthropic-streaming-cleanup-hang
Closed

fix(agent): route Anthropic stream cleanup through _anthropic_client, not OpenAI rebuild#34203
liuhao1024 wants to merge 2 commits into
NousResearch:mainfrom
liuhao1024:fix/anthropic-streaming-cleanup-hang

Conversation

@liuhao1024

@liuhao1024 liuhao1024 commented May 29, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

When api_mode == "anthropic_messages", three streaming-cleanup paths in agent/chat_completion_helpers.py unconditionally called _replace_primary_openai_client() to rebuild the connection pool after stream errors. For Anthropic-only configurations this:

  1. Failed with Missing credentials. Please pass an api_key because OPENAI_API_KEY is unset
  2. Left the Anthropic httpx stream open — the worker thread blocked on the dead socket until the 900s read-timeout fired

User-visible symptom: the agent appears to hang ~15 minutes on stuck streams before any retry or fallback engages.

Related Issue

N/A

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

File Change
agent/chat_completion_helpers.py Route 3 cleanup paths through Anthropic client when api_mode == "anthropic_messages"
tests/run_agent/test_streaming.py Update existing retry test assertions + add regression test for stale-stream cleanup

How to Test

  1. Run pytest tests/ -q — all tests should pass
  2. Verify the specific scenario described above is resolved

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 26.4.1

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture and workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A

Code Intelligence

  • Analyzed: agent/chat_completion_helpers.py:interruptible_streaming_api_call (streaming cleanup paths)
  • Blast radius: LOW — change is guarded by api_mode check, OpenAI path unchanged
  • Related patterns: existing _interrupt_requested handler at lines ~2090-2092 already uses the same Anthropic cleanup pattern

Three streaming-cleanup paths (stream_mid_tool_retry_pool_cleanup,
stream_retry_pool_cleanup, stale_stream_pool_cleanup) unconditionally
called _replace_primary_openai_client() even when api_mode was
"anthropic_messages". For Anthropic-only configs this:

1. Failed with "Missing credentials" (no OPENAI_API_KEY)
2. Left the in-flight Anthropic httpx stream open, causing a ~15-min
   hang until the 900s read-timeout fired

Fix: check api_mode and call _anthropic_client.close() +
_rebuild_anthropic_client() instead, matching the existing pattern
used by the _interrupt_requested handler.

Fixes NousResearch#28161
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/anthropic Anthropic native Messages API labels May 29, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Competes with #28240 and #33855 — all fix the same 3 streaming-cleanup paths routing through OpenAI client instead of Anthropic (#28161). This PR and #28240 cover the same scope.

@teknium1

Copy link
Copy Markdown
Contributor

Closed in favor of #53926, now merged to main (commit a0b9663).

This was a popular bug — 12 independent PRs fixed the same issue (#28161): the three stream-cleanup paths in the streaming code rebuilt the OpenAI primary client unconditionally, which on Anthropic-native sessions both failed (no OPENAI_API_KEY) and left the wedged stream open, causing the ~15-minute hang.

We salvaged @EloquentBrush0x's #28240 (the earliest dedicated fix for this issue) onto current main, resolved the conflict against the newer request-client cleanup helper, repointed the bug-encoding regression test, and added coverage for the two reachable Anthropic cleanup sites. Verified live: stale stream torn down in <1s instead of ~900s, Anthropic client closed+rebuilt, OpenAI rebuild never called on the Anthropic path.

Thank you for the fix — closing as a duplicate of the merged work. Credit to everyone who reported and fixed this.

#53926

@teknium1 teknium1 closed this Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants