fix(agent): route Anthropic stream cleanup through _anthropic_client, not OpenAI rebuild - #34203
fix(agent): route Anthropic stream cleanup through _anthropic_client, not OpenAI rebuild#34203liuhao1024 wants to merge 2 commits into
Conversation
Three streaming-cleanup paths (stream_mid_tool_retry_pool_cleanup, stream_retry_pool_cleanup, stale_stream_pool_cleanup) unconditionally called _replace_primary_openai_client() even when api_mode was "anthropic_messages". For Anthropic-only configs this: 1. Failed with "Missing credentials" (no OPENAI_API_KEY) 2. Left the in-flight Anthropic httpx stream open, causing a ~15-min hang until the 900s read-timeout fired Fix: check api_mode and call _anthropic_client.close() + _rebuild_anthropic_client() instead, matching the existing pattern used by the _interrupt_requested handler. Fixes NousResearch#28161
|
Closed in favor of #53926, now merged to main (commit a0b9663). This was a popular bug — 12 independent PRs fixed the same issue (#28161): the three stream-cleanup paths in the streaming code rebuilt the OpenAI primary client unconditionally, which on Anthropic-native sessions both failed (no OPENAI_API_KEY) and left the wedged stream open, causing the ~15-minute hang. We salvaged @EloquentBrush0x's #28240 (the earliest dedicated fix for this issue) onto current main, resolved the conflict against the newer request-client cleanup helper, repointed the bug-encoding regression test, and added coverage for the two reachable Anthropic cleanup sites. Verified live: stale stream torn down in <1s instead of ~900s, Anthropic client closed+rebuilt, OpenAI rebuild never called on the Anthropic path. Thank you for the fix — closing as a duplicate of the merged work. Credit to everyone who reported and fixed this. |
What does this PR do?
When
api_mode == "anthropic_messages", three streaming-cleanup paths inagent/chat_completion_helpers.pyunconditionally called_replace_primary_openai_client()to rebuild the connection pool after stream errors. For Anthropic-only configurations this:Missing credentials. Please pass an api_keybecauseOPENAI_API_KEYis unsetUser-visible symptom: the agent appears to hang ~15 minutes on stuck streams before any retry or fallback engages.
Related Issue
N/A
Type of Change
Changes Made
agent/chat_completion_helpers.pyapi_mode == "anthropic_messages"tests/run_agent/test_streaming.pyHow to Test
pytest tests/ -q— all tests should passChecklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture and workflows — or N/ACode Intelligence
agent/chat_completion_helpers.py:interruptible_streaming_api_call(streaming cleanup paths)api_modecheck, OpenAI path unchanged_interrupt_requestedhandler at lines ~2090-2092 already uses the same Anthropic cleanup pattern