Skip to content

fix(run_agent): skip OpenAI client rebuild on Anthropic-native sessions - #51508

Closed
noodlemctwoodle wants to merge 1 commit into
NousResearch:mainfrom
noodlemctwoodle:fix/anthropic-stream-pool-cleanup-wrong-provider
Closed

fix(run_agent): skip OpenAI client rebuild on Anthropic-native sessions#51508
noodlemctwoodle wants to merge 1 commit into
NousResearch:mainfrom
noodlemctwoodle:fix/anthropic-stream-pool-cleanup-wrong-provider

Conversation

@noodlemctwoodle

Copy link
Copy Markdown

What

_replace_primary_openai_client was called unconditionally by all three stream-cleanup paths in chat_completion_helpers (stale_stream_pool_cleanup, stream_retry_pool_cleanup, stream_mid_tool_retry_pool_cleanup). When api_mode=anthropic_messages the agent owns _anthropic_client and self.client is None; _client_kwargs is an empty dict. The unconditional OpenAI(**{}) call raised AuthenticationError with "The api_key client option must be set … OPENAI_API_KEY" even though the correct Anthropic key was present.

This produced a deterministic false-alarm WARNING in errors.log on every Anthropic stream stall for profiles that have ANTHROPIC_API_KEY but no OPENAI_API_KEY — which is correct configuration. 3 occurrences verified on a single session.

Root cause

All three cleanup sites call agent._replace_primary_openai_client(). That method calls _create_openai_client(self._client_kwargs, ...)OpenAI(**{}). The OpenAI SDK then reads the env for OPENAI_API_KEY, finds nothing, and raises. The exception is caught by the except Exception: pass guard at each call site (so the agent continues), but the logger.warning inside _replace_primary_openai_client fires first and lands in errors.log with a misleading message.

Fix

Short-circuit on api_mode == "anthropic_messages" and call _rebuild_anthropic_client() instead — the existing method that already handles both direct Anthropic and Bedrock-hosted variants and honours _oauth_1m_beta_disabled. The OpenAI path is unchanged; the guard only fires on the Anthropic wire.

Tests

4 new regression tests in tests/run_agent/test_anthropic_stream_pool_cleanup.py:

  • OpenAI() constructor never called for Anthropic-native sessions
  • _rebuild_anthropic_client is invoked instead
  • Failure emits the right warning text (Anthropic client, not OpenAI client)
  • OpenAI path still fires for api_mode=chat_completions

All 4 pass; 19 adjacent lifecycle/switch-model tests unaffected.

_replace_primary_openai_client was called unconditionally by all three
stream-cleanup paths in chat_completion_helpers (stale_stream_pool_cleanup,
stream_retry_pool_cleanup, stream_mid_tool_retry_pool_cleanup).  When
api_mode=anthropic_messages the agent owns _anthropic_client and
self.client is None; _client_kwargs is an empty dict.  The unconditional
OpenAI(**{}) call raised AuthenticationError with 'OPENAI_API_KEY not set'
even though the correct Anthropic key was present, producing a
deterministic false-alarm warning on every stream stall for profiles
without OPENAI_API_KEY.

Fix: short-circuit on api_mode=="anthropic_messages" and call
_rebuild_anthropic_client() instead, which handles both direct Anthropic
and Bedrock-hosted variants and honours _oauth_1m_beta_disabled.

The OpenAI path is unchanged; the guard only fires on the Anthropic wire.

Adds 4 regression tests covering: no OpenAI constructor call, correct
Anthropic rebuild invoked, failure logged under the right client name,
and the OpenAI path unaffected for non-Anthropic sessions.
@alt-glitch alt-glitch added P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists provider/anthropic Anthropic native Messages API type/bug Something isn't working labels Jun 23, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #36719 — same guard in the same file (run_agent.py _replace_primary_openai_client short-circuit for api_mode==anthropic_messages) fixing the misleading OPENAI_API_KEY warning on Anthropic stream stalls. #36719 no-ops; this rebuilds the Anthropic client (the approach #44076 takes in chat_completion_helpers). Related to the broader stale-stream cluster #44076/#28161/#36693.

@teknium1

Copy link
Copy Markdown
Contributor

Closed in favor of #53926, now merged to main (commit a0b9663).

This was a popular bug — 12 independent PRs fixed the same issue (#28161): the three stream-cleanup paths in the streaming code rebuilt the OpenAI primary client unconditionally, which on Anthropic-native sessions both failed (no OPENAI_API_KEY) and left the wedged stream open, causing the ~15-minute hang.

We salvaged @EloquentBrush0x's #28240 (the earliest dedicated fix for this issue) onto current main, resolved the conflict against the newer request-client cleanup helper, repointed the bug-encoding regression test, and added coverage for the two reachable Anthropic cleanup sites. Verified live: stale stream torn down in <1s instead of ~900s, Anthropic client closed+rebuilt, OpenAI rebuild never called on the Anthropic path.

Thank you for the fix — closing as a duplicate of the merged work. Credit to everyone who reported and fixed this.

#53926

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants