fix(streaming): rebuild correct client on stale stream for anthropic mode

[IEatCodeDaily]

Problem

When a streaming connection goes stale (no chunks received within the stale timeout), the cleanup path in interruptible_streaming_api_call kills the connection and rebuilds the primary client so the retry loop gets a fresh connection pool.

The rebuild unconditionally called _replace_primary_openai_client(), which rebuilds the OpenAI SDK client (self.client).

In anthropic_messages mode the primary streaming client is _anthropic_client (the Anthropic SDK client), not self.client. So for non-OpenAI providers served via an Anthropic-compatible endpoint — e.g. ZAI/GLM-5.x on api.z.ai/api/anthropic, MiniMax, DashScope — two things go wrong:

1. The OpenAI rebuild fails (OPENAI_API_KEY must be set) — there is no OpenAI key for these providers.
2. The actually-stale _anthropic_client is never rebuilt → every retry reuses the dead Anthropic connection → infinite retry loop, hammering the provider.

Observed in production

A ~1.3k-message / ~420k-token ZAI/GLM-5.2 session recovered via Z.AI's Anthropic endpoint hit a stale stream (slow prefill on a cold prefix cache). Hermes retried for hours against a closed client, never recovering and starving the gateway of capacity, with logs:

Stream stale for 300s (threshold 300s) — no chunks received. model=glm-5.2 context=~421,366 tokens. Killing connection.
Failed to rebuild shared OpenAI client (stale_stream_pool_cleanup) ... error=The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
API call failed (attempt N/500) error_type=APIStatusError ... overloaded_error ... (looping forever)

Fix

Branch on api_mode in the streaming stale-stream cleanup. For anthropic_messages and bedrock_converse (both stream through _anthropic_client), rebuild via _rebuild_anthropic_client(); for all other modes, keep the legacy _replace_primary_openai_client() path.

The non-streaming stale path (interruptible_api_call) already branched correctly on api_mode — only the streaming path was broken. This change brings it in line.

try:
    if agent.api_mode in ("anthropic_messages", "bedrock_converse"):
        agent._rebuild_anthropic_client()
    else:
        agent._replace_primary_openai_client(reason="stale_stream_pool_cleanup")
except Exception:
    pass

Testing

• New regression test tests/run_agent/test_stale_stream_anthropic_client_rebuild.py drives the real interruptible_streaming_api_call stale path with a hung stream worker and asserts:
  • anthropic_messages mode → _rebuild_anthropic_client is called, _replace_primary_openai_client is not
  • chat_completions mode → _replace_primary_openai_client is called (legacy behaviour preserved), _rebuild_anthropic_client is not
• Verified the anthropic-mode test fails without the fix and passes with it.
• Existing streaming + client-lifecycle suites (test_streaming, test_stream_interrupt_retry, test_openai_client_lifecycle, test_create_openai_client_reuse): 49 passed, 0 failed.

Scope

This PR contains only this bugfix + its regression test. One file changed in agent/, one test added.

[alt-glitch]

This was generated by AI during triage.

Duplicate of #28240 — same mechanism (branch on api_mode; rebuild _anthropic_client via _rebuild_anthropic_client() for anthropic_messages/bedrock_converse) in the same file agent/chat_completion_helpers.py. This PR fixes the single stale_stream_pool_cleanup site; the earlier-open #28240 applies the identical fix at all three streaming cleanup sites. Sibling #51851 was also marked a duplicate of #28240. Related cluster: #51851, #34761, #38644.

[teknium1]

Closed in favor of #53926, now merged to main (commit a0b9663).

This was a popular bug — 12 independent PRs fixed the same issue (#28161): the three stream-cleanup paths in the streaming code rebuilt the OpenAI primary client unconditionally, which on Anthropic-native sessions both failed (no OPENAI_API_KEY) and left the wedged stream open, causing the ~15-minute hang.

We salvaged @EloquentBrush0x's #28240 (the earliest dedicated fix for this issue) onto current main, resolved the conflict against the newer request-client cleanup helper, repointed the bug-encoding regression test, and added coverage for the two reachable Anthropic cleanup sites. Verified live: stale stream torn down in <1s instead of ~900s, Anthropic client closed+rebuilt, OpenAI rebuild never called on the Anthropic path.

Thank you for the fix — closing as a duplicate of the merged work. Credit to everyone who reported and fixed this.

#53926