fix(streaming): handle Anthropic client in stale stream detector

[xssssrf]

Summary

The streaming stale detector in _run_streaming_api_call() does not handle the anthropic_messages api_mode. When the stale timeout fires, the code attempts to close request_client_holder["client"] — but for Anthropic, this holder is always None (the Anthropic SDK manages its own transport internally). As a result, the stale connection is never actually closed and no fresh connection is ever established.

Root Cause

Three connection-reset sites exist in the streaming path:

Location	Purpose	Has Anthropic branch?
~line 5235	Non-streaming stale detector	✅
~line 6179	Streaming stale detector	❌ Missing
~line 6199	Interrupt handler	✅

The streaming stale detector was copied from the OpenAI path without being extended to cover Anthropic.

Symptom

When the agent is running on the anthropic_messages api_mode and a stream stalls (e.g. due to an SSE keep-alive with no real chunks), the stale detector fires and emits "⚠️ No response from provider... Reconnecting..." — but the underlying Anthropic client is never closed. The inner _call_anthropic() thread keeps waiting on the same dead stream. Each subsequent detector tick emits another status message while doing nothing, leaving the agent stuck until the stream eventually errors out on its own.

Fix

Mirror the pattern already used by the non-streaming stale detector (~line 5235) and the interrupt handler (~line 6199): branch on self.api_mode and call self._anthropic_client.close() + rebuild via build_anthropic_client() for the Anthropic path, keeping the existing OpenAI path unchanged. Guard _replace_primary_openai_client() to the non-Anthropic path only, since it operates on the OpenAI connection pool.

Testing

• python -m py_compile run_agent.py passes
• pytest tests/agent/ -q — 1735 passed, 1 skipped. The 1 pre-existing failure (test_minimax_provider.py::TestMinimaxSwitchModelCredentialGuard) reproduces identically on the unmodified upstream run_agent.py, confirming it is unrelated to this change.
• Tested on macOS (Apple Silicon) with anthropic_messages api_mode via direct Anthropic API key
• Existing OpenAI/chat-completions path behavior is unchanged

[im-sham]

Thanks for tracking this down. I hit the same failure mode on a Linux gateway using anthropic_messages: Hermes repeatedly emitted No response from provider ... Reconnecting..., but the Anthropic stream was not actually closed/rebuilt, so the session stayed stuck until a harder restart/recovery path.

The root cause here matches what I saw. One suggestion before merge: it would be worth adding regression coverage for the anthropic_messages stale-stream path, since prior fixes covered adjacent reconnect sites but this branch was still able to regress. I tested a local variant with a fake blocking Anthropic stream that asserts the stale detector closes the old Anthropic client and rebuilds it.

Validation from that variant:

• python3 -m py_compile run_agent.py tests/run_agent/test_streaming.py
• pytest -o addopts= tests/run_agent/test_streaming.py -q -> 31 passed
• adjacent Anthropic credential/OAuth slice -> 60 passed

Also minor implementation note: current origin/main has _rebuild_anthropic_client(), so the stale detector can call self._anthropic_client.close() followed by self._rebuild_anthropic_client() rather than importing/reconstructing the client inline. Happy to open a tiny follow-up PR with just the regression test/helper-based version if that would help.

[dbreslavets]

Same bug in prod with provider: claude-code (OAuth → anthropic_messages mode) — six 180s stalls in one day, same OPENAI_API_KEY warning.

One nit before merge: re-implementing the Anthropic rebuild inline skips two things _rebuild_anthropic_client already handles — _oauth_1m_beta_disabled carry-forward (matters for OAuth) and the Bedrock branch:

try:
    if self.api_mode == "anthropic_messages":
        try:
            self._anthropic_client.close()
        except Exception:
            pass
        self._rebuild_anthropic_client()
    else:
        rc = request_client_holder.get("client")
        if rc is not None:
            self._close_request_openai_client(rc, reason="stale_stream_kill")
except Exception:
    pass

I have a 4-case dispatch test (addresses @im-sham's coverage note) — can attach if useful.

[teknium1]

Closed in favor of #53926, now merged to main (commit a0b9663).

This was a popular bug — 12 independent PRs fixed the same issue (#28161): the three stream-cleanup paths in the streaming code rebuilt the OpenAI primary client unconditionally, which on Anthropic-native sessions both failed (no OPENAI_API_KEY) and left the wedged stream open, causing the ~15-minute hang.

We salvaged @EloquentBrush0x's #28240 (the earliest dedicated fix for this issue) onto current main, resolved the conflict against the newer request-client cleanup helper, repointed the bug-encoding regression test, and added coverage for the two reachable Anthropic cleanup sites. Verified live: stale stream torn down in <1s instead of ~900s, Anthropic client closed+rebuilt, OpenAI rebuild never called on the Anthropic path.

Thank you for the fix — closing as a duplicate of the merged work. Credit to everyone who reported and fixed this.

#53926