feat(agent): finite first-chunk timeout + failover for local OpenAI-compatible streams - #37168
feat(agent): finite first-chunk timeout + failover for local OpenAI-compatible streams#37168OmarB97 wants to merge 1 commit into
Conversation
f03ceec to
f105b3b
Compare
f105b3b to
c4aad3f
Compare
Reapply the local-provider stale/TTFB timeout bounding (stream, first-chunk, and non-stream resolvers; local_first_chunk_timeout tracked through the streaming worker result; the reasoning-floor and explicit-config interplay fix) onto the post-history-replacement mainline. Conflict resolution (agent/chat_completion_helpers.py, two spots): upstream grew its own independent, unrelated addition in the same locations since this branch's last refresh — the cross-turn stale-call circuit breaker (NousResearch#58962: _stale_streak/_bump_stale_streak/_reset_stale_streak/_check_stale_giveup). Both features are additive; kept upstream's circuit-breaker block ahead of the local-provider timeout resolvers, and folded _check_stale_giveup(agent) into the same result-dict block that carries local_first_chunk_timeout rather than picking one side. Everything else (agent_runtime_helpers.py, conversation_loop.py, error_classifier.py, run_agent.py, and the four test files) auto-merged with zero conflict. Ran the full touched-module suite locally: tests/agent/test_local_stream_timeout.py, tests/agent/test_error_classifier.py, tests/hermes_cli/test_timeouts.py, tests/run_agent/test_run_agent.py — 684 passed. Refresh of PR NousResearch#37168; prior head 0440c4c625ccc57fb1aad1eb1eb5e919f3d8726f (a follow-up fix commit on top of an earlier orphan-commit refresh, c4aad3f1fc, which itself carries no original-head trailer — PR timeline shows force-pushes on 2026-06-10 and 2026-07-05 for anyone reconstructing provenance further back).
0440c4c to
82d4649
Compare
|
Refreshed onto current upstream/main via rebase (real merge-base — an earlier "no common ancestor" read on a sibling PR in this wave turned out to be a shallow-clone artifact, not true history divergence). Two commits replayed (an earlier orphan refresh + a follow-up bug-fix that patched two regressions in it), then squashed to one. One real conflict, in |
|
Thanks for the careful refresh, focused tests, and preservation of the existing stale-call circuit-breaker work. The local no-first-chunk problem is real, but this implementation cannot land in its current configuration form.
This is an automated hermes-sweeper review. Closed as not-planned per standing maintainer policy ( |
What
Local OpenAI-compatible backends currently get an unbounded wait in two failure modes: a stream that is accepted but never emits its first SSE chunk, and a non-streaming call whose server sends no response headers until the entire completion finishes. Both wedge the turn instead of failing over.
How
HERMES_LOCAL_FIRST_CHUNK_TIMEOUT(default 90s, scales with estimated request context) bounds time-to-first-chunk for local streams; on expiry the connection is killed, the error is marked, andFailoverReason.local_first_chunk_timeoutroutes straight to the fallback chain instead of rebuilding the same wedged client.HERMES_LOCAL_NON_STREAM_STALE_TIMEOUT(default 120s, context-scaled) makes local non-streaming calls finite.HERMES_LOCAL_STALE_TIMEOUTis an opt-in bound for backends known to park requests on open sockets.Tests
tests/agent/test_local_stream_timeout.py(boundaries, opt-in, env precedence),tests/agent/test_error_classifier.py(marker classification),tests/hermes_cli/test_timeouts.py(finite local non-stream default), plus an ordering assertion that first-chunk fallback precedes primary-transport recovery. 230+ tests pass locally.This is part 4 of the former #35642 split; rewritten per review feedback to be model-name agnostic (no model-specific env vars or heuristics) and to drop the unrelated oneshot logging changes.