fix(fallback): trigger immediate failover on overloaded (529) errors - #60790
Open
AIalliAI wants to merge 1 commit into
Open
fix(fallback): trigger immediate failover on overloaded (529) errors#60790AIalliAI wants to merge 1 commit into
AIalliAI wants to merge 1 commit into
Conversation
…ousResearch#60761) Two fixes for the fallback chain: Bug 1: Provider overload (529) errors required retry_count >= 2 before fallback would activate, but with default api_max_retries=3 the retry budget was exhausted before the threshold was reached — fallback never triggered. Treat overloaded like rate_limit: immediate failover without retry threshold (matches Anthropic SDK behavior). Z.AI Coding Plan overload 429s preserve their existing long-backoff schedule. Bug 2: The _unavailable_fallback_keys set permanently suppressed nous fallback entries on ANY exception, including transient network errors. Remove the blanket suppression — the local_skip_reason + fb_client-is-None guards already handle permanent config/auth problems. Fixes: NousResearch#60761
alt-glitch
added
type/bug
Collaborator
Duplicate of #60785 (created ~20 min earlier by @liuhao1024, also |
13 tasks
teknium1
added
sweeper:risk-session-state
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes for
fallback_providersthat were broken in real-world conditions:Bug 1: Overloaded (529) never triggers fallback
FailoverReason.overloadedwas in the transport-failure set requiringretry_count >= 2, but with defaultapi_max_retries=3the retry budget was exhausted before reaching the threshold — fallback never activated.Fix: Move
overloadedto the rate-limit set (immediate failover, no retry threshold), matching Anthropic SDK behavior. Z.AI Coding Plan overload 429s preserve their existing long-backoff schedule.Bug 2: Fallback entries permanently suppressed on transient errors
_unavailable_fallback_keyspermanently suppressed nous fallback entries on ANY exception, including transient network errors, for the entire session lifetime.Fix: Remove the blanket nous suppression from the except block. The existing
local_skip_reasonandfb_client-is-Noneguards already handle permanent config/auth problems.Changes
agent/conversation_loop.py: overloaded → immediate failover (with Z.AI exception)agent/chat_completion_helpers.py: remove permanent nous suppressionFixes: #60761