fix(error_classifier): treat Z.AI 1305 overload as overloaded, not rate_limit - #77771
Open
Rams-Lab-01 wants to merge 1 commit into
Open
Rams-Lab-01 wants to merge 1 commit into
Rams-Lab-01 wants to merge 1 commit into
Conversation
…te_limit NousResearch#53578 added _OVERLOADED_PATTERNS so a 429 carrying overload language takes the transient-overload path instead of rotating the credential. The pattern list is English-only, but Z.AI/Zhipu return the code-1305 overload message localized ("该模型当前访问量过大,请您稍后再试") depending on account locale. The motivating provider therefore still fell through to rate_limit and exhausted the pool after two errors — fatal for a single-key user, who has nothing to rotate to. Two gaps are closed: 1. error_classifier now checks the structured error code before the message patterns. The code is locale-independent, so classification no longer depends on the account's message language. Only 1305 is treated as overload; 1302 and 1313 (genuine rate limits) and 1308, 1310, 1316-1321 (usage/spend limits) keep their existing behavior. The zh-CN overload phrasings are added to _OVERLOADED_PATTERNS as a secondary net, deliberately excluding "请求过于频繁" and the generic "请稍后再试", which also appear on real rate limits. 2. The auxiliary path (vision, web_extract, compression, cron) never consulted the classifier: _is_rate_limit_error() returns True for any openai RateLimitError before inspecting the message, so a 1305 burned the pool there regardless of the first fix. _recover_provider_pool — the single choke point for auxiliary pool marking — now leaves the credential intact on an overload and lets the caller retry or fall back. It shares the pattern and code tables with error_classifier so both layers agree. Observed in production: a transient Z.AI overload on glm-4.6v-flash marked a barely-used GLM_API_KEY exhausted, collapsed the pool to "no available entries", and fell back to a text-only model that cannot accept image_url, degrading vision to OCR for the end user. Adds regression tests for the localized 1305 body, code-based classification without recognizable message text, and guards asserting that 1302 and "请求过于频繁" still classify as rate_limit and still rotate. Refs NousResearch#14038
13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#53578 added
_OVERLOADED_PATTERNSso a 429 carrying overload language takes the transient-overload path instead of rotating the credential.The pattern list is English-only, but Z.AI / Zhipu return the code-1305 overload message localized depending on account locale:
None of the English patterns match that text, so the provider that motivated #53578 still falls through to
rate_limitand exhausts the pool after two errors — fatal for a single-key user, who has nothing to rotate to.Per Z.AI's error reference,
1305is "the service may be temporarily overloaded, please try again later" — a server-side capacity condition, not a per-credential rate limit.Production impact
A transient Z.AI overload on
glm-4.6v-flashmarked a barely-usedGLM_API_KEYexhausted, collapsed the pool tono available entries, and fell back to a text-only model that rejectsimage_url:The user-visible result was vision silently degrading to OCR, on a key that had barely been used.
Fix
1.
error_classifier— classify on the structured error code first.The code is locale-independent, so classification no longer depends on the account's message language. Only
1305is treated as overload; the other Z.AI 429 codes keep their existing behavior:13051302,13131308,1310,1316–1321The zh-CN overload phrasings are also added to
_OVERLOADED_PATTERNSas a secondary net. This deliberately excludes请求过于频繁("requests too frequent") and the generic请稍后再试("try again later"), which also appear on genuine rate limits and must keep rotating.2.
auxiliary_client— the auxiliary path never consulted the classifier.Vision /
web_extract/ compression / cron go through_is_rate_limit_error(), which returnsTruefor anyopenaiRateLimitErrorbefore inspecting the message:So a 1305 burned the pool there regardless of fix (1) — this is the path that actually broke vision in the report above.
_recover_provider_pool, the single choke point for auxiliary pool marking, now leaves the credential intact on an overload and lets the caller retry / fall back. It shares the pattern and code tables witherror_classifierso both layers classify identically.Tests
5 regression tests added covering:
1302and请求过于频繁must still classify asrate_limitand still rotatetest_credential_pool_routing.pyreports 2 failures (test_resolve_turn_includes_pool,test_unmatched_key_does_not_retry_only_pool_entry) — these are pre-existing onmainand unrelated to this change; verified identical with and without it by stashing.The change is purely additive (+151 lines, no deletions, no behavior removed).
Refs #14038