fix(errors): classify throttle messages before token-overflow patterns; add new overflow shapes (port opencode#37848) - #70394
Merged
Conversation
…s; add new overflow shapes Port from anomalyco/opencode#37848 (+ dev-branch twin #37840): expand context-overflow patterns and guard against rate-limit messages that mention tokens. - 'Throttling error: Too many tokens, please wait before trying again.' (AWS Bedrock / proxy shape) classified as context_overflow and routed a healthy session into compression on every throttle. Added 'throttling' to _RATE_LIMIT_PATTERNS, which the message-only path checks BEFORE the overflow list. - 'Input length N exceeds the maximum allowed input length of M tokens.' (Together/Fireworks shape) fell through to unknown — no compression recovery. Added 'maximum allowed input length' to overflow patterns. - 'request_too_large' / 'Request exceeds the maximum size' (Anthropic 413 type re-wrapped without a status code by aggregators/proxies) fell through to unknown. Added to _PAYLOAD_TOO_LARGE_PATTERNS. All three shapes proven live on main before the fix; 265 classifier + bedrock tests and 238 sibling rate-guard/compression tests pass.
Contributor
૮ >ﻌ< ა ci reviewran on 372e94b all good! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Throttle messages that mention tokens no longer route a healthy session into the compression loop, and two previously-unclassified overflow/payload error shapes now trigger compression recovery. Port of anomalyco/opencode#37848 (and its dev-branch twin #37840).
Root cause:
"too many tokens"sits in_CONTEXT_OVERFLOW_PATTERNS, and the AWS Bedrock / proxy throttle shape"Throttling error: Too many tokens, please wait before trying again."matched no rate-limit pattern — so the message-only path fell through to the overflow list and compressed on every throttle instead of backing off.All three gaps were proven live on current
mainbefore the fix (probe output in the validation table).Changes
agent/error_classifier.py:_RATE_LIMIT_PATTERNS+="throttling"— checked BEFORE the overflow list in the message-only path, so token-mentioning throttles now classify asrate_limit._CONTEXT_OVERFLOW_PATTERNS+="maximum allowed input length"(Together/Fireworks-style wording: "Input length N exceeds the maximum allowed input length of M tokens.")._PAYLOAD_TOO_LARGE_PATTERNS+="request_too_large","request exceeds the maximum size"(Anthropic's structured 413 type re-wrapped without a status code by aggregators/proxies).tests/agent/test_error_classifier.py: newTestThrottleVsOverflowDisambiguation+TestExpandedOverflowPatternsclasses (positive + regression guards, including plain"Too many tokens"still classifying as overflow).Adaptation notes: OpenCode implemented this as a global
exclusionslist gatingisContextOverflow. Hermes already orders rate-limit checks before overflow in_classify_by_message, so the correct adaptation is one rate-limit pattern, not a new exclusion mechanism. OpenCode's"token limit exceeded"overflow pattern was deliberately NOT adopted — hermes routes"limit exceeded"through the usage-limit billing/rate-limit disambiguation on purpose. Theirrequest_too_large→ overflow mapping was adapted to hermes'spayload_too_largetaxonomy (same compression recovery, more precise reason).Validation
context_overflow, compress=Truerate_limit, compress=Falseunknowncontext_overflow, compress=True{"error":{"type":"request_too_large","message":"Request exceeds the maximum size"}}unknownpayload_too_large, compress=Truerate_limitrate_limit(unchanged)context_overflowcontext_overflow(unchanged)Tests: 265 passed (
test_error_classifier.py+test_bedrock_integration.py), 238 passed sibling (test_bedrock_adapter.py,test_nous_rate_guard.py,test_24996_fallback_exhaustion_cooldown.py,test_413_compression.py).Infographic