Skip to content

fix(errors): classify throttle messages before token-overflow patterns; add new overflow shapes (port opencode#37848) - #70394

Merged
teknium1 merged 1 commit into
mainfrom
opencode-port/overflow-throttle-classifier
Jul 27, 2026
Merged

fix(errors): classify throttle messages before token-overflow patterns; add new overflow shapes (port opencode#37848)#70394
teknium1 merged 1 commit into
mainfrom
opencode-port/overflow-throttle-classifier

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Throttle messages that mention tokens no longer route a healthy session into the compression loop, and two previously-unclassified overflow/payload error shapes now trigger compression recovery. Port of anomalyco/opencode#37848 (and its dev-branch twin #37840).

Root cause: "too many tokens" sits in _CONTEXT_OVERFLOW_PATTERNS, and the AWS Bedrock / proxy throttle shape "Throttling error: Too many tokens, please wait before trying again." matched no rate-limit pattern — so the message-only path fell through to the overflow list and compressed on every throttle instead of backing off.

All three gaps were proven live on current main before the fix (probe output in the validation table).

Changes

  • agent/error_classifier.py:
    • _RATE_LIMIT_PATTERNS += "throttling" — checked BEFORE the overflow list in the message-only path, so token-mentioning throttles now classify as rate_limit.
    • _CONTEXT_OVERFLOW_PATTERNS += "maximum allowed input length" (Together/Fireworks-style wording: "Input length N exceeds the maximum allowed input length of M tokens.").
    • _PAYLOAD_TOO_LARGE_PATTERNS += "request_too_large", "request exceeds the maximum size" (Anthropic's structured 413 type re-wrapped without a status code by aggregators/proxies).
  • tests/agent/test_error_classifier.py: new TestThrottleVsOverflowDisambiguation + TestExpandedOverflowPatterns classes (positive + regression guards, including plain "Too many tokens" still classifying as overflow).

Adaptation notes: OpenCode implemented this as a global exclusions list gating isContextOverflow. Hermes already orders rate-limit checks before overflow in _classify_by_message, so the correct adaptation is one rate-limit pattern, not a new exclusion mechanism. OpenCode's "token limit exceeded" overflow pattern was deliberately NOT adopted — hermes routes "limit exceeded" through the usage-limit billing/rate-limit disambiguation on purpose. Their request_too_large → overflow mapping was adapted to hermes's payload_too_large taxonomy (same compression recovery, more precise reason).

Validation

Input (no status code) Before After
"Throttling error: Too many tokens, please wait before trying again." context_overflow, compress=True rate_limit, compress=False
"Input length 131393 exceeds the maximum allowed input length of 131040 tokens." unknown context_overflow, compress=True
{"error":{"type":"request_too_large","message":"Request exceeds the maximum size"}} unknown payload_too_large, compress=True
"Rate limit exceeded, please retry after 30 seconds." rate_limit rate_limit (unchanged)
"Too many tokens" (plain, Z.AI/GLM) context_overflow context_overflow (unchanged)

Tests: 265 passed (test_error_classifier.py + test_bedrock_integration.py), 238 passed sibling (test_bedrock_adapter.py, test_nous_rate_guard.py, test_24996_fallback_exhaustion_cooldown.py, test_413_compression.py).

Infographic

overflow-throttle-classifier

…s; add new overflow shapes

Port from anomalyco/opencode#37848 (+ dev-branch twin #37840): expand
context-overflow patterns and guard against rate-limit messages that
mention tokens.

- 'Throttling error: Too many tokens, please wait before trying again.'
  (AWS Bedrock / proxy shape) classified as context_overflow and routed a
  healthy session into compression on every throttle. Added 'throttling'
  to _RATE_LIMIT_PATTERNS, which the message-only path checks BEFORE the
  overflow list.
- 'Input length N exceeds the maximum allowed input length of M tokens.'
  (Together/Fireworks shape) fell through to unknown — no compression
  recovery. Added 'maximum allowed input length' to overflow patterns.
- 'request_too_large' / 'Request exceeds the maximum size' (Anthropic 413
  type re-wrapped without a status code by aggregators/proxies) fell
  through to unknown. Added to _PAYLOAD_TOO_LARGE_PATTERNS.

All three shapes proven live on main before the fix; 265 classifier +
bedrock tests and 238 sibling rate-guard/compression tests pass.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/compression Context compression and continuation sessions P2 Medium — degraded but workaround exists labels Jul 24, 2026
@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

ran on 372e94b

all good!

@teknium1
teknium1 merged commit 53bfe40 into main Jul 27, 2026
40 checks passed
@teknium1
teknium1 deleted the opencode-port/overflow-throttle-classifier branch July 27, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compression Context compression and continuation sessions comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants