Skip to content

fix(error_classifier): gate absolute msg/token heuristics to small context windows - #16380

Closed
Sanjays2402 wants to merge 1 commit into
NousResearch:mainfrom
Sanjays2402:fix/error-classifier-1m-context-16351
Closed

fix(error_classifier): gate absolute msg/token heuristics to small context windows#16380
Sanjays2402 wants to merge 1 commit into
NousResearch:mainfrom
Sanjays2402:fix/error-classifier-1m-context-16351

Conversation

@Sanjays2402

Copy link
Copy Markdown
Contributor

Closes #16351.

Problem

agent/error_classifier.py flagged non-context errors as context_overflow in long-context (1M) Codex/GPT-5.x sessions, purely because num_messages > 80 (generic 400) or num_messages > 200 (disconnect) — even when approx_tokens was a fraction of the actual budget.

Repro from the issue:

classify_api_error(
    FakeHTTP400(),
    provider="openai-codex",
    model="gpt-5.5",
    approx_tokens=74320,
    context_length=1_000_000,
    num_messages=432,
)
# Before: FailoverReason.context_overflow (retryable=True, should_compress=True)
# After:  FailoverReason.format_error      (retryable=False, should_compress=False)

That sent format errors into the compression/probe-down path, causing unnecessary compaction and stale handoff pollution on 1M sessions.

Fix

Apply exactly the gate suggested in the issue body: scope absolute token/message-count fallbacks to context_length <= 256000. Relative pressure thresholds (> 0.6 for disconnect, > 0.4 for generic 400) still fire on any context size.

# server disconnect path
is_large = approx_tokens > context_length * 0.6 or (
    context_length <= 256000 and (approx_tokens > 120000 or num_messages > 200)
)

# generic 400 path
is_large = approx_tokens > context_length * 0.4 or (
    context_length <= 256000 and (approx_tokens > 80000 or num_messages > 80)
)

Existing behavior for ~128K/200K context windows is unchanged.

Tests

tests/agent/test_error_classifier.py — 4 new tests covering the 1M-context regime:

  • test_400_generic_1m_context_high_message_count_not_overflow — exact repro from issue (74K tokens, 432 msgs, 1M ctx) → format_error.
  • test_400_generic_1m_context_relative_pressure_still_overflow — 500K tokens / 1M ctx still → context_overflow.
  • test_disconnect_1m_context_high_message_count_is_timeout — 150K tokens, 300 msgs, 1M ctx → timeout.
  • test_disconnect_1m_context_relative_pressure_still_overflow — 700K tokens / 1M ctx still → context_overflow.
pytest tests/agent/test_error_classifier.py -q
122 passed (118 pre-existing + 4 new)

…ntext windows (closes NousResearch#16351)

Generic 400s and server disconnects in long Codex/GPT-5.x sessions
(1M context window) were being classified as context_overflow purely
because num_messages > 80 (or > 200 for disconnects), even when
approx_tokens was well under the relative threshold (e.g. 74K against 1M).

That sent non-context errors into the compression/probe-down recovery
path, causing unnecessary compaction and stale handoff pollution on
explicit 1M sessions.

Gate the absolute fallbacks behind context_length <= 256000 — preserves
existing behavior for ~128K/200K windows while letting 1M sessions
classify correctly via relative pressure only.

Tests: pytest tests/agent/test_error_classifier.py -q -> 122/122 pass
(118 pre-existing + 4 new for 1M-context generic-400 / disconnect
low and high token-pressure cases).
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Apr 27, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #16352 — same fix (gate absolute overflow heuristics to <=256K context) for #16351. Both by Sanjays2402.

@Sanjays2402

Copy link
Copy Markdown
Contributor Author

CI status note for maintainers — the failing test check on this PR is from a set of 15 pre-existing test failures on main, not regressions introduced here.

Verified by diffing the failing-test sets:

  • Latest main push run (25250051126): 16 failed
  • This PR's run: same 15 (subset)
  • Net new failures introduced by this PR: 0

The clusters on main:

Cluster Tests Likely cause
Systemd TimeoutStopSec test_*_unit_avoids_recursive_execstop_and_uses_extended_stop_timeout Code emits 210, test asserts 90 — drift
Gateway restart kill semantics test_update_* Recent change in expected .kill() call counts
update --yes flag test_update_yes_flag TTY prompt / stash restore behavior changed
Dockerfile pid1 test_dockerfile_* Dockerfile regenerated, dropped TUI ink references
Concurrent interrupt _Stub test fixture missing _tool_guardrails attr
dotenv vs os.environ test_os_environ_still_wins_over_dotenv Same class as #18757; happy to add to my fix PR if useful
ACP commands test_send_available_commands_update Command list ordering
Teams typing test_send_typing Mock not awaited
TUI pending_title test_session_create_drops_pending_title_on_valueerror ValueError no longer drops title

Happy to open targeted fix PRs for any of these clusters if it helps unblock the queue. Otherwise this PR is ready whenever main is green.

@teknium1

Copy link
Copy Markdown
Contributor

This appears to be implemented on current main now.

Automated hermes-sweeper review found that the same large-context error-classifier fix was salvaged and merged via #19723 after the duplicate discussion noted on this PR.

Evidence:

  • agent/error_classifier.py:705 gates the server-disconnect absolute fallback behind context_length <= 256000 while keeping the relative > 0.6 pressure check.
  • agent/error_classifier.py:1022 gates the generic-400 absolute fallback behind context_length <= 256000 while keeping the relative > 0.4 pressure check.
  • tests/agent/test_error_classifier.py:599 covers the exact 1M-context generic-400 repro shape and expects FailoverReason.format_error with no compression.
  • tests/agent/test_error_classifier.py:632 covers the 1M-context disconnect low-pressure shape and expects FailoverReason.timeout with no compression.
  • The merged salvage commit is d29f90e89d0263d390a71b359e1afa4f5a91e1e9 from fix(error_classifier): avoid large-context false overflow heuristics #19723, first contained in tag v2026.5.7.

Thanks for the original fix and for the CI analysis on the PR.

@teknium1 teknium1 closed this Jun 10, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generic 400/disconnect errors misclassified as context_overflow in 1M-context sessions

3 participants