Skip to content

fix(error_classifier): avoid large-context false overflow heuristics - #16352

Closed
JayGwod wants to merge 1 commit into
NousResearch:mainfrom
JayGwod:fix/error-classifier-large-context-heuristics
Closed

fix(error_classifier): avoid large-context false overflow heuristics#16352
JayGwod wants to merge 1 commit into
NousResearch:mainfrom
JayGwod:fix/error-classifier-large-context-heuristics

Conversation

@JayGwod

@JayGwod JayGwod commented Apr 27, 2026

Copy link
Copy Markdown

Summary

Fixes a large-context false-positive in agent.error_classifier: generic HTTP 400 responses and server disconnects should not be classified as context_overflow solely because a 1M-context session has many messages.

The previous heuristic used absolute fallbacks:

approx_tokens > 80000 or num_messages > 80
approx_tokens > 120000 or num_messages > 200

Those thresholds are useful proxies for smaller context windows, but they are too aggressive for explicitly large windows. A 1M-context session can have 432 messages and ~74K estimated tokens while still being far below the real budget.

This patch keeps the relative pressure checks for all models, but gates the absolute token/message-count fallbacks to smaller context windows (<= 256000).

Behavior covered

  • Generic 400 with approx_tokens=74320, context_length=1_000_000, num_messages=432 is now format_error, not context_overflow.
  • Server disconnect with the same low-pressure 1M context shape is now timeout, not context_overflow.
  • Existing smaller-window behavior remains covered by existing tests.

Test plan

RED before fix:

/home/ubuntu/.hermes/hermes-agent/venv/bin/python -m pytest \
  tests/agent/test_error_classifier.py::TestClassifyApiError::test_400_generic_many_messages_below_large_context_pressure_is_format_error \
  tests/agent/test_error_classifier.py::TestClassifyApiError::test_disconnect_many_messages_below_large_context_pressure_is_timeout \
  -v -o 'addopts='

Both tests failed with FailoverReason.context_overflow.

GREEN after fix:

/home/ubuntu/.hermes/hermes-agent/venv/bin/python -m pytest tests/agent/test_error_classifier.py -q -o 'addopts='
/home/ubuntu/.hermes/hermes-agent/venv/bin/python -m py_compile agent/error_classifier.py tests/agent/test_error_classifier.py
git diff --check

Result:

120 passed

Manual reproduction after fix:

FakeHTTP400 FailoverReason.format_error False False
Exception FailoverReason.timeout True False

Fixes #16351

Related: #14499, #14858, #14953, #15844, #6751

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Apr 27, 2026
@JayGwod
JayGwod force-pushed the fix/error-classifier-large-context-heuristics branch 3 times, most recently from a10d5f1 to 083fcb1 Compare April 28, 2026 21:09
@JayGwod

JayGwod commented May 2, 2026

Copy link
Copy Markdown
Author

Bumping — open for 5 days, no review activity.

Status on current upstream/main (f903ceece):

  • Applies cleanly, no rebase needed.
  • pytest tests/agent/test_error_classifier.py → 123 passed on a worktree at f903ceece with this commit on top.
  • Running on main for ~5 days; previously, large-context (~500K) responses were getting misclassified as transient overflow and triggering full-compression cycles. Hasn't recurred since the heuristic tightening.

Happy to clarify or rework the heuristic if there's a preferred boundary. Thanks!

@JayGwod
JayGwod force-pushed the fix/error-classifier-large-context-heuristics branch from 083fcb1 to fdadc26 Compare May 3, 2026 02:42
Generic 400 and server-disconnect heuristics used absolute token/message-count fallbacks that are too aggressive for 1M context sessions. Gate those absolute fallbacks to smaller context windows while preserving relative pressure checks.

Fixes NousResearch#16351
@teknium1

teknium1 commented May 4, 2026

Copy link
Copy Markdown
Contributor

Salvaged via #19723 onto current main - your commit authorship was preserved. Thanks!

@teknium1 teknium1 closed this May 4, 2026
@JayGwod
JayGwod deleted the fix/error-classifier-large-context-heuristics branch May 5, 2026 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generic 400/disconnect errors misclassified as context_overflow in 1M-context sessions

3 participants