Skip to content

fix: #38652 - detect OpenRouter/Nous "in the output" format in parse_available_output_tokens_from_error() - #38726

Closed
lfgogogo wants to merge 1 commit into
NousResearch:mainfrom
lfgogogo:fix/38652-openrouter-output-cap-parse
Closed

fix: #38652 - detect OpenRouter/Nous "in the output" format in parse_available_output_tokens_from_error()#38726
lfgogogo wants to merge 1 commit into
NousResearch:mainfrom
lfgogogo:fix/38652-openrouter-output-cap-parse

Conversation

@lfgogogo

@lfgogogo lfgogogo commented Jun 4, 2026

Copy link
Copy Markdown

Summary

When using OpenRouter-compatible providers (Nous Research inference, OpenRouter), the parse_available_output_tokens_from_error() function fails to detect "output cap too large" errors, causing an infinite auto-reset loop.

This fix adds regex-based detection for the OpenRouter/Nous error format:

maximum context length is 256000 … 5683 of text input, 13410 of tool input, 262000 in the output

It calculates available_output = context_length - text_input - tool_input, allowing Hermes to auto-reduce max_tokens and retry instead of falling into the input-compression recovery path (which fails on fresh sessions).

Before: "Session auto-reset" loops forever because /new doesn't fix the root cause (the max_tokens config value).

After: Hermes detects the output-cap error, reduces max_tokens, retries successfully, and warns the user.

Changes

  • agent/model_metadata.pyparse_available_output_tokens_from_error():
    • Broadened is_output_cap_error guard to also match \d+\s+in\s+the\s+output
    • Added OpenRouter/Nous regex extraction to compute available = context_length - text_input - tool_input
    • Anthropic-format extraction preserved as fallback

Closes #38652

🤖 Generated with Claude Code

…_output_tokens_from_error()

When using OpenRouter-compatible providers, max_tokens-too-large errors use
"X in the output" phrasing instead of Anthropic's "available_tokens" keyword.
This caused parse_available_output_tokens_from_error() to return None, falling
through to input-compression recovery (which fails on fresh sessions) and
triggering an infinite auto-reset loop.

Add regex-based detection for the OpenRouter/Nous error format to extract
context_length, text_input, and tool_input, then calculate available output
tokens as context_length - text_input - tool_input.

Closes NousResearch#38652

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openrouter OpenRouter aggregator provider/nous Nous Research API (OAuth) labels Jun 9, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #38659 (first competing open fix for #38652, same file/function parse_available_output_tokens_from_error() in agent/model_metadata.py). Several open alternates also exist: #38689, #39175, #39957. Consolidating on one fix would help the maintainer pick. Flagging for dedup; closure goes through review.

@teknium1

Copy link
Copy Markdown
Contributor

Closing as already implemented on main.

Automated hermes-sweeper review found the same OpenRouter/Nous output-cap parsing is already shipped:

  • agent/model_metadata.py:1055 now recognizes the OpenRouter/Nous "in the output" + "maximum context length" error shape.
  • agent/model_metadata.py:1086 computes available output from the breakdown: context_length - text_input - tool_input.
  • tests/test_output_cap_parsing.py:8 covers the OpenRouter/Nous breakdown format.
  • The implementation landed in 1fb99b1f229a700f8ee1a3e90ff1d44f85e963d4 (fix(stream+output-cap): guard empty streams and parse OpenRouter output-cap errors (#40589)) and is contained in v2026.6.19.

Thanks for the fix; the prior duplicate discussion was right that this needed consolidation, and the current main branch now covers the requested behavior.

@teknium1 teknium1 closed this Jun 21, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P1 High — major feature broken, no workaround provider/nous Nous Research API (OAuth) provider/openrouter OpenRouter aggregator sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: parse_available_output_tokens_from_error() misses OpenRouter/Nous "in the output" format — causes infinite auto-reset loop

3 participants