Skip to content

fix(anthropic): use model-native output limits instead of hardcoded 16K - #3426

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-9420d6a3
Mar 27, 2026
Merged

fix(anthropic): use model-native output limits instead of hardcoded 16K#3426
teknium1 merged 1 commit into
mainfrom
hermes/hermes-9420d6a3

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

The Anthropic adapter hardcoded max_tokens=16384 as the default when the user didn't configure one explicitly. Since Anthropic's API requires max_tokens (unlike OpenAI where it's optional), this meant every direct-Anthropic session was capped at 16K output tokens — regardless of the model's actual capability.

This is the root cause of two frequently reported errors:

  • "Response truncated (finish_reason='length')" — thinking consumed most of the 16K budget, leaving little for the response
  • "Response only contains think block with no content after it" — thinking consumed ALL 16K, zero tokens left for any response

The problem in numbers

Model Actual max output What we sent Wasted capacity
Claude Opus 4.6 128,000 16,384 87%
Claude Sonnet 4.6 64,000 16,384 74%
Claude Sonnet 4.5 64,000 16,384 74%
Claude 3.5 Sonnet 8,192 16,384 (overcounted but harmless)

With adaptive thinking on Claude 4.6, the model self-manages its thinking budget within max_tokens. At 16K, it might think for 14K and have only 2K left for the response — or use all 16K on thinking and produce nothing.

What other agents do

Cross-referenced with Cline, Clawdbot, and OpenCode:

  • Cline: Uses the model's full output limit (64K Sonnet, 128K Opus) — sourced from a per-model catalog
  • OpenCode: Caps at min(model_limit, 32K) — still 2x our old default
  • Clawdbot: Defaults to 8192 via external library

None of them hardcode 16K for models that support 64-128K.

Changes

agent/anthropic_adapter.py:

  • Added _ANTHROPIC_OUTPUT_LIMITS — static lookup table with output limits for all Claude model families (sourced from Anthropic docs + Cline's model catalog)
  • Added _get_anthropic_max_output(model) — substring matching that handles date-stamped IDs and variant suffixes (:1m, :fast)
  • Unknown/future models default to 128K (future models won't have less capacity)
  • build_anthropic_kwargs now accepts optional context_length parameter — clamps max_tokens so it never exceeds the context window (protects custom endpoints with limited capacity)

run_agent.py:

  • Passes context_compressor.context_length to the adapter for clamping

Why this only affects direct Anthropic

OpenRouter users aren't affected because we don't send max_tokens through the chat_completions path — OpenRouter handles it. The Codex/OpenAI paths also don't send it when unconfigured. Only the Anthropic adapter required it (mandatory API field) and hardcoded 16K.

Test plan

  • 19 new tests: model lookup accuracy, date-stamped IDs, variant suffixes, unknown models, explicit override, context_length clamping
  • Updated existing test_default_max_tokens to expect model-native limits
  • Full suite: 6477 passed, 0 failed

Closes #2706

The Anthropic adapter defaulted to max_tokens=16384 when no explicit value
was configured.  This severely limits thinking-enabled models where thinking
tokens count toward max_tokens:

- Claude Opus 4.6 supports 128K output but was capped at 16K
- Claude Sonnet 4.6 supports 64K output but was capped at 16K

With extended thinking (adaptive or budget-based), the model could exhaust
the entire 16K on reasoning, leaving zero tokens for the actual response.
This caused two user-visible errors:
- 'Response truncated (finish_reason=length)' — thinking consumed most tokens
- 'Response only contains think block with no content' — thinking consumed all

Fix: add _ANTHROPIC_OUTPUT_LIMITS lookup table (sourced from Anthropic docs
and Cline's model catalog) and use the model's actual output limit as the
default.  Unknown future models default to 128K (the current maximum).

Also adds context_length clamping: if the user configured a smaller context
window (e.g. custom endpoint), max_tokens is clamped to context_length - 1
to avoid exceeding the window.

Closes #2706
@teknium1
teknium1 merged commit 6f11ff5 into main Mar 27, 2026
4 checks passed
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…6K (NousResearch#3426)

The Anthropic adapter defaulted to max_tokens=16384 when no explicit value
was configured.  This severely limits thinking-enabled models where thinking
tokens count toward max_tokens:

- Claude Opus 4.6 supports 128K output but was capped at 16K
- Claude Sonnet 4.6 supports 64K output but was capped at 16K

With extended thinking (adaptive or budget-based), the model could exhaust
the entire 16K on reasoning, leaving zero tokens for the actual response.
This caused two user-visible errors:
- 'Response truncated (finish_reason=length)' — thinking consumed most tokens
- 'Response only contains think block with no content' — thinking consumed all

Fix: add _ANTHROPIC_OUTPUT_LIMITS lookup table (sourced from Anthropic docs
and Cline's model catalog) and use the model's actual output limit as the
default.  Unknown future models default to 128K (the current maximum).

Also adds context_length clamping: if the user configured a smaller context
window (e.g. custom endpoint), max_tokens is clamped to context_length - 1
to avoid exceeding the window.

Closes NousResearch#2706
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…6K (NousResearch#3426)

The Anthropic adapter defaulted to max_tokens=16384 when no explicit value
was configured.  This severely limits thinking-enabled models where thinking
tokens count toward max_tokens:

- Claude Opus 4.6 supports 128K output but was capped at 16K
- Claude Sonnet 4.6 supports 64K output but was capped at 16K

With extended thinking (adaptive or budget-based), the model could exhaust
the entire 16K on reasoning, leaving zero tokens for the actual response.
This caused two user-visible errors:
- 'Response truncated (finish_reason=length)' — thinking consumed most tokens
- 'Response only contains think block with no content' — thinking consumed all

Fix: add _ANTHROPIC_OUTPUT_LIMITS lookup table (sourced from Anthropic docs
and Cline's model catalog) and use the model's actual output limit as the
default.  Unknown future models default to 128K (the current maximum).

Also adds context_length clamping: if the user configured a smaller context
window (e.g. custom endpoint), max_tokens is clamped to context_length - 1
to avoid exceeding the window.

Closes NousResearch#2706
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…6K (NousResearch#3426)

The Anthropic adapter defaulted to max_tokens=16384 when no explicit value
was configured.  This severely limits thinking-enabled models where thinking
tokens count toward max_tokens:

- Claude Opus 4.6 supports 128K output but was capped at 16K
- Claude Sonnet 4.6 supports 64K output but was capped at 16K

With extended thinking (adaptive or budget-based), the model could exhaust
the entire 16K on reasoning, leaving zero tokens for the actual response.
This caused two user-visible errors:
- 'Response truncated (finish_reason=length)' — thinking consumed most tokens
- 'Response only contains think block with no content' — thinking consumed all

Fix: add _ANTHROPIC_OUTPUT_LIMITS lookup table (sourced from Anthropic docs
and Cline's model catalog) and use the model's actual output limit as the
default.  Unknown future models default to 128K (the current maximum).

Also adds context_length clamping: if the user configured a smaller context
window (e.g. custom endpoint), max_tokens is clamped to context_length - 1
to avoid exceeding the window.

Closes NousResearch#2706
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…6K (NousResearch#3426)

The Anthropic adapter defaulted to max_tokens=16384 when no explicit value
was configured.  This severely limits thinking-enabled models where thinking
tokens count toward max_tokens:

- Claude Opus 4.6 supports 128K output but was capped at 16K
- Claude Sonnet 4.6 supports 64K output but was capped at 16K

With extended thinking (adaptive or budget-based), the model could exhaust
the entire 16K on reasoning, leaving zero tokens for the actual response.
This caused two user-visible errors:
- 'Response truncated (finish_reason=length)' — thinking consumed most tokens
- 'Response only contains think block with no content' — thinking consumed all

Fix: add _ANTHROPIC_OUTPUT_LIMITS lookup table (sourced from Anthropic docs
and Cline's model catalog) and use the model's actual output limit as the
default.  Unknown future models default to 128K (the current maximum).

Also adds context_length clamping: if the user configured a smaller context
window (e.g. custom endpoint), max_tokens is clamped to context_length - 1
to avoid exceeding the window.

Closes NousResearch#2706
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Response truncated due to output length limit

1 participant