feat(config): add per-model max_tokens overlay

[digitalbase]

If you use providers like openrouter, bedrock the single max_token setting isn't enough. It needs to be dynamic based on the model.

Summary

Adds a model.models.<id>.max_tokens overlay so a single profile can switch between models with different output-token ceilings without mutating the flat model.max_tokens fallback.

model:
  default: anthropic/claude-opus-4.6
  max_tokens: 8192              # fallback for unmatched models
  models:
    anthropic/claude-opus-4.6:
      max_tokens: 32768
    gpt-5.5:
      max_tokens: 65536

Resolution order is:

1. Explicit constructor/programmatic max_tokens
2. model.models.<active_model>.max_tokens
3. Flat model.max_tokens
4. Provider/model default

Why this is not a duplicate

Related PRs solve adjacent scopes, but not this specific config shape:

• feat(config): per-model context_length and provider_routing overrides #24495 establishes the same model.models.<id> overlay pattern for context_length and provider_routing, and explicitly notes max_tokens as a natural future extension. This PR is that focused extension for output-token caps.
• fix(agent_init): read max_tokens from custom_providers per-model config #28142 / feat: per-provider max_tokens via custom_providers models.<model>.max_tokens #28786 target custom_providers[].models.<model>.max_tokens, which helps custom-provider entries but not built-in providers like Bedrock/Anthropic/OpenAI or normal model switching within one profile.
• fix(config): propagate max_tokens from config.yaml to AI transport (#20741) #20769 covers flat/global model.max_tokens propagation, but not per-active-model overrides.

Changes

• agent/agent_init.py: resolve model.models.<active>.max_tokens before flat model.max_tokens, preserving constructor precedence and positive-int validation.
• gateway/run.py: include model.models in gateway agent cache-busting keys so edits to overlays rebuild cached agents.
• cli-config.yaml.example: document the per-model output-token overlay.
• Tests cover override wins, unmatched fallback, invalid per-model fallback, constructor precedence, and gateway cache invalidation.

[teknium1]

Thanks for the focused implementation and the coverage of precedence, invalid values, and gateway cache invalidation.

• This is an automated hermes-sweeper review.
• The PR's substance is a new user-facing model.models.<id>.max_tokens configuration surface (agent/agent_init.py, cli-config.yaml.example; commit c9dee52a71de).
• Hermes has a standing policy against user-facing max_tokens knobs, including per-model and config-based variants (max-tokens-knob). The configuration shape does not change that policy.
• The timeline's related feat(custom_providers): per-model max_tokens with switch/fallback re-resolution #35518 reference does not alter this policy match.

Please keep provider-native output ceilings rather than adding a configurable output-token cap. A separate focused fix for a concrete transport/provider bug that does not introduce a user-facing max_tokens setting can be proposed independently.

---

Closed as not-planned per standing maintainer policy (max-tokens-knob). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

[digitalbase]

Thanks for the focused implementation and the coverage of precedence, invalid values, and gateway cache invalidation.

• This is an automated hermes-sweeper review.
• The PR's substance is a new user-facing model.models.<id>.max_tokens configuration surface (agent/agent_init.py, cli-config.yaml.example; commit c9dee52a71de).
• Hermes has a standing policy against user-facing max_tokens knobs, including per-model and config-based variants (max-tokens-knob). The configuration shape does not change that policy.
• The timeline's related feat(custom_providers): per-model max_tokens with switch/fallback re-resolution #35518 reference does not alter this policy match.

Please keep provider-native output ceilings rather than adding a configurable output-token cap. A separate focused fix for a concrete transport/provider bug that does not introduce a user-facing max_tokens setting can be proposed independently.

Closed as not-planned per standing maintainer policy (max-tokens-knob). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

Great automation. And i agree. One small note is that it would be nice to link to the standing policy as part of this review. More context = better