Skip to content

fix(agent): include max_tokens in _resolve_runtime_agent_kwargs_for_provider return dict (#59763) - #59792

Open
webtecnica wants to merge 1 commit into
NousResearch:mainfrom
webtecnica:fix/59763-max-tokens-provider-routes
Open

fix(agent): include max_tokens in _resolve_runtime_agent_kwargs_for_provider return dict (#59763)#59792
webtecnica wants to merge 1 commit into
NousResearch:mainfrom
webtecnica:fix/59763-max-tokens-provider-routes

Conversation

@webtecnica

Copy link
Copy Markdown
Contributor

Descrição

_resolve_runtime_agent_kwargs_for_provider() não incluía max_tokens em seu dict de retorno, fazendo com que model.max_tokens do config.yaml, HERMES_MAX_TOKENS env var, e per-provider max_output_tokens fossem ignorados em qualquer rota com provider fixo (channel overrides, etc.).

A função irmã _resolve_runtime_agent_kwargs() já implementava essa resolução corretamente — _resolve_runtime_agent_kwargs_for_provider() simplesmente não tinha o bloco equivalente.

O que foi feito

Adicionada a mesma lógica de resolução de max_tokens que existe em _resolve_runtime_agent_kwargs() para _resolve_runtime_agent_kwargs_for_provider():

  1. HERMES_MAX_TOKENS env var — lida primeiro (priority mais alta)
  2. model.max_tokens do config.yaml — via _get_model_config()
  3. Per-provider max_output_tokens — fallback de custom_providers no runtime, apenas quando os anteriores não estão definidos
  4. Retornomax_tokens incluído no dict retornado

Arquivos alterados

  • gateway/run.py — +22 linhas em _resolve_runtime_agent_kwargs_for_provider()

Contexto adicional

O issue também mencionava _build_call_kwargs() em agent/auxiliary_client.py dropando max_tokens para OpenRouter no caminho auxiliar. Analisando o código, a omissão de max_tokens em _build_call_kwargs para providers não-Anthropic/NVIDIA é intencional (documentado no comentário das linhas 5870-5902) — omitir o parâmetro permite que a maioria dos providers use o limite máximo do modelo, que é o desejado para calls auxiliares (sumarização, títulos, etc.). Portanto o fix se concentra exclusivamente em _resolve_runtime_agent_kwargs_for_provider.

Closes #59763

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery area/config Config system, migrations, profiles P2 Medium — degraded but workaround exists labels Jul 6, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for tracing the pinned-provider path. The reported channel-override defect is present on current main: gateway/run.py:3870 replaces runtime_kwargs with _resolve_runtime_agent_kwargs_for_provider(), whose current return dict at gateway/run.py:1935-1953 omits max_tokens; the turn configuration then reads it at gateway/run.py:3946.

Problems

  • The same class remains on persisted /model rehydration. gateway/run.py:16201-16206 resolves the provider but does not retain max_tokens; the fast override route reads override.get("max_tokens") at gateway/run.py:3811.
  • Add regression coverage. tests/gateway/test_max_tokens_propagation.py:61-155 covers only the default resolver, not this provider-pinned route.

Suggested changes

  • Cover pinned-provider precedence (HERMES_MAX_TOKENS, model.max_tokens, per-provider fallback, and unset) and a channel override end-to-end through runtime resolution.
  • Carry the resolved cap through the persisted override rehydration path as well.

Automated hermes-sweeper review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

3 participants