Skip to content

fix(auxiliary): forward max_tokens to all providers, not just Anthropic/NIM (#60388) - #60454

Closed
kyssta-exe wants to merge 1 commit into
NousResearch:mainfrom
kyssta-exe:fix/60388-max-tokens-forward-all-providers
Closed

kyssta-exe wants to merge 1 commit into
NousResearch:mainfrom
kyssta-exe:fix/60388-max-tokens-forward-all-providers

Conversation

@kyssta-exe

Copy link
Copy Markdown
Contributor

Description

_build_call_kwargs() previously only included max_tokens in the API request when the endpoint was Anthropic-compat or NVIDIA NIM. For all other providers (OpenRouter, OpenAI direct, custom/local, Copilot, Nous), the value was silently dropped — so reference_max_tokens in MoA presets, title_generation caps, and any other caller-supplied max_tokens had no effect.

The call_llm function already has a retry mechanism that strips max_tokens when a provider rejects it (e.g. ZAI vision models with error 1210, OpenAI GPT-5 models that require max_completion_tokens), so forwarding max_tokens unconditionally is safe: providers that accept it respect the cap, providers that reject it fall through to the existing retry path (lines 6524-6540 of auxiliary_client.py).

Testing

  • Existing test suite covers provider fallback behavior when max_tokens is rejected
  • The retry mechanism at lines 6524-6540 handles ZAI vision error 1210, OpenAI GPT-5 max_completion_tokens requirements, and generic unsupported_parameter errors

Closes #60388

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/config Config system, migrations, profiles labels Jul 7, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Related to #60391 (also fixes #60388): this PR is the single-file _build_call_kwargs fix; #60391 makes the same agent/auxiliary_client.py change plus adds MoA per-slot max_tokens (moa_loop.py, moa_config.py, tests) and was opened earlier. Subset/superset competing PRs for the same issue — not a duplicate. Also related: #60008 (gateway pinned/rehydrate superset), #58261 (gemini-native-only). Maintainer to pick the canonical fix.

@kyssta-exe

Copy link
Copy Markdown
Contributor Author

Stale — 6-7 days without merge activity. Can resubmit if still needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: max_tokens settings across config.yaml are silently dropped

2 participants