fix(gateway): propagate max_tokens through pinned-provider and rehydrate paths

[Kewe63]

Summary

Three sibling paths were silently dropping the resolved max_tokens cap whenever a provider was pinned instead of taking the default env route. HERMES_MAX_TOKENS / model.max_tokens / per-provider max_output_tokens only reached the wire on the default route (#59763); every pinned-provider route sent requests uncapped.

This PR addresses all three paths in one consolidation by extracting the cap-resolution logic into a shared helper, with full regression test coverage and the explicit-caller-cap fix on the auxiliary-client side.

---

Bug Surface Per Path

Path	File	Symptom before
Channel override pinning a provider	gateway/run.py:_resolve_runtime_agent_kwargs_for_provider	Return dict omitted max_tokens → runtime_kwargs.get("max_tokens") was None for every turn
Persisted /model rehydration after restart	gateway/run.py:_rehydrate_session_model_override	Override dict carried api_key / api_mode / credential_pool / base_url but never max_tokens → fast-path read None for the lifetime of the session
Auxiliary caller cap on OpenAI-wire	agent/auxiliary_client.py:_build_call_kwargs	Explicit caller-supplied max_tokens silently dropped on every wire except Anthropic-Messages + NVIDIA NIM — no way to bound an auxiliary call to OpenRouter

---

This PR:

1. Extracts the cap resolution into _resolve_max_tokens_cap(runtime) — single source of truth, used by both provider-resolution paths.
2. Adds the same max_tokens to _rehydrate_session_model_override's override dict (second bug class).
3. Fixes _build_call_kwargs to forward caller-supplied explicit caps on OpenAI-wire / OpenRouter / custom, while preserving the documented default-cap-omitted behaviour for max_tokens=None (third bug class). The retry ladder at line 6288-6294 already strips a 400-rejected max_tokens, so the wire-rejection safety argument survives this change.
4. Adds regression tests covering every change of behaviour.

---

Changes

gateway/run.py

• Extracted helper _resolve_max_tokens_cap (~30 lines + docstring).
• Delegated from _resolve_runtime_agent_kwargs — drops inline resolution block, becomes a ~5-line call.
• Added "max_tokens" to _resolve_runtime_agent_kwargs_for_provider's return dict.
• Added override["max_tokens"] = runtime.get("max_tokens") to the rehydration block.

agent/auxiliary_client.py

• Second arg branch in _build_call_kwargs: max_tokens is not None blocks now have an else: kwargs["max_tokens"] = max_tokens for non-Anthropic / non-NVIDIA-NIM wires, so caller-supplied caps reach the wire.
• Comment updated to document the new contract.

tests/gateway/test_max_tokens_propagation.py — 3 new tests:

• test_pinned_provider_resolves_max_tokens_via_helper — _resolve_runtime_agent_kwargs() and _resolve_runtime_agent_kwargs_for_provider("openrouter") agree on every field, including max_tokens (regression for the helper unification).
• test_pinned_provider_env_var_wins — HERMES_MAX_TOKENS env var overrides model.max_tokens: 8192 on the pinned-provider route too.
• test_pinned_provider_no_cap_means_none — max_tokens is None when no cap is configured (same contract as the default route, preserves back-compat).

tests/agent/test_auxiliary_build_call_kwargs.py (new file) — 5 tests:

• test_default_omits_max_tokens — max_tokens=None → neither key in kwargs (preserves the documented quirk-sidestep behaviour on ZAI vision 1210, GitHub Copilot, GPT-5 needing max_completion_tokens).
• test_caller_supplied_cap_reaches_openrouter — caller cap reaches OpenRouter.
• test_caller_supplied_cap_reaches_openai_compat — caller cap reaches the custom OpenAI-compatible path.
• test_caller_supplied_cap_reaches_anthropic_compat — Anthropic-Messages branch continues to forward (was already correct, pinned so the new default-forwarding branch doesn't break it).
• test_caller_supplied_cap_reaches_nvidia_nim — NVIDIA NIM branch continues to forward (was already correct, empty-choices workaround).

---

How to Test

pytest tests/gateway/test_max_tokens_propagation.py tests/agent/test_auxiliary_build_call_kwargs.py -v

All 9 existing test_max_tokens_propagation.py tests still pass; the three new tests cover pinned-provider consistency. All five new _build_call_kwargs tests run against the real auxiliary_client via the standard _current_custom_base_url mock — no end-to-end fixtures needed.

---

Manual Reproduction (Before)

# config.yaml
model:
  provider: openrouter
  max_tokens: 8192

Channel override (e.g. Discord) pins provider: openrouter. Observer captures outbound request — max_tokens field is absent, request asks for the model's default maximum (e.g. 65,536 tokens). Account daily credit limit is exceeded; #59763.

After: outbound request carries max_tokens: 8192 on the default route AND the pinned-provider route AND the rehydrated override route. Auxiliary calls with call_llm(provider="openrouter", max_tokens=512) carry "max_tokens": 512.

---

Related

• Supersedes fix(agent): include max_tokens in _resolve_runtime_agent_kwargs_for_provider return dict (#59763) #59792 (single-path hotfix with duplicated logic and no tests).
• Retry ladder at agent/auxiliary_client.py:6288-6294 already strips 400-rejected max_tokens parameters, so the wire-rejection safety argument for omitting max_tokens by default is covered there — not at _build_call_kwargs.

---

Checklist

• Tests pass — 9/9 existing + 3 new in test_max_tokens_propagation.py, 5/5 new in test_auxiliary_build_call_kwargs.py
• Follows Conventional Commits
• Changes scoped to this fix only — 2 source files + 2 test files

---

Risk & Impact

Low. The extraction is behavior-preserving on the default route — _resolve_max_tokens_cap is the same logic, just shared. The two new propagation points (pinned-provider return dict, rehydration override dict) are additive fields that were previously None. The auxiliary-client fix only changes behavior for callers that explicitly pass max_tokens on non-Anthropic/non-NVIDIA-NIM wires — the default None case is unchanged.

Type: 🐛 Bug fix
Closes: #59763

[alt-glitch]

This was generated by AI during triage.

Related: fixes issue #59763 and supersets competing open PR #59792 (single-file, path 1 only, no tests). This PR is the broader consolidation covering all three drop sites (pinned-provider resolve, /model rehydration, auxiliary-client cap) via a shared helper. Both are open — a maintainer should pick the canonical fix.

[teknium1]

Thanks for consolidating the pinned-provider and auxiliary explicit-cap fixes. The reported omissions still exist on current main (gateway/run.py:1935-1953; agent/auxiliary_client.py:6410-6414).

Problems

• The rehydration addition does not cover an immediate /model switch. Both live override writers omit max_tokens at gateway/slash_commands.py:1643-1649 and gateway/slash_commands.py:1891-1897; the API-key fast path returns override.get("max_tokens") at gateway/run.py:3804-3824. Thus the first turn after a live switch remains uncapped.
• The added resolver tests do not cover that session-override flow (tests/gateway/test_max_tokens_propagation.py:201-272).

Suggested changes

• Carry the resolved cap into both live /model override writers (or resolve it in the fast path), then add live and rehydrated override-to-turn-runtime regression tests.
• Remove or exercise the unused _pinned_provider_returns helper in the new test file.

Automated hermes-sweeper review.

[teknium1]

This repairs only restart rehydration. The two live /model override writers still omit max_tokens (gateway/slash_commands.py:1643-1649 and :1891-1897), while the API-key fast path reads that missing value at gateway/run.py:3804-3824. Please cover the live path too so the first post-switch turn has the same cap as a rehydrated session.