Conversation
Per-provider max_output_tokens (providers/custom_providers) was applied by the gateway but silently dropped on the interactive CLI and oneshot (hermes chat / hermes -q), so the same config capped output differently across surfaces. Extract a shared resolve_effective_max_tokens() resolver enforcing the documented precedence (HERMES_MAX_TOKENS > model.max_tokens > per-provider max_output_tokens) and apply it in the gateway, the interactive CLI (_ensure_runtime_credentials), and oneshot (_run_agent). Ref: 14275d7 (fix(gateway): honor per-provider max_output_tokens in max_tokens chain), which established the precedence chain for gateway + CLI.
Slices 1 and 5 hit pre-existing flaky tests unrelated to this change: tests/tools/test_browser_hardening.py (agent-browser install subprocess timed out >30s) and tests/run_agent/test_run_agent.py::TestConcurrentToolExecution::test_concurrent_handles_tool_error (concurrency race; passes locally). No source or test changes.
|
Positive verification ✅ Reviewed the full diff — this is a clean, well-structured fix. What it does: Extracts Why the old behavior was a bug: The oneshot path ( Correctness checks:
Test coverage: Five tests across two files cover precedence, env override, bool rejection, oneshot forwarding, and interactive CLI. The oneshot test uses a No issues found. The extraction is minimal, the precedence logic is correct, and the tests cover the important edge cases. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating a real cross-surface configuration bug. The premise still holds on current main: hermes_cli/runtime_provider.py:592-603 preserves provider caps, while hermes_cli/oneshot.py:375-417 resolves that runtime but does not pass max_tokens to AIAgent.
Problems
- The interactive CLI edit is stale.
_ensure_runtime_credentialsmoved fromcli.pytohermes_cli/cli_agent_setup_mixin.py:25in094aa85c3; the PR patch no longer applies tocli.py. - The same loss remains for
/background:hermes_cli/cli_agent_setup_mixin.py:184-192omitsmax_tokensfrom the turn runtime, buthermes_cli/cli_commands_mixin.py:1644consumes that key for the background agent. - Gateway provider-override runtime construction also omits the cap at
gateway/run.py:1907-1925.
Suggested changes
- Move the CLI resolver call into
CLIAgentSetupMixin._ensure_runtime_credentialsand add the cap to the CLI turn runtime used by/background. - Add regression coverage for the background path and provider-override runtime path alongside the existing CLI/oneshot cases.
Automated hermes-sweeper review.
|
|
||
| # Honor a per-provider output cap (providers/custom_providers | ||
| # max_output_tokens) the same way the gateway does, so the same config | ||
| # caps output identically across surfaces. HERMES_MAX_TOKENS and the |
There was a problem hiding this comment.
Current main moved _ensure_runtime_credentials into hermes_cli/cli_agent_setup_mixin.py in 094aa85, so this cli.py hunk no longer applies. Please transplant this change to the mixin method during salvage.
What
Per-provider
max_output_tokens(set on aproviders:/custom_providers:entry)was honored by the gateway but silently dropped on the interactive CLI and oneshot
(
hermes chat/hermes -q). The same config therefore capped model outputdifferently depending on the surface.
Why
14275d7 (fix(gateway): honor per-provider max_output_tokens in max_tokens chain),
extended by 1c909e7, established the precedence chain
HERMES_MAX_TOKENS>model.max_tokens> per-providermax_output_tokensfor gateway + CLI. In practice only the gateway read
runtime["max_output_tokens"]:the interactive CLI set
max_tokensfrom env/global config only, and oneshot didn'tforward
max_tokensat all.Change
resolve_effective_max_tokens()resolver inhermes_cli/runtime_provider.pyas the single source of truth for the precedence.(
_ensure_runtime_credentials), and oneshot (_run_agent).Global
model.max_tokensandHERMES_MAX_TOKENSkeep priority; the per-providercap only fills in when no global cap is configured.
Tests
Added:
tests/hermes_cli/test_max_tokens_parity.py— resolver precedence(env / global / provider cap / none / invalid) + oneshot end-to-end
(
AIAgent.max_tokens == provider cap).tests/cli/test_cli_provider_resolution.py— CLI applies the provider cap;global still wins.
Results:
No regressions in the runtime-resolution suites.