Skip to content

fix: accept max reasoning effort - #12211

Closed
SamuelZ12 wants to merge 1 commit into
NousResearch:mainfrom
SamuelZ12:fix/reasoning-max-support
Closed

fix: accept max reasoning effort#12211
SamuelZ12 wants to merge 1 commit into
NousResearch:mainfrom
SamuelZ12:fix/reasoning-max-support

Conversation

@SamuelZ12

Copy link
Copy Markdown
Contributor

summary

  • accept max as a valid reasoning_effort across Hermes parsing and command surfaces
  • update reasoning menus/help/docs so max appears anywhere users choose a reasoning level
  • add regression coverage for CLI parsing, gateway /reasoning max, and the reasoning effort picker order

root cause

The Anthropic adapter already supported Claude 4.7 output_config.effort = max, but the shared reasoning parser and command validators still only accepted up to xhigh. That made agent.reasoning_effort: max silently parse as invalid, so Hermes skipped sending adaptive reasoning fields even though downstream Anthropic code could handle them.

testing

  • /Users/samuelzhang/.hermes/hermes-agent/venv/bin/python -m pytest tests/cli/test_reasoning_command.py tests/gateway/test_reasoning_command.py tests/agent/test_anthropic_adapter.py tests/hermes_cli/test_reasoning_effort_menu.py -q -o 'addopts='

notes

  • verified locally after patch that Claude requests include thinking: {"type": "adaptive", "display": "summarized"} and output_config: {"effort": "max"} for claude-opus-4-7

Copilot AI review requested due to automatic review settings April 18, 2026 16:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Hermes’ reasoning-effort surfaces to accept max end-to-end (shared parsing, CLI + gateway commands, menus, and user-facing docs), closing a gap where downstream adapters supported max but upstream validation rejected it.

Changes:

  • Extend shared reasoning effort parsing/validation to include max and propagate it through CLI + gateway /reasoning flows.
  • Update UX surfaces (CLI command registry, reasoning effort picker ordering, docs) to list max as a valid option.
  • Add regression tests covering CLI parsing, gateway /reasoning max, and reasoning picker ordering.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
website/docs/user-guide/features/batch-processing.md Documents --reasoning_effort including max.
website/docs/user-guide/configuration.md Updates config docs to list max as a valid agent.reasoning_effort.
tests/hermes_cli/test_reasoning_effort_menu.py Adds test ensuring menu ordering places max after xhigh.
tests/gateway/test_reasoning_command.py Adds gateway regression test for /reasoning max persistence + cache update.
tests/cli/test_reasoning_command.py Extends CLI parsing test matrix to include max.
skills/autonomous-ai-agents/hermes-agent/SKILL.md Updates slash-command docs to include max.
hermes_constants.py Adds max to VALID_REASONING_EFFORTS and parser docstring.
hermes_cli/main.py Updates canonical reasoning effort ordering to include max.
hermes_cli/commands.py Adds max to /reasoning command subcommands for help/completions.
gateway/run.py Updates gateway /reasoning help + validation to accept max.
cli.py Updates CLI /reasoning help + “valid levels” output to include max.
batch_runner.py Extends batch-runner --reasoning_effort validation/docs to accept max.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cli.py
Comment on lines 6450 to 6453
Usage:
/reasoning Show current effort level and display state
/reasoning <level> Set reasoning effort (none, minimal, low, medium, high, xhigh)
/reasoning <level> Set reasoning effort (none, minimal, low, medium, high, xhigh, max)
/reasoning show|on Show model thinking/reasoning in output

Copilot AI Apr 18, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/reasoning help text now lists max, but the no-args status output later in this handler still prints Usage: /reasoning <none|minimal|low|medium|high|xhigh|show|hide> (missing max). Please update that usage hint too so all CLI help surfaces are consistent.

Copilot uses AI. Check for mistakes.
Comment thread gateway/run.py
Comment on lines 6371 to 6375

Usage:
/reasoning Show current effort level and display state
/reasoning <level> Set reasoning effort (none, minimal, low, medium, high, xhigh)
/reasoning <level> Set reasoning effort (none, minimal, low, medium, high, xhigh, max)
/reasoning show|on Show model reasoning in responses

Copilot AI Apr 18, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The handler docstring/usage now includes max, but the no-args response still returns _Usage:_ /reasoning <none|minimal|low|medium|high|xhigh|show|hide>`` (missing max). Please update that returned usage string as well so gateway help output matches the accepted levels.

Copilot uses AI. Check for mistakes.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/anthropic Anthropic native Messages API labels Apr 23, 2026
@counterposition

counterposition commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

The root cause analysis looks right: the Anthropic adapter already knows how to emit output_config.effort = max, and the shared parser/command surfaces were rejecting it too early.

A couple suggestions before merging to make this fully robust:

  1. Avoid leaking Anthropic-specific max to other transports.

VALID_REASONING_EFFORTS makes max globally accepted, but some non-Anthropic paths appear to forward reasoning_config["effort"] directly. For example, the Codex/Responses transport only clamps minimal -> low, so a user with agent.reasoning_effort: max who switches to a GPT/Codex model may send {"effort": "max"} to a backend that likely only supports up to high/xhigh. The generic Chat Completions OpenRouter/Nous path similarly forwards the config as-is.

I’d suggest normalizing max at the provider/model boundary, the same way the code already does for GitHub Models and Anthropic 4.6 xhigh -> max. For non-Anthropic providers, either clamp max to the strongest supported effort or omit it unless the provider explicitly advertises support.

  1. There are a few stale help strings after applying this on current main:
  • cli.py no-arg /reasoning usage still shows <none|minimal|low|medium|high|xhigh|show|hide>.
  • gateway/run.py no-arg /reasoning usage still shows <none|minimal|low|medium|high|xhigh|show|hide>.
  • gateway/platforms/discord.py slash-command description still says “none, minimal, low, medium, high, or xhigh.”
  1. It would be useful to add one regression test for the cross-provider behavior, e.g. reasoning_effort=max on a Codex/OpenAI Responses path should not produce an unsupported reasoning.effort=max unless that path explicitly supports it.

Overall the patch is pointed in the right direction; I’d just make the accepted user-level enum distinct from the provider-specific wire enum so model switches don’t turn a Claude-only maximum into invalid requests elsewhere.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for identifying the upstream validation gap. The gateway defect is still present on current main: gateway/slash_commands.py:2732 accepts only through xhigh, while hermes_constants.py:794-819 already accepts max.

Problems

  • The patch targets the pre-refactor handler in gateway/run.py; current gateway dispatch lives in gateway/slash_commands.py:2634, so this needs a conflict-aware salvage rather than applying the old hunk.
  • Provider projection is still needed. agent/transports/codex.py:155-166 only clamps minimal → low, so accepting max at the command/config layer can forward literal max to Codex/Responses.
  • The live CLI help now resides in hermes_cli/cli_commands_mixin.py:2469-2537 and still omits max.

Suggested changes

  • Port the gateway fix to gateway/slash_commands.py, update the live CLI/registry surfaces, and add a Codex wire-level regression that clamps or omits unsupported max values.

Automated hermes-sweeper review.

Comment thread hermes_constants.py
@@ -138,13 +138,13 @@ def get_subprocess_home() -> str | None:
return None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accepting max globally also reaches non-Anthropic transports. Please add provider-boundary normalization: current Codex handling only maps minimal → low, so it otherwise forwards max as reasoning.effort (agent/transports/codex.py:155-166).

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
@counterposition

Copy link
Copy Markdown
Contributor

Thanks for this contribution @SamuelZ12 — you correctly identified a real gap at the time: the Anthropic adapter already understood max, but the shared parser and command surfaces still rejected it.

This has since been implemented on main by #62650 (merged 2026-07-12), which added both max and a new ultra level across all the surfaces this PR touched — the shared validation constants, the gateway and CLI /reasoning commands, the batch runner, docs, and tests — plus provider-side handling so the new levels are mapped safely per backend.

Since there's nothing left for this branch to add, closing as implemented-on-main. That's a timing outcome, not a knock on the change — thanks again for the well-tested PR.

@teknium1

Copy link
Copy Markdown
Contributor

Closing — fully superseded on current main. You were the first to propose max as a reasoning effort level (April 18), and the feature has since landed piecemeal: max and ultra were added to VALID_REASONING_EFFORTS in #62650 (GPT-5.6 launch), and the remaining discoverability gap (help/status strings across CLI, gateway, and all 16 locales) was closed in #65651. Every surface this PR touched — constants, menus, batch runner, command registry, docs — now carries the extended levels.

Thanks for being early on this one; sorry it sat as long as it did.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants