Skip to content

fix(reasoning): align max and ultra controls with model limits - #61648

Closed
alanwilhelm wants to merge 5 commits into
NousResearch:mainfrom
alanwilhelm:feat/gpt56-max-ultra-reasoning
Closed

fix(reasoning): align max and ultra controls with model limits#61648
alanwilhelm wants to merge 5 commits into
NousResearch:mainfrom
alanwilhelm:feat/gpt56-max-ultra-reasoning

Conversation

@alanwilhelm

@alanwilhelm alanwilhelm commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Withdrawn: This broad aggregate is no longer the canonical review unit.
The focused provider-only fix is #61834. PR #61638 remains independent,
and the remaining ultra/Codex/UI work is not currently proposed here.

What does this PR do?

Aligns max and ultra across Hermes reasoning controls and prevents unsupported values from reaching provider APIs.

Hermes now uses the ordered contract minimal < low < medium < high < xhigh < max < ultra: when a provider or model does not support the requested level, request shaping selects the strongest advertised level at or below it. ultra remains a Hermes client level. Direct non-app-server requests normalize it to max before provider-specific projection; Codex app-server preserves literal ultra only when live catalog capabilities allow it, enabling native proactive delegation on GPT-5.6 Sol/Terra. This PR does not implement the public Responses Multi-agent beta.

Related Issue

Addresses #61634

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Security fix
  • Documentation update
  • Tests (adding or improving test coverage)
  • Refactor (no behavior change)
  • New skill (bundled or hub)

Changes Made

Controls and persistence

  • Use VALID_REASONING_EFFORTS for CLI/TUI autocomplete, CLI help/errors, setup ordering, gateway validation/status/errors, and dashboard delegation config.
  • Update stale non-WebUI lists in CLI, gateway, locale catalogs, Discord, batch flags, config examples, bundled skill docs, desktop controls, and English/Chinese user docs.
  • Preserve stored max and ultra values across desktop model controls, global defaults, delegation settings, and status labels.
  • Build batch reasoning config through the canonical parser. --reasoning_disabled and --reasoning_effort none use {"enabled": false}.

Provider and runtime projection

  • Add one ordered projection helper and use provider-advertised capabilities instead of fallback-to-medium branches.
  • Project Gemini 3 Pro/Flash high-end requests to high; LM Studio requests to the strongest advertised option; Anthropic legacy/manual max/ultra to the 32,000-token xhigh budget; and GitHub Models/Copilot/Kimi high-end requests to high when that is their ceiling.
  • Clamp direct Codex Responses efforts by model family: GPT-5/5.1 and unknown models to high; GPT-5.2+ and gpt-5.1-codex-max to xhigh; GPT-5.6 to max.
  • Clamp xAI Responses efforts: Grok 4.3/4.5 to high; Grok 4.20 multi-agent to xhigh; unsupported Grok models receive no effort field.
  • Make Codex app-server model/list authoritative when a selected model (including Hermes synthetic -pro aliases) has live capability evidence. Static family ceilings are used only when catalog evidence is unavailable.
  • Preserve the existing app-server thread. /reasoning none sends explicit JSON null to clear a prior effort override and resume the model default; it is not presented as fully disabling app-server reasoning.
  • Keep request shaping copy-on-write; stored config, prompts, history, tools, and prompt-cache keys are not mutated.

Existing work and overlap

, as required by the repository's contributor-attribution check for the follow-up fix commit.

Reviewer Map

Area Primary paths Review focus
Runtime hermes_constants.py, agent/codex_runtime.py, agent/model_metadata.py, agent/transports/codex*.py Ordered projection, direct Responses ceilings, catalog-first app-server behavior, JSON-null reset
Providers agent/transports/chat_completions.py, agent/lmstudio_reasoning.py, agent/anthropic_adapter.py, plugins/model-providers/{copilot,kimi-coding} Gemini, LM Studio, Anthropic manual budgets, GitHub/Copilot/Kimi capability projection
UI/config CLI, gateway, locale, batch, and desktop files in the earlier commits Global vocabulary/persistence only; WebUI-only #61625 surface remains out of scope
Docs Config/batch/app-server docs and bundled skill mirrors max/ultra vocabulary and accurate app-server reset semantics
Tests Touched transport/provider/runtime tests plus existing CLI/gateway/desktop suites Relationship invariants and provider wire payloads, not catalog snapshots
Housekeeping scripts/release.py Contributor email maps to the verified PR author identity

How to Test

Focused implementation and directly surrounding files:

env HERMES_HOME=/tmp/hermes-pr61648-tests-final \
  $HOME/.hermes/hermes-agent/venv/bin/python -m pytest \
  tests/test_hermes_constants.py \
  tests/agent/transports/test_chat_completions.py \
  tests/agent/test_anthropic_adapter.py \
  tests/agent/transports/test_codex_app_server_session.py \
  tests/plugins/model_providers/test_kimi_profile.py \
  tests/run_agent/test_codex_app_server_integration.py \
  tests/run_agent/test_run_agent_codex_responses.py \
  tests/run_agent/test_run_agent.py -q
# 1010 passed in 115.51s

env HERMES_HOME=/tmp/hermes-pr61648-tests-neighbors \
  $HOME/.hermes/hermes-agent/venv/bin/python -m pytest \
  tests/agent/test_model_metadata.py \
  tests/agent/transports/test_codex_transport.py \
  tests/providers/test_provider_profiles.py \
  tests/providers/test_profile_wiring.py \
  tests/providers/test_transport_parity.py \
  tests/run_agent/test_provider_parity.py \
  tests/hermes_cli/test_gpt56_registration.py -q
# 389 passed in 19.29s

Static checks for the 18 touched Python source/test files:

PY_FILES=(
  hermes_constants.py
  agent/anthropic_adapter.py agent/codex_runtime.py agent/lmstudio_reasoning.py
  agent/model_metadata.py agent/transports/chat_completions.py
  agent/transports/codex_app_server_session.py
  plugins/model-providers/copilot/__init__.py
  plugins/model-providers/kimi-coding/__init__.py run_agent.py
  tests/agent/test_anthropic_adapter.py
  tests/agent/transports/test_chat_completions.py
  tests/agent/transports/test_codex_app_server_session.py
  tests/plugins/model_providers/test_kimi_profile.py
  tests/run_agent/test_codex_app_server_integration.py
  tests/run_agent/test_run_agent.py
  tests/run_agent/test_run_agent_codex_responses.py
  tests/test_hermes_constants.py
)

$HOME/.hermes/hermes-agent/venv/bin/ruff check "${PY_FILES[@]}"
# All checks passed

$HOME/.hermes/hermes-agent/venv/bin/python -m py_compile "${PY_FILES[@]}"
git diff --check
# passed

CI-style ty comparison against incoming head 25bd6c4: 293 diagnostics on both sides, zero new issues. Independent read-only review of the readiness diff found no code blockers.

The contributor-attribution workflow logic passes locally after adding the required author mapping. ruff check scripts/release.py, python -m py_compile scripts/release.py, and git diff --check also pass.

The preceding head's unchanged UI slice also passed desktop typecheck, focused Vitest (14 tests), Prettier, and all required CI checks. Manual app-server readback on codex-cli 0.144.0 confirmed Sol/Terra keep ultra, Luna maps to max, and absent extended capability maps to the strongest supported lower effort.

Platform: Pop!_OS 24.04, Linux 6.18.7, x86_64.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits
  • I searched existing PRs and documented overlap above
  • My PR contains only reasoning-control, wire-safety, test, documentation, and required contributor-attribution changes
  • Focused and neighboring test slices pass
  • I've added regression tests for the changed behavior
  • I've tested on Pop!_OS 24.04, Linux 6.18.7, x86_64

Documentation & Housekeeping

  • Relevant config, command, batch, skill, desktop, and app-server docs are updated
  • cli-config.yaml.example is updated
  • CONTRIBUTING.md / AGENTS.md: N/A; no workflow or architecture contract changed
  • Cross-platform impact considered; no platform-specific I/O or process behavior changed
  • Tool descriptions/schemas: N/A

Screenshots / Logs

N/A. Command completion, config, transport payloads, and app-server protocol behavior are covered by tests and the manual readback above.

@alanwilhelm
alanwilhelm force-pushed the feat/gpt56-max-ultra-reasoning branch from 1d5e259 to e9c3474 Compare July 9, 2026 21:11
@alanwilhelm
alanwilhelm marked this pull request as draft July 9, 2026 21:25
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API labels Jul 9, 2026
@alanwilhelm
alanwilhelm force-pushed the feat/gpt56-max-ultra-reasoning branch from e9c3474 to 25bd6c4 Compare July 9, 2026 22:52
@alanwilhelm alanwilhelm changed the title feat(codex): support model-safe ultra reasoning fix(reasoning): align max and ultra controls with model limits Jul 9, 2026
@alanwilhelm
alanwilhelm marked this pull request as ready for review July 9, 2026 23:04
@alanwilhelm
alanwilhelm marked this pull request as draft July 10, 2026 00:06
@nullptr0807

Copy link
Copy Markdown
Contributor

I reproduced one remaining Copilot-specific gap on this branch: Telegram accepts /reasoning max, but gpt-5.6-sol still projects it to high because the request path uses the static GPT-5 effort list instead of the authenticated live catalog. I opened a supplement against your branch (not another upstream-main PR): https://github.com/alanwilhelm/hermes-agent/pull/2. It adds account-scoped catalog caching and verifies the live ladder none, low, medium, high, xhigh, max reaches the wire as max.

@alanwilhelm

Copy link
Copy Markdown
Contributor Author

Scope note: this PR is the single canonical unit for the max/ultra reasoning workstream — it includes the provider-capability projection slice (Gemini, LM Studio, Anthropic manual, GitHub/Copilot/Kimi, Codex) that briefly lived as #61772/#61834; both are closed-consolidated here. WebUI picker work remains with #61625 and the command surface with #61638 as before.

@alanwilhelm

Copy link
Copy Markdown
Contributor Author

Withdrawing this broad aggregate to reduce overlap and review burden. The focused provider-only fix is #61834. PR #61638 remains independent. The remaining ultra/Codex/UI work is preserved on the branch but is not currently proposed upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants