Skip to content

fix(reasoning): project max to provider capabilities - #61834

Closed
alanwilhelm wants to merge 4 commits into
NousResearch:mainfrom
alanwilhelm:fix/reasoning-provider-projection-current-main
Closed

fix(reasoning): project max to provider capabilities#61834
alanwilhelm wants to merge 4 commits into
NousResearch:mainfrom
alanwilhelm:fix/reasoning-provider-projection-current-main

Conversation

@alanwilhelm

@alanwilhelm alanwilhelm commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes provider request-shaping regressions exposed after max became a valid Hermes reasoning effort. The change was re-derived from current upstream/main at 1a477697156cd50a3c2fc5d6a10af6ca89f02d06; each candidate defect was reproduced on that exact base before implementation.

The shared helper preserves exact supported efforts and otherwise selects the strongest supported lower tier. Provider-specific behavior remains at the request boundary, so the global configuration and command surfaces do not change.

This is the current-main, provider-only replacement for draft #61772 and the provider-projection slice of #61648. It deliberately excludes #61638 command/help/locales/Discord/docs work, global ultra, Codex app-server/runtime/catalog changes, desktop UI, aliases, global validation, Responses Multi-agent beta work, and contributor-map changes. Its only catalog addition is the account-scoped Copilot cache required to shape Copilot requests from authenticated live capabilities. It does not close or supersede #61638.

Related Issue

Related: #61638, #61648, #61772, and #51953.

No issue is closed by this PR.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Add an import-safe ordered projection helper beside VALID_REASONING_EFFORTS.
  • Project Gemini 3 Pro/Flash max to high for native and OpenAI-compatible payloads.
  • Project Anthropic legacy/manual max to the existing 32,000-token strongest budget.
  • Make LM Studio preserve advertised max, select advertised high when that is the ceiling, and retain the legacy medium fallback when catalog evidence is unavailable.
  • Project max through both Copilot request paths using the authenticated account catalog; gpt-5.6-sol retains max when the live ladder advertises it.
  • Cache Copilot catalogs by credential fingerprint with a 1-hour positive TTL and 60-second per-account negative TTL; authenticated failures never fall back to an anonymous catalog.
  • Project Kimi profile and legacy max to high while preserving the effort/thinking XOR wire contract.
  • Preserve TokenHub's production path unchanged; a negative-control test proves literal max never reaches its payload.

Contributor supplement

alanwilhelm/hermes-agent#2 by @nullptr0807 was merged into the fork as 915f38b6f27b488caabd9e75a26784831855d23d. Its original contributor commit 747a0a16192f89f19ce4ccd98afd122a3d3bceeb was folded here as 6926c806aa24d37c2885ffba0f8461e78dc3d73b with authorship preserved.

This overlaps the live-catalog direction in #51953; the supplement adds credential-scoped caching, prevents authenticated-to-anonymous fallback, and covers the GPT-5.6 max wire path.

Current-main reproduction before this change:

Path Before After
Gemini 3 Pro thinkingLevel: low thinkingLevel: high
Gemini 3 Flash thinkingLevel: medium thinkingLevel: high
Anthropic manual budget_tokens: 8000 budget_tokens: 32000
LM Studio, max advertised medium max
LM Studio, high ceiling medium high
Copilot chat reasoning omitted advertised xhigh
Copilot Responses medium advertised xhigh
Copilot GPT-5.6 Sol (live max account) static high ceiling authenticated live max
Kimi profile thinking toggle only reasoning_effort: high only
Kimi legacy medium plus thinking reasoning_effort: high only
TokenHub high unchanged high

Changed files:

agent/anthropic_adapter.py
agent/chat_completion_helpers.py
agent/lmstudio_reasoning.py
agent/transports/chat_completions.py
hermes_cli/models.py
hermes_constants.py
plugins/model-providers/copilot/__init__.py
plugins/model-providers/kimi-coding/__init__.py
run_agent.py
tests/agent/test_anthropic_adapter.py
tests/agent/transports/test_chat_completions.py
tests/hermes_cli/test_copilot_context.py
tests/run_agent/test_run_agent.py
tests/run_agent/test_run_agent_codex_responses.py

How to Test

  1. Run the direct Copilot/catalog/request-shaping suites:

    env HERMES_HOME=/tmp/hermes-pr61834-copilot-supplement-focused \
      scripts/run_tests.sh -j 4 \
      tests/hermes_cli/test_copilot_context.py \
      tests/agent/transports/test_chat_completions.py \
      tests/run_agent/test_run_agent.py \
      tests/run_agent/test_run_agent_codex_responses.py -q
    # 624 passed
  2. Run the model metadata and neighboring provider contracts:

    env HERMES_HOME=/tmp/hermes-pr61834-copilot-supplement-neighbors \
      scripts/run_tests.sh -j 4 \
      tests/agent/test_model_metadata.py \
      tests/hermes_cli/test_model_validation.py \
      tests/providers/test_provider_profiles.py \
      tests/providers/test_profile_wiring.py \
      tests/providers/test_transport_parity.py \
      tests/run_agent/test_provider_parity.py -q
    # 400 passed
  3. Run focused static, cross-platform, diff, and attribution checks:

    PY_FILES=(
      agent/chat_completion_helpers.py
      agent/transports/chat_completions.py
      hermes_cli/models.py
      plugins/model-providers/copilot/__init__.py
      run_agent.py
      tests/agent/transports/test_chat_completions.py
      tests/hermes_cli/test_copilot_context.py
      tests/run_agent/test_run_agent.py
      tests/run_agent/test_run_agent_codex_responses.py
    )
    
    $HOME/.hermes/hermes-agent/venv/bin/python -m ruff check "${PY_FILES[@]}"
    $HOME/.hermes/hermes-agent/venv/bin/python -m py_compile "${PY_FILES[@]}"
    $HOME/.hermes/hermes-agent/venv/bin/python scripts/check-windows-footguns.py --all
    git diff --check 362cc928ac6fb1003bbac4920b5dc0bab91a33e5..HEAD
    # passed; Windows check scanned 753 files

The contributor workflow's local equivalent passes: 175353+alanwilhelm@users.noreply.github.com resolves through the canonical noreply rule, and the existing AUTHOR_MAP entry maps iamgexin@qq.com to nullptr0807.

Checklist

Code

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — N/A; no user-facing surface changes
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A; no config keys changed
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A; neither changed
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — pure request shaping, plus the Windows footgun scan passes
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A; no model-tool surface changed

For New Skills

N/A.

Screenshots / Logs

N/A. The behavior is provider request shaping and is covered by payload assertions and the before/after reproduction above.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/anthropic Anthropic native Messages API provider/gemini Google Gemini (AI Studio, Cloud Code) provider/copilot GitHub Copilot (ACP + Chat) provider/kimi Kimi / Moonshot duplicate This issue or pull request already exists labels Jul 10, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Supersedes #61772 -- same author, same title, and the body describes this as the current-main provider-only replacement for that draft. Marking #61772 as the earliest-open canonical of the pair; please close whichever is stale. This is the narrow provider-projection slice of the broader #61648 (ultra + codex resolver + UI) and is distinct from #61638 (command surface).

@alanwilhelm

Copy link
Copy Markdown
Contributor Author

Consolidating into #61648 per author decision — the split was an artificial file-count scoping, and #61648's branch already carries this provider-projection work in full (commit 1693562) alongside its ultra/Codex scope. #61648 is the single canonical PR for this workstream (also supersedes #61772).

@alanwilhelm

Copy link
Copy Markdown
Contributor Author

Correction: reopening #61834 as the canonical provider-only draft. #61648 has been withdrawn and #61772 remains closed. This is the focused 10-file, one-commit provider-projection fix. No code or branch history changed, and no reviewers have been requested.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused provider-boundary fix. The reported regression remains present on current main: max is accepted globally in hermes_constants.py:794, but Gemini 3 maps it to medium in agent/transports/chat_completions.py:55-72, legacy Anthropic manual thinking defaults it to 8,000 tokens in agent/anthropic_adapter.py:2643-2644, and Copilot Responses has no max projection in run_agent.py:5422-5452.

The shared ordered projection is applied at those request-shaping boundaries, preserves provider-specific wire contracts such as Kimi's thinking/reasoning-effort XOR, and is covered by targeted payload assertions. Current main remains unimplemented for this behavior; GitHub reports the PR as cleanly mergeable and required checks are passing.

Automated hermes-sweeper review.

@alt-glitch alt-glitch removed the duplicate This issue or pull request already exists label Jul 10, 2026
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 11, 2026
@alt-glitch alt-glitch added comp/cli CLI entry point, hermes_cli/, setup wizard and removed sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades comp/cli CLI entry point, hermes_cli/, setup wizard labels Jul 11, 2026
@teknium1 teknium1 added the sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades label Jul 11, 2026
@alt-glitch alt-glitch removed the sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades label Jul 12, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Re-triage update: #61834 is now the canonical live PR for the max-reasoning provider-projection workstream. Both competing PRs are closed — #61648 (the broader ultra/Codex/UI superset) was withdrawn by the author and #61772 remains closed — and the author has reopened this as the focused provider-only draft. Relating to #61648, #61772, #62156, and #61638 (not a duplicate). Priority held at P3 (degraded-but-functional reasoning effort with an explicit-high workaround).

@alt-glitch alt-glitch added the comp/cli CLI entry point, hermes_cli/, setup wizard label Jul 12, 2026
@alanwilhelm
alanwilhelm force-pushed the fix/reasoning-provider-projection-current-main branch from a61cbc5 to 7726bd9 Compare July 12, 2026 20:01
@alt-glitch alt-glitch removed the comp/cli CLI entry point, hermes_cli/, setup wizard label Jul 12, 2026
@alanwilhelm alanwilhelm closed this by deleting the head repository Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/anthropic Anthropic native Messages API provider/copilot GitHub Copilot (ACP + Chat) provider/gemini Google Gemini (AI Studio, Cloud Code) provider/kimi Kimi / Moonshot sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants