Skip to content

fix(desktop,web): filter reasoning efforts by model capabilities - #90801

Closed
higgs1729 wants to merge 1 commit into
NousResearch:mainfrom
higgs1729:fix/reasoning-effort-model-picker-capabilities-v2
Closed

higgs1729 wants to merge 1 commit into
NousResearch:mainfrom
higgs1729:fix/reasoning-effort-model-picker-capabilities-v2

Conversation

@higgs1729

Copy link
Copy Markdown

What does this PR do?

This is a focused current-main implementation of #85209.

Desktop and Dashboard model pickers now use exact per-model reasoning capability metadata when the serving catalog provides it, instead of always presenting Hermes' full reasoning ladder. Models without valid capability metadata retain the existing full-ladder fallback for backward compatibility.

The change intentionally excludes the unrelated sequential-tool-timeout changes that are present in PR #85246.

Related Issue

Fixes #85209

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • hermes_cli/inventory.py
    • Preserve valid, canonicalized supported_efforts from Nous/OpenRouter route catalogs in the model-options capability payload.
    • Preserve the existing fallback when capability metadata is absent or malformed.
  • hermes_cli/web_server.py and web/src/
    • Expose the same route capability data through /api/model/info and filter the Dashboard reasoning picker.
  • apps/desktop/src/app/shell/
    • Filter the main model catalog submenu to supported effort levels.
    • Resolve stale model presets to a deterministic supported level.
    • Keep none as the separate Thinking-off state.
    • Restore a real enabled level when turning thinking back on, including when the configured default is none.
  • apps/desktop/src/lib/ and web/src/lib/
    • Add shared filtering and fallback helpers with regression tests.
  • No auxiliary/submodel settings were changed.
  • No runtime wire clamping or Codex Multi-agent behavior was changed. Codex-specific capability discovery remains a separate follow-up; unknown Codex metadata uses the existing full-ladder fallback.

How to Test

  1. For a model whose serving catalog exposes a restricted effort set, open the Desktop model menu and verify that only the advertised levels appear in canonical order.
  2. Verify that Off is controlled separately and is hidden for reasoning-mandatory routes.
  3. Switch from a model with a stale/unsupported saved effort and verify that a supported level remains selected.
  4. Verify that a model with absent or malformed capability metadata retains the existing full ladder.

Focused verification performed on Windows 11:

Desktop Vitest: 17 passed
Dashboard Vitest: 8 passed
Dashboard typecheck: passed
Desktop lint: passed with pre-existing warnings
Dashboard lint: passed with pre-existing warnings
Python focused suite: 208 passed, 0 failed, 4 skipped
Git diff --check: passed

Commands included:

cd apps/desktop && npm run test:ui -- --run src/app/shell/model-catalog-menu.test.tsx src/app/shell/model-edit-submenu.test.tsx src/lib/reasoning-effort.test.ts
cd apps/desktop && npm run lint -- --no-warn-ignored
cd web && npm run test -- --run src/lib/reasoning-effort.test.ts
cd web && npm run typecheck
cd web && npm run lint -- --no-warn-ignored
HERMES_PYTHON=/c/hermes-data/hermes-agent/venv/Scripts/python.exe bash scripts/run_tests.sh tests/hermes_cli/test_inventory_reasoning_caps.py tests/hermes_cli/test_openrouter_reasoning_metadata.py tests/hermes_cli/test_nous_reasoning_metadata.py tests/hermes_cli/test_web_server.py
git diff --check

The full Desktop typecheck was also attempted but is currently blocked by pre-existing missing module/type declarations for blobatar/blob and blobatar/react in apps/desktop/src/sdk/index.ts; the error is outside this PR's changed files.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs and issues to make sure this isn't a duplicate
  • My PR contains only changes related to this fix
  • I've run pytest tests/ -q and all tests pass — focused tests were run instead; the full suite was not run
  • I've added tests for the changed behavior
  • I've tested on my platform: Windows 11

Documentation & Housekeeping

  • Relevant documentation update: N/A — this is a UI/API capability propagation fix with no new user configuration
  • cli-config.yaml.example update: N/A — no config keys changed
  • CONTRIBUTING.md or AGENTS.md update: N/A — no architecture/workflow change
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide
  • Tool descriptions/schemas update: N/A — no tool behavior changed

Screenshots / Logs

Focused test and lint results are summarized above. No screenshot is required for this metadata/filtering change.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/desktop Electron desktop app (apps/desktop/*) comp/dashboard Web dashboard / control panel UI (dashboard/, landing) labels Aug 20, 2026
@alt-glitch

Copy link
Copy Markdown

This was generated by AI during triage.

Related: #85246 implements the same model-capability picker behavior but bundles unrelated changes. #90801 is the focused current-main lane; maintainers should choose a canonical implementation.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks @higgs1729 — this is a clean implementation of #85209, but I live-tested the premise before deciding on either PR and the catalog data can't carry it. I sent every ladder level to 7 OpenRouter models (49 calls) and compared against each model's published supported_efforts: no unlisted level was rejected anywhere, and gpt-5.4 honored the unlisted max with the most reasoning tokens of any level. Aggregators translate effort names upstream rather than reject them, so supported_efforts describes the vendor's native vocabulary, not what the route accepts; filtering the picker by it would hide levels that work. Full table in #85246. The one real failure (none on reasoning-mandatory routes) is already hidden on main via can_disable_reasoning (#90412). Closing on that evidence, same verdict as #85246 / #85209.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/dashboard Web dashboard / control panel UI (dashboard/, landing) comp/desktop Electron desktop app (apps/desktop/*) P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model picker should show only the reasoning-effort levels a model actually supports

4 participants