Repository navigation
fix(reasoning): hide unsupported effort choices in model pickers - #85246
fangliquanflq wants to merge 7 commits into
Conversation
Clarify waits on a human for up to 3600s or unlimited. The generic sequential timeout was aborting that wait at 420s and leaving the prompt and worker active.
fix(reasoning): hide unsupported effort choices in model pickers
|
|
Thanks @fangliquanflq — the direction is sound in principle, but I live-tested the premise before salvaging and the catalog data can't support it. Sent every ladder level (
No unlisted level was rejected anywhere; the only 400s were "reasoning is mandatory" on Closing on that evidence rather than merit. If a vendor's direct endpoint (not an aggregator) ever rejects unlisted levels, a per-provider send-path clamp ( |
What does this PR do?
Model pickers in Desktop and Dashboard currently offer the full reasoning-effort scale for every reasoning model, including levels the selected model cannot represent. This change preserves exact models.dev reasoning controls in the shared model-options payload and filters both pickers, preventing users from saving unsupported choices while keeping the existing full-list fallback for models without metadata.
Symptom
Selecting a model with a restricted reasoning scale, such as a model that supports only low, high, and max, still shows minimal, medium, xhigh, and ultra. The Off choice also appears even when the model exposes effort selection without toggle support.
Impact
Desktop and Dashboard users can select reasoning values that the active model does not support. Those choices are then folded or ignored downstream, so the picker advertises control that the model cannot honor.
Bug Cause
Trigger:
hermes_cli/inventory.py:404/_apply_capabilities, followed by the Desktop and Dashboard reasoning pickers.Causal chain:
reasoning_optionsfor a model.reasoningboolean.Why it is wrong: The backend discards the model-specific effort and toggle capabilities before either UI can use them.
Working sibling / contrast: Models without known metadata safely use the existing complete effort list. The new filtering applies only when exact capability metadata is available.
Ruled out: Provider request serialization is not the source of the picker mismatch. The unsupported choices are already visible before a request is sent, and tests reproduce the defect entirely in metadata parsing, payload shaping, and UI option filtering.
Fix
Parse models.dev effort and toggle options into shared model capabilities, expose them through
/api/model/options, and consume the same metadata in Desktop and Dashboard. The UI helpers preserve canonical ordering, omit Off when toggling is unsupported, select a supported fallback when a saved/default effort is invalid, and retain the full scale for unknown or older metadata.Related Issue
Closes #85209
Type of Change
Changes Made
agent/models_dev.py- preserve exact reasoning efforts and toggle support from models.dev metadata.hermes_cli/inventory.pyandhermes_cli/web_server.py- include model-specific reasoning controls in the shared model-options payload.apps/desktop/src/- filter model effort rows and toggle behavior with supported fallbacks.web/src/- filter Dashboard reasoning options from the same capability payload.tests/,apps/desktop/src/**/*.test.ts*, andweb/src/**/*.test.ts- cover metadata parsing, payload propagation, fallback compatibility, and both UI consumers.How to Test
Verified results: 36 Python tests passed, 9 Dashboard tests passed with Dashboard typecheck, and 15 Desktop tests passed.
Checklist
Code
fix(scope):,feat(scope):, etc.)Documentation & Housekeeping
cli-config.yaml.exampleupdate: N/A - no config keys changedCONTRIBUTING.mdorAGENTS.mdupdate: N/A - no architecture or workflow changedScreenshots / Logs
Automated behavior tests cover the exact supported-option sets, toggle visibility, supported fallback selection, and unknown-metadata compatibility.