feat(reasoning): derive OpenRouter reasoning support and effort clamping from catalog metadata (prime-agent#1258 port) - #84909
Merged
Conversation
…ning support and effort levels from catalog metadata OpenRouter's /v1/models entries advertise reasoning capability (supported_parameters + reasoning.mandatory/supported_efforts). Use that metadata as the primary gate in _supports_reasoning_extra_body instead of the hand-maintained vendor-prefix allowlist, which went stale one vendor at a time (nvidia/ missing -> #75386). Also clamp the requested effort to the nearest LOWER catalog-supported level in the OpenRouter profile so ultra/max against a high-capped route no longer 4xxes. Cache-only on the hot path: capabilities parse for free out of the existing fetch_openrouter_models() payload, a background warmer covers cold starts, and unknown models/offline catalogs fall back to the static prefix list unchanged.
magicbluesmoke
pushed a commit
to magicbluesmoke/hermes-agent
that referenced
this pull request
Aug 28, 2026
… cross-ref reasoning_effort Rebased onto current main and added a cross-reference to Hermes' higher-level reasoning_effort control, which landed after this PR was opened (NousResearch#84909 / NousResearch#58884), so readers know the server-side request knobs here are distinct from (and complementary to) that config.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/reasoning ultra(or any effort) on an OpenRouter model now works for every reasoning-capable vendor — including ones we never hardcoded — and never 4xxes on a route that caps out at a lower level. Hermes reads OpenRouter's live model-catalog reasoning metadata (supported_parameters+ per-modelreasoning.supported_efforts/mandatory) to decide whether to emit reasoning controls and to clamp the requested effort to the nearest supported level (always downward, never a silent cost escalation).Port of PrimeIntellect-ai/prime-agent#1258 ("derive reasoning levels from provider metadata"), adapted to Hermes' catalog/profile architecture.
How it fits what Hermes already has
_supports_reasoning_extra_body()(run_agent.py) gated OpenRouter reasoning on a hand-maintained vendor-prefix allowlist that went stale one vendor at a time —nvidia/missing is open issue [Bug]: reasoning_effort: none is ignored for NVIDIA models on OpenRouter #75386 with two competing 1-line PRs (fix(agent): reasoning_effort cannot disable Nemotron reasoning via OpenRouter — nvidia/ missing from allowlist (#75386) #75402/fix(agent): add nvidia/ to OpenRouter reasoning model prefixes (#75386) #75407);tencent/,xiaomi/were earlier same-class additions. Catalog metadata is now the primary gate; the static list survives as the fallback for unknown/unlisted models and offline catalogs, so nothing regresses when the catalog is unreachable. This fixes the CLASS (any future vendor) instead of patching the predicate one prefix at a time.build_api_kwargs_extraspassedreasoning_configthrough verbatim; routes that stop athigh4xx onultra/max(issue Custom providers crash with /reasoning ultra (HTTP 422: unknown variant) #80242 shape). It now clamps to the nearest LOWER catalog-supported level, logging the downgrade at debug. The Claude-adaptiveverbositypath (fix(anthropic): default new Claude models to the modern thinking contract #42991/reasoning_effort is a silent no-op for Claude Fable 5 (and 4.6+) over OpenRouter — verbosity never sent #43432) is untouched.Changes
hermes_cli/models.py:parse_openrouter_reasoning_capabilities()(entry normalization, trusts thereasoningobject only whensupported_parametersincludes"reasoning"), process-lifetime caps cache with a 60s failure TTL, cache-onlyopenrouter_model_reasoning_capabilities()(tri-state: supported / definitively-not / unknown), backgroundwarm_openrouter_reasoning_caps_async(), andclamp_reasoning_effort_to_supported()(nearest-lower semantics; unrecognized custom levels pass through).run_agent.py:_supports_reasoning_extra_body()consults catalog metadata first on the OpenRouter branch; unknown → existing static prefix list. Cold cache kicks the async warmer — the hot path never blocks on HTTP.plugins/model-providers/openrouter/__init__.py:_clamp_reasoning_to_catalog()applied to the emittedextra_body.reasoning.hermes_cli/models.py(fetch_openrouter_models): parses reasoning caps out of the catalog payload it already downloads, so picker/setup runs warm the cache for free.website/docs/user-guide/configuration.md: note under reasoning-effort docs.tests/hermes_cli/test_openrouter_reasoning_metadata.py: 30 tests — parsing, clamping table, cache/TTL/no-fetch-on-hot-path, metadata-first gate with static fallback (realAIAgentconstruction), and profile clamp wiring viaget_provider_profile("openrouter").Validation
tests/providers/test_profile_wiring.py+tests/hermes_cli/test_models.pytests/run_agent/test_run_agent.pytest_interruptible_anthropic_interrupt_never_closes_shared_client, missinganthropicpkg locally — fails identically on cleanmain)Cache-safety: no system-prompt or message-history interaction — this only shapes per-request
extra_body, same as the existing reasoning config path.Infographic