Skip to content

feat(reasoning): derive OpenRouter reasoning support and effort clamping from catalog metadata (prime-agent#1258 port) - #84909

Merged
teknium1 merged 1 commit into
mainfrom
prime-agent-port/openrouter-reasoning-metadata
Aug 13, 2026
Merged

teknium1 merged 1 commit into
mainfrom
prime-agent-port/openrouter-reasoning-metadata

Conversation

@teknium1

Copy link
Copy Markdown
Collaborator

Summary

/reasoning ultra (or any effort) on an OpenRouter model now works for every reasoning-capable vendor — including ones we never hardcoded — and never 4xxes on a route that caps out at a lower level. Hermes reads OpenRouter's live model-catalog reasoning metadata (supported_parameters + per-model reasoning.supported_efforts/mandatory) to decide whether to emit reasoning controls and to clamp the requested effort to the nearest supported level (always downward, never a silent cost escalation).

Port of PrimeIntellect-ai/prime-agent#1258 ("derive reasoning levels from provider metadata"), adapted to Hermes' catalog/profile architecture.

How it fits what Hermes already has

Changes

  • hermes_cli/models.py: parse_openrouter_reasoning_capabilities() (entry normalization, trusts the reasoning object only when supported_parameters includes "reasoning"), process-lifetime caps cache with a 60s failure TTL, cache-only openrouter_model_reasoning_capabilities() (tri-state: supported / definitively-not / unknown), background warm_openrouter_reasoning_caps_async(), and clamp_reasoning_effort_to_supported() (nearest-lower semantics; unrecognized custom levels pass through).
  • run_agent.py: _supports_reasoning_extra_body() consults catalog metadata first on the OpenRouter branch; unknown → existing static prefix list. Cold cache kicks the async warmer — the hot path never blocks on HTTP.
  • plugins/model-providers/openrouter/__init__.py: _clamp_reasoning_to_catalog() applied to the emitted extra_body.reasoning.
  • hermes_cli/models.py (fetch_openrouter_models): parses reasoning caps out of the catalog payload it already downloads, so picker/setup runs warm the cache for free.
  • website/docs/user-guide/configuration.md: note under reasoning-effort docs.
  • tests/hermes_cli/test_openrouter_reasoning_metadata.py: 30 tests — parsing, clamping table, cache/TTL/no-fetch-on-hot-path, metadata-first gate with static fallback (real AIAgent construction), and profile clamp wiring via get_provider_profile("openrouter").

Validation

Check Result
New tests 30/30 pass
Sabotage run (metadata gate no-op'd) 2 gate tests fail as expected, restore → green
tests/providers/test_profile_wiring.py + tests/hermes_cli/test_models.py 42/42 pass
tests/run_agent/test_run_agent.py 242 pass; 1 pre-existing failure (test_interruptible_anthropic_interrupt_never_closes_shared_client, missing anthropic pkg locally — fails identically on clean main)
Windows-footgun scan + ruff clean

Cache-safety: no system-prompt or message-history interaction — this only shapes per-request extra_body, same as the existing reasoning config path.

Infographic

OpenRouter reasoning metadata infographic

…ning support and effort levels from catalog metadata

OpenRouter's /v1/models entries advertise reasoning capability
(supported_parameters + reasoning.mandatory/supported_efforts). Use that
metadata as the primary gate in _supports_reasoning_extra_body instead of
the hand-maintained vendor-prefix allowlist, which went stale one vendor at
a time (nvidia/ missing -> #75386). Also clamp the requested effort to the
nearest LOWER catalog-supported level in the OpenRouter profile so ultra/max
against a high-capped route no longer 4xxes.

Cache-only on the hot path: capabilities parse for free out of the existing
fetch_openrouter_models() payload, a background warmer covers cold starts,
and unknown models/offline catalogs fall back to the static prefix list
unchanged.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/plugins Plugin system and bundled plugins provider/openrouter OpenRouter aggregator P3 Low — cosmetic, nice to have labels Aug 13, 2026
@teknium1
teknium1 merged commit 314968f into main Aug 13, 2026
49 checks passed
@teknium1
teknium1 deleted the prime-agent-port/openrouter-reasoning-metadata branch August 13, 2026 02:44
magicbluesmoke pushed a commit to magicbluesmoke/hermes-agent that referenced this pull request Aug 28, 2026
… cross-ref reasoning_effort

Rebased onto current main and added a cross-reference to Hermes'
higher-level reasoning_effort control, which landed after this PR was opened
(NousResearch#84909 / NousResearch#58884), so readers know the server-side request knobs here are
distinct from (and complementary to) that config.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/openrouter OpenRouter aggregator type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants