Skip to content

fix(agent): honor explicit model_overrides.supports_reasoning in the reasoning gate - #92768

Open
chelsealong wants to merge 1 commit into
NousResearch:mainfrom
chelsealong:fix/reasoning-gate-explicit-override
Open

chelsealong wants to merge 1 commit into
NousResearch:mainfrom
chelsealong:fix/reasoning-gate-explicit-override

Conversation

@chelsealong

Copy link
Copy Markdown
Contributor

What

AIAgent._supports_reasoning_extra_body() (run_agent.py) decides whether to emit
reasoning parameters purely by route — hostname/provider checks (Nous Portal, Vercel
AI Gateway, GitHub Models/Copilot, LM Studio probe, ollama.com probe, OpenRouter
model-prefix list). Everything else falls through to False.

That leaves operators on a self-hosted or unrecognized OpenAI-compatible endpoint
(e.g. a local Ollama server behind provider: custom) with no way to say "this model
supports reasoning" even though model_overrides.<provider>.<model>.supports_reasoning
is already a documented, first-class override field (agent/models_dev.py:860-884)
that get_model_capabilities() already consults elsewhere in the codebase. There were
no references to model_overrides anywhere in run_agent.py.

Fix

Consult the explicit override first, via the existing _explicit_model_override()
resolver in agent/models_dev.py (no new key space, no new machinery), before any
hostname/provider heuristic runs. Operator intent wins when set; everything falls
through to the unchanged existing route-based logic when it isn't. Only explicit
per-model overrides participate — the _default fill-gap entries are intentionally
not consulted here, matching the "operator intent wins" precedence the issue asks for.

This deliberately scopes to just the model_overrides half of the issue's proposal.
The issue's other ask — widening the Ollama /api/show probe beyond ollama.com to
any Ollama-compatible host — is already covered by other open PRs against this same
function (e.g. #86197, #83566) and is left untouched here to avoid overlapping edits
on the same lines.

Testing

Added two tests to TestSupportsReasoningExtraBody in tests/run_agent/test_run_agent.py:

  • test_explicit_override_enables_reasoning_on_unrouted_host — a provider: custom
    agent pointed at a local Ollama server (a host none of the existing route checks
    recognize) with supports_reasoning: true in model_overrides now reports
    reasoning-capable.
  • test_explicit_override_disables_reasoning_on_otherwise_capable_route — an explicit
    supports_reasoning: false overrides a route that would otherwise say True (a
    deepseek/ model on OpenRouter).

Confirmed both fail without the fix (git checkout HEAD~1 -- run_agent.py):

FAILED tests/run_agent/test_run_agent.py::TestSupportsReasoningExtraBody::test_explicit_override_enables_reasoning_on_unrouted_host - assert False is True
FAILED tests/run_agent/test_run_agent.py::TestSupportsReasoningExtraBody::test_explicit_override_disables_reasoning_on_otherwise_capable_route - assert True is False
2 failed, 1 passed, 274 deselected in 3.49s

And pass with it:

$ python -m pytest tests/run_agent/test_run_agent.py -k TestSupportsReasoningExtraBody -q
...                                                                      [100%]
3 passed, 274 deselected in 1.13s

Full existing suite, green:

$ python -m pytest tests/run_agent/test_run_agent.py -q
277 passed, 1 warning in 55.86s

$ python -m pytest tests/agent/test_models_dev.py -q
68 passed in 0.78s

$ ruff check run_agent.py tests/run_agent/test_run_agent.py
All checks passed!

Fixes #92759

AI assistance disclosure

This change was authored with the assistance of an AI coding agent (Claude).

…reasoning gate

_supports_reasoning_extra_body() decided whether to emit reasoning
parameters purely by route (hostname/provider), so an operator running
a self-hosted or unrecognized OpenAI-compatible endpoint had no way to
tell it their model does (or does not) support reasoning, even though
model_overrides.<provider>.<model>.supports_reasoning is a documented,
first-class override field already consulted by get_model_capabilities()
elsewhere in the codebase.

Consult the explicit override first, before any hostname/provider
heuristic, so operator intent always wins. Falls through to the
existing route-based logic unchanged when no explicit override is set.

Fixes NousResearch#92759
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/config Config system, migrations, profiles labels Aug 23, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference, author can ignore or act on any point.

Overall: clean precedence fix — an explicit model_overrides...supports_reasoning beating every hostname/family heuristic is the right authority order (the operator's declaration is strictly more informed than a route guess), and both directions are tested, including the valuable "explicit false disables an otherwise-capable route" case. Two small points:

  1. run_agent.py:7630-7634 — the bare except Exception: override = None makes config-loading failures indistinguishable from "no override configured": a broken models-dev cache or YAML typo silently reverts to heuristics and looks like the feature doesn't work. A logger.debug("...", exc_info=True) in the except branch (matching the style used elsewhere, e.g. the capability lookup in anthropic_prompt_cache_policy) costs nothing and saves real debugging time.

  2. Tests — consider one more case where _explicit_model_override returns an override without the supports_reasoning key, asserting fall-through to route heuristics. That pins the "supports_reasoning" in override presence check, which is the subtle half of the contract (partial overrides must not clobber unrelated capability decisions).

The bool() coercion of the raw YAML value correctly handles truthy/falsy spellings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Let the reasoning gate consult model capabilities (Ollama /api/show + model_overrides.supports_reasoning) for OpenAI-compatible endpoints

3 participants