Skip to content

fix(reasoning): probe Ollama thinking capability for local servers, not just ollama.com - #86197

Open
hefestocode-max wants to merge 1 commit into
NousResearch:mainfrom
hefestocode-max:fix/ollama-local-reasoning-capability
Open

hefestocode-max wants to merge 1 commit into
NousResearch:mainfrom
hefestocode-max:fix/ollama-local-reasoning-capability

Conversation

@hefestocode-max

Copy link
Copy Markdown

The bug

A profile whose fallback_providers points at a local Ollama server running a model without the thinking capability fails every request:

HTTP 400: "qwen2.5:7b" does not support thinking

Because the failing entry was last in the fallback chain, this 400 also became the terminal error surfaced to the user, hiding the original failure that triggered the fallback in the first place.

Reproduction

fallback_providers:
  - provider: custom
    model: qwen2.5:7b
    base_url: http://localhost:11434/v1

Any turn with a non-empty agent.reasoning_effort that reaches this entry 400s.

Cause

Two defects combine.

1. The capability gate never runs for local Ollama.

AIAgent._supports_reasoning_extra_body() only probed /api/show when the base URL host was ollama.com. A local server falls through to if "openrouter" not in self._base_url_lower: return False, so the probe was never consulted — even though hermes_cli.models.ollama_model_supports_thinking() is documented for Ollama "Cloud or local", already normalises /v1 to the native base, and is cached per (model, base_url).

2. CustomProfile ignored the capability it was handed.

build_api_kwargs_extras() absorbed supports_reasoning into **ctx and never read it, emitting reasoning_effort unconditionally. The sibling ollama-cloud profile already honours the flag.

Fix

Both parts are required together. Fixing only (2) would be a regression: because the gate always returned False for localhost, a local thinking-capable model such as deepseek-r1 would stop receiving reasoning_effort at all.

Port 11434 is Ollama's default across every platform and install method, so it is as reliable a signal as the ollama.com hostname is for Ollama Cloud.

Only the enable branch is gated

Measured against Ollama /v1/chat/completions with qwen2.5:7b:

request result
reasoning_effort="medium" HTTP 400 — does not support thinking
reasoning_effort="none" HTTP 200
think=false HTTP 200

The endpoint rejects enabling thinking, not the presence of the field. So the disable branch stays ungated: gating it would silently drop a user's explicit "don't reason" whenever the capability probe is unavailable, leaving a thinking-capable model reasoning against instructions.

Tests

New tests/hermes_cli/test_ollama_local_reasoning_gate.py, plus additions to the custom-profile, transport and parity suites:

  • local Ollama is probed for both outcomes, across several host forms (localhost, 127.0.0.1, LAN IP, with and without /v1)
  • the probe is asserted to actually be consulted — the previous behaviour silently skipped it, so a plain return-value assertion would have passed unchanged
  • a non-Ollama local server on another port is left untouched: this is Ollama-specific, not "any local endpoint"
  • ollama.com keeps its existing behaviour
  • the disable branch survives a missing capability
  • the transport is pinned to keep forwarding supports_reasoning, since the profile's fail-closed default is only safe while it does
514 passed

(tests/plugins/model_providers/, tests/providers/, tests/agent/transports/, and the new file)

No behaviour change for OpenRouter, LM Studio, GitHub Models, Nous Portal or Vercel routes.

…ot just ollama.com

A profile whose `fallback_providers` points at a local Ollama server running a
model without the `thinking` capability fails every request:

    HTTP 400: "qwen2.5:7b" does not support thinking

Two defects combine to produce it.

**1. The capability gate never runs for local Ollama.**
`AIAgent._supports_reasoning_extra_body()` only probed `/api/show` when the
base URL host was `ollama.com`. A local server (`http://localhost:11434/v1`)
fell through to `if "openrouter" not in self._base_url_lower: return False`,
so the probe was never consulted — even though
`hermes_cli.models.ollama_model_supports_thinking()` is documented for Ollama
"Cloud or local", already normalises `/v1` to the native base, and is cached
per (model, base_url). Port 11434 is Ollama's default across every platform
and install method, so it is as reliable a signal as the ollama.com hostname.

**2. `CustomProfile` ignored the capability it was handed.**
`build_api_kwargs_extras()` absorbed `supports_reasoning` into `**ctx` and
never read it, emitting `reasoning_effort` unconditionally. The sibling
`ollama-cloud` profile already honours the flag; this brings `custom` in line.

Both fixes are required together. Fixing only (2) would be a regression:
because the gate always returned False for localhost, a local
thinking-capable model such as `deepseek-r1` would stop receiving
`reasoning_effort` at all.

Only the *enable* branch is gated. Measured against Ollama
/v1/chat/completions with qwen2.5:7b:

    reasoning_effort="medium" -> HTTP 400 (does not support thinking)
    reasoning_effort="none"   -> HTTP 200
    think=false               -> HTTP 200

The endpoint rejects enabling thinking, not the presence of the field, so the
disable branch stays ungated — gating it would silently drop a user's explicit
"don't reason" whenever the probe is unavailable, leaving a thinking-capable
model reasoning against instructions.

Tests: local Ollama is probed for both outcomes and on several host forms; a
non-Ollama local server on another port is left untouched; ollama.com keeps
its behaviour; the disable branch survives a missing capability; and the
transport is pinned to keep forwarding `supports_reasoning`, since the
profile's fail-closed default is only safe while it does.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins provider/ollama Ollama / local models P3 Low — cosmetic, nice to have sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 14, 2026
@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference, author can ignore or act on any point.

fix(reasoning): probe Ollama thinking capability for local servers, not just ollama.com

  1. ":11434" in self._base_url_lower is a substring test — it also matches ports like :114340 or a URL where :11434 appears in a path/query segment. urlparse(self._base_url).port == 11434 would be an exact port match and avoids false positives probing a non-Ollama endpoint.
  2. test_transport_passes_supports_reasoning_to_profile uses inspect.getsource(chat_completions) and asserts on source text — this is the exact "never read source code in tests" anti-pattern in AGENTS.md: it passes when the wiring is subtly broken and fails on pure refactors. Replace with a behavioral test — e.g. invoke the real transport with a stub/recorded profile and assert supports_reasoning is forwarded (or assert the resulting kwargs), rather than grepping the source.
  3. The fix itself is well-reasoned: the enable branch is gated while the disable branch stays ungated, with empirical evidence (200 vs 400 against qwen2.5:7b) documented. The fail-closed default supports_reasoning=False is safe only because the single transport call site passes the resolved value explicitly — a comment noting that contract keeps it auditable.
  4. Minor: test_ollama_cloud_still_probed records seen == ["https://ollama.com/v1"] — since the probe result is also recorded for local URLs in the parametrized tests, the pattern is consistent; just noting the new test file and existing transport tests could be merged to reduce duplication.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/local-models Local model inference/runtimes: llama.cpp, Ollama, LM Studio, MLX/vLLM, GGUF, VRAM and offload comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/ollama Ollama / local models sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants