fix(reasoning): probe Ollama thinking capability for local servers, not just ollama.com - #86197
Open
hefestocode-max wants to merge 1 commit into
Open
hefestocode-max wants to merge 1 commit into
hefestocode-max wants to merge 1 commit into
Conversation
…ot just ollama.com
A profile whose `fallback_providers` points at a local Ollama server running a
model without the `thinking` capability fails every request:
HTTP 400: "qwen2.5:7b" does not support thinking
Two defects combine to produce it.
**1. The capability gate never runs for local Ollama.**
`AIAgent._supports_reasoning_extra_body()` only probed `/api/show` when the
base URL host was `ollama.com`. A local server (`http://localhost:11434/v1`)
fell through to `if "openrouter" not in self._base_url_lower: return False`,
so the probe was never consulted — even though
`hermes_cli.models.ollama_model_supports_thinking()` is documented for Ollama
"Cloud or local", already normalises `/v1` to the native base, and is cached
per (model, base_url). Port 11434 is Ollama's default across every platform
and install method, so it is as reliable a signal as the ollama.com hostname.
**2. `CustomProfile` ignored the capability it was handed.**
`build_api_kwargs_extras()` absorbed `supports_reasoning` into `**ctx` and
never read it, emitting `reasoning_effort` unconditionally. The sibling
`ollama-cloud` profile already honours the flag; this brings `custom` in line.
Both fixes are required together. Fixing only (2) would be a regression:
because the gate always returned False for localhost, a local
thinking-capable model such as `deepseek-r1` would stop receiving
`reasoning_effort` at all.
Only the *enable* branch is gated. Measured against Ollama
/v1/chat/completions with qwen2.5:7b:
reasoning_effort="medium" -> HTTP 400 (does not support thinking)
reasoning_effort="none" -> HTTP 200
think=false -> HTTP 200
The endpoint rejects enabling thinking, not the presence of the field, so the
disable branch stays ungated — gating it would silently drop a user's explicit
"don't reason" whenever the probe is unavailable, leaving a thinking-capable
model reasoning against instructions.
Tests: local Ollama is probed for both outcomes and on several host forms; a
non-Ollama local server on another port is left untouched; ollama.com keeps
its behaviour; the disable branch survives a missing capability; and the
transport is pinned to keep forwarding `supports_reasoning`, since the
profile's fail-closed default is only safe while it does.
Contributor
fix(reasoning): probe Ollama thinking capability for local servers, not just ollama.com
|
13 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A profile whose
fallback_providerspoints at a local Ollama server running a model without thethinkingcapability fails every request:Because the failing entry was last in the fallback chain, this 400 also became the terminal error surfaced to the user, hiding the original failure that triggered the fallback in the first place.
Reproduction
Any turn with a non-empty
agent.reasoning_effortthat reaches this entry 400s.Cause
Two defects combine.
1. The capability gate never runs for local Ollama.
AIAgent._supports_reasoning_extra_body()only probed/api/showwhen the base URL host wasollama.com. A local server falls through toif "openrouter" not in self._base_url_lower: return False, so the probe was never consulted — even thoughhermes_cli.models.ollama_model_supports_thinking()is documented for Ollama "Cloud or local", already normalises/v1to the native base, and is cached per(model, base_url).2.
CustomProfileignored the capability it was handed.build_api_kwargs_extras()absorbedsupports_reasoninginto**ctxand never read it, emittingreasoning_effortunconditionally. The siblingollama-cloudprofile already honours the flag.Fix
Both parts are required together. Fixing only (2) would be a regression: because the gate always returned
Falsefor localhost, a local thinking-capable model such asdeepseek-r1would stop receivingreasoning_effortat all.Port 11434 is Ollama's default across every platform and install method, so it is as reliable a signal as the
ollama.comhostname is for Ollama Cloud.Only the enable branch is gated
Measured against Ollama
/v1/chat/completionswithqwen2.5:7b:reasoning_effort="medium"reasoning_effort="none"think=falseThe endpoint rejects enabling thinking, not the presence of the field. So the disable branch stays ungated: gating it would silently drop a user's explicit "don't reason" whenever the capability probe is unavailable, leaving a thinking-capable model reasoning against instructions.
Tests
New
tests/hermes_cli/test_ollama_local_reasoning_gate.py, plus additions to the custom-profile, transport and parity suites:localhost,127.0.0.1, LAN IP, with and without/v1)ollama.comkeeps its existing behavioursupports_reasoning, since the profile's fail-closed default is only safe while it does(
tests/plugins/model_providers/,tests/providers/,tests/agent/transports/, and the new file)No behaviour change for OpenRouter, LM Studio, GitHub Models, Nous Portal or Vercel routes.