Skip to content

fix(ollama): clamp reasoning for non-thinking local models - #81574

Draft
kh980975 wants to merge 1 commit into
NousResearch:mainfrom
kh980975:fix/ollama-nonthinking-reasoning
Draft

fix(ollama): clamp reasoning for non-thinking local models#81574
kh980975 wants to merge 1 commit into
NousResearch:mainfrom
kh980975:fix/ollama-nonthinking-reasoning

Conversation

@kh980975

@kh980975 kh980975 commented Aug 8, 2026

Copy link
Copy Markdown

Summary

  • Prevent persisted reasoning_effort values from breaking local Ollama models that do not support thinking.
  • Preserve requested reasoning for thinking-capable local Ollama models.
  • Preserve existing behavior for non-Ollama custom OpenAI-compatible endpoints.

Technical implementation details

  • Recognize Ollama reasoning routes by the explicit ollama provider, the genuine ollama.com host, or parsed port 11434 (without matching path fragments).
  • Reuse Ollama's cached /api/show capability probe for local models, rather than assuming every local model can reason.
  • When a local Ollama model reports no thinking capability, clamp a stale session-level effort to top-level reasoning_effort="none" and extra_body.think=false.
  • Leave thinking-capable Ollama models and non-Ollama custom endpoints on their existing reasoning paths.
  • Add provider-wire and runtime-detection regression coverage.

Testing steps

  • uv run --isolated --with ruff==0.15.10 ruff check plugins/model-providers/custom/__init__.py run_agent.py tests/plugins/model_providers/test_custom_profile.py tests/run_agent/test_run_agent.py
    • Result: all checks passed.
  • uv run --isolated --with pytest==9.1.1 --with 'anthropic>=0.39.0' python -m pytest tests/plugins/model_providers tests/run_agent/test_run_agent.py -q
    • Result: 418 passed, 1 warning.
    • The warning is an existing thread-fixture warning in test_direct_session_db_flushes_share_marker_claim; no test failed.
  • Manual local smoke with granite4.1:8b-hermes64k and a persisted medium effort returned HERMES_MEDIUM_CLAMP_OK instead of HTTP 400.

Related issues/tickets

KHAD
fix(ollama): clamp reasoning for non-thinking models
dd2b058 
- Detect local Ollama endpoints without matching path fragments
- Probe native model capabilities for local thinking support
- Force stale persisted effort to none for non-thinking models
- Preserve thinking-capable and non-Ollama custom endpoints
- Add provider and runtime regression coverage
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins provider/ollama Ollama / local models P3 Low — cosmetic, nice to have labels Aug 8, 2026
@lucas-alpaca lucas-alpaca mentioned this pull request Aug 26, 2026
Open
13 tasks
@alt-glitch alt-glitch mentioned this pull request Sep 11, 2026
Open
@alt-glitch alt-glitch added the area/local-models Local model inference/runtimes: llama.cpp, Ollama, LM Studio, MLX/vLLM, GGUF, VRAM and offload label Sep 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/local-models Local model inference/runtimes: llama.cpp, Ollama, LM Studio, MLX/vLLM, GGUF, VRAM and offload comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/ollama Ollama / local models type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Bug: CustomProfile (Ollama fallback) forwards reasoning_effort to non-reasoning models, causing HTTP 400 'does not support thinking'

2 participants

@kh980975 @alt-glitch