fix: gate Ollama think/reasoning_effort on model's declared capabilities

[Tim-ImpendingTech]

Problem

The custom provider profile (provider=custom, covers local Ollama, vLLM, GLM/ARK, llama.cpp) unconditionally sends think/reasoning_effort on every request once agent.reasoning_effort is set to anything other than none.

Ollama models that don't declare thinking support — e.g. llama3.3:70b, whose /api/tags and /api/show capabilities are ["completion", "tools"] with no "thinking" entry — reject both fields outright:

HTTP 400: "llama3.3:70b" does not support thinking

This breaks every single turn for any non-thinking model as soon as the user configures a non-none reasoning_effort, which is the default for most setups. The only workaround today is disabling reasoning globally, even though the model works fine otherwise (including tool calling).

Fix

Ollama already publishes per-model capabilities via POST /api/show. This adds a small cached probe (_ollama_model_capabilities, permanent cache on success, 5-minute TTL on failure/empty) and gates think/reasoning_effort emission on "thinking" being present in that list.

The gate is scoped tightly:

• Only triggers when the configured base_url looks like Ollama (port 11434 or "ollama" in the host) — mirrors the existing _is_ollama_glm_backend detection pattern already used elsewhere in run_agent.py.
• Non-Ollama custom backends (vLLM / GLM on Volcengine ARK / llama.cpp) keep the original unconditional behavior — they don't expose an equivalent capability endpoint, and the existing wire-shape tests for those pin that contract.
• When no base_url is passed at all (e.g. delegation/summary paths that don't thread it through), behavior is unchanged from before this patch.

Testing

PYTHONPATH=$(pwd) venv/bin/python3.11 -m pytest tests/plugins/model_providers/ tests/providers/ -v
# 293 passed

Added 12 new tests in tests/plugins/model_providers/test_custom_profile.py:

• TestOllamaThinkingCapabilityGate — thinking vs non-thinking models, disabled path also gated, non-Ollama base_url bypasses the probe, no base_url bypasses the probe.
• TestOllamaCapabilityProbe — fetch/parse /api/show, /v1 suffix stripping so the probe hits the Ollama root not the OpenAI-compat path, failure caching, missing model/base_url short-circuits.
• TestIsOllamaBaseUrl — port 11434 matching, hostname matching, negative cases.

All 13 pre-existing tests in that file continue to pass unchanged (they don't pass base_url, so the new gate is a no-op for them — confirms the change is additive, not a behavior change for the documented GLM/ARK contract).

Also manually verified end-to-end against a real Ollama instance serving llama3.3:70b: confirmed /api/show reports ["completion", "tools"] (no thinking), confirmed the pre-patch 400 reproduces with agent.reasoning_effort: medium, confirmed post-patch hermes chat succeeds with reasoning enabled including a tool-calling round-trip.

Related

Builds on the reasoning wiring introduced for the custom provider in #57601 — see the docstring reference in the test file.

[teknium1]

Thanks for tracing this to the live CustomProfile path. Current main still emits the incompatible fields at plugins/model-providers/custom/__init__.py:52-58, and the live transport forwards base_url and model to this hook (agent/transports/chat_completions.py:582-590).

Problems

• plugins/model-providers/custom/__init__.py:70-74 maps every probe failure to [], and line 124 treats that exactly like a successful non-thinking declaration. A transient /api/show failure therefore suppresses a configured reasoning_effort for the five-minute negative-cache TTL, instead of preserving current behavior.

Suggested changes

• Return a distinct unknown/failure value from the probe and preserve existing emission when capability discovery is unavailable; suppress fields only after a successful capability response lacks thinking.
• Add a failed-probe wire-shape regression test alongside the existing success-path tests.

Automated hermes-sweeper review.

[teknium1]

_ollama_model_capabilities() returns [] both for a confirmed non-thinking model and for any exception. This makes a transient /api/show failure suppress configured reasoning for the negative-cache TTL. Please distinguish unknown from a successful empty declaration and retain the existing emission behavior when the probe is unavailable.

[GottZ]

This was generated by AI during triage.

Summary

Thirty-seven PRs address or reference this broad Ollama/provider complex, spanning Cloud provider/catalog behavior, response recovery, Windows support, and reasoning-control wire compatibility. For the target failure—non-thinking Ollama models rejecting think or reasoning_effort—#63315 is the recorded best fix because it consults /api/show capabilities, while adjacent PRs scope controls by endpoint or model-name heuristics.

Related pull requests

• feat(ollama): pass think=false to custom providers when reasoning_effort is none #3197 [closed] partial — (+51/-0) — n/a: Adds think=false for disabled reasoning to every custom endpoint; closed after cherry-pick into feat: Ollama improvements — Cloud provider, GLM continuation, think=false, surrogate sanitization, /v1 hint #10782, but later /v1/chat/completions evidence required top-level reasoning_effort=none too.
• fix(agent): sanitize surrogate characters from Ollama model output #5176 [closed] best fix — (+21/-2) — n/a: Sanitizes surrogates before requests, before storing assistant output, and in the actual retry payload; closed in favor of earlier duplicate fix(agent): sanitize surrogate characters from API responses and before API calls #5074.
• feat: add Ollama Cloud as built-in provider #6038 [closed] fixes — (+596/-50) — n/a: Adds first-class Ollama Cloud registration, routing, credentials, dynamic discovery, caching, auxiliary defaults, and tests; closed after salvage into merged feat: Ollama improvements — Cloud provider, GLM continuation, think=false, surrogate sanitization, /v1 hint #10782.
• fix(agent): continue Ollama GLM replies after stop misreports #10740 [closed] best fix — (+172/-0) — n/a: Reclassifies narrowly detected Ollama/GLM truncated stop responses after tool turns so the continuation path runs; closed after cherry-pick into feat: Ollama improvements — Cloud provider, GLM continuation, think=false, surrogate sanitization, /v1 hint #10782.
• feat: Ollama improvements — Cloud provider, GLM continuation, think=false, surrogate sanitization, /v1 hint #10782 [merged] best fix — (+822/-37) — n/a: Merged reference implementation consolidating Ollama Cloud support, GLM continuation, surrogate sanitization, think=false, and /v1 guidance; later PRs refine its generic-custom reasoning controls.
• fix(agent): stop sending think=false to remote custom providers #11296 partial — (+22/-7) — n/a: Gates think=false in the obsolete run_agent.py builder; the keep-open review requires salvaging the predicate into live CustomProfile code with remote and Ollama tests.
• fix(agent): restrict think=False to Ollama/local endpoints only #11362 partial — (+10/-2) — n/a: Uses URL, port, and locality heuristics in the obsolete builder; its keep-open verdict requires positive Ollama identification because local vLLM and llama.cpp are not Ollama.
• fix: only send think=False to Ollama endpoints (#11237) #11395 [closed] partial — (+34/-7) — n/a: Adds URL/port Ollama detection plus Mistral and Fireworks exclusions in the old builder; closed voluntarily to reduce duplicate volume.
• fix: only send think=false to Ollama, not all custom providers #11520 partial — (+8/-1) — n/a: Scopes think=false using ollama_num_ctx, URL text, or port 11434; its high-salvage keep-open verdict requires moving the guard to the live profile path.
• fix: restrict think=False to Ollama endpoints only #12488 [closed] partial — (+3/-1) — n/a: Adds an untested URL/port gate in stale run_agent.py; closed as a duplicate of fix(agent): restrict think=False to Ollama/local endpoints only #11362 but remains relevant as the same historical fix shape.
• fix(agent): scope think=False to Ollama endpoints only (#11237) #12914 [closed] partial — (+292/-3) — n/a: Adds probe/URL-based Ollama detection and broad old-path tests; closed after its author found that provider-profile dispatch had superseded the patched builder.
• fix(models): strip :cloud/-cloud suffix from models.dev Ollama Cloud IDs #16192 [closed] fixes — (+114/-3) — n/a: Normalizes models.dev :cloud/-cloud IDs before deduplication with focused tests; closed after exact salvage into merged fix(models): strip :cloud/-cloud suffix from models.dev Ollama Cloud IDs #19881.
• fix(cli): implement self-healing sanitization for Ollama Cloud model IDs #16196 [closed] fixes — (+261/-9) — n/a: Adds suffix normalization, deduplication, and legacy-cache repair; closed as superseded by fix(models): strip :cloud/-cloud suffix from models.dev Ollama Cloud IDs #19881, whose narrower merged fix covers the core merge defect.
• fix(windows): improve path normalization, shell execution, and encoding #16212 [closed] best fix — (+261/-9) — n/a: Despite its Windows title, the visible diff contains Ollama Cloud suffix/cache sanitization rather than path, shell, or encoding changes; closed under the contributor-documented consolidated Windows-support decision.
• fix(models): strip :cloud/-cloud suffix from models.dev Ollama Cloud IDs #19881 [merged] best fix — (+114/-3) — n/a: Merged recorded best fix for hosted Ollama Cloud suffix normalization and duplicate suppression, implementing fix(models): strip :cloud/-cloud suffix from models.dev Ollama Cloud IDs #16192 as commit eadf34633e038c595dcf615845b992a45443e380.
• fix(models): prioritize Ollama Cloud models #25174 related — (+53/-1) — n/a: Prioritizes Qwen 3.5 and Gemma 4 only after a fresh merge; the keep-open review requires the same ordering on fresh-cache and stale-cache returns.
• fix(agent): propagate reasoning_effort to Ollama + curator fork #25866 [closed] partial — (+23/-3) — n/a: Adds top-level reasoning_effort=none and curator-fork propagation; closed by its author, while fix(ollama): emit top-level reasoning_effort=none on /v1/chat/completions (#25758) #64608 and existing main code cover the two live defects.
• fix(ollama): emit top-level reasoning_effort=none on /v1/chat/complet… #29820 [closed] best fix — (+170/-5) — n/a: Recorded best fix for [Bug]: agent.reasoning_effort: none silently ignored on Ollama — main agent stuck in medium mode, bg-review fork can spiral (up to 65k tokens / 28 min) #25758, combining Ollama /v1 disablement with background-review inheritance; closed after its provider half was salvaged into fix(ollama): emit top-level reasoning_effort=none on /v1/chat/completions (#25758) #64608 and the fork half already existed on main.
• fix(ollama): add nemotron-3-ultra to model selector #42135 related — (+32/-0) — n/a: Injects nemotron-3-ultra beside existing Nemotron entries; the keep-open review requires cache-path coverage and evidence for this exception to dynamic discovery.
• Disable qwen3 thinking on local Ollama so tool-calling works (fixes #6152) #42201 [closed] fixes — (+132/-0) — n/a: Adds local auto-disable and explicit remote-Ollama request overrides; closed as implemented on main by 67df958dbe06bb10ca16b8686c76baf0de3bac03, with later PRs refining endpoint scope.
• fix: respect reasoning_config for custom providers and fix OpenRouter… #43613 [closed] partial — (+21/-1) — n/a: Makes legacy OpenRouter/custom reasoning honor configuration and forwards top-level effort; closed as implemented through the live profile path by 67df958dbe06bb10ca16b8686c76baf0de3bac03.
• fix(transports): promote Ollama reasoning field to content when content is empty #46489 best fix — (+66/-3) — n/a: Recorded best fix for Fix: Ollama reasoning models return empty content — send reasoning_effort to disable thinking #46131 because it extracts Ollama's separate reasoning field, but its keep-open review requires preserving reasoning separation or proving provider-scoped promotion end to end.
• fix: respect reasoning_config effort for custom providers #49607 [closed] partial — (+85/-1) — n/a: Reads configured effort and adds a config-based custom-model opt-in in legacy paths; closed after its author determined provider normalization prevented an end-to-end fix.
• feat(custom-provider): forward session_id as x-session-id HTTP header #52413 related — (+132/-2) — n/a: Adds x-session-id forwarding through CustomProfile with profile and transport tests; its keep-open review supports salvaging the header while omitting the stale custom:* lookup hunk.
• fix(ollama): preserve cloud model suffixes in picker #52613 partial — (+38/-18) — n/a: Preserves suffixed models.dev IDs while deduplicating by normalized keys; the keep-open review says this reverses hosted-API normalization without distinguishing hosted and local-proxy endpoints.
• fix(custom-provider): pass through reasoning_effort to extra_body for OpenAI-compatible backends #55280 [closed] partial — (+121/-2) — n/a: Emits enabled custom reasoning as extra_body.reasoning; closed because fix(custom-provider): forward reasoning_effort at the live profile path (GLM-5.2/ARK) #58156 implemented the umbrella issue on the live path using top-level reasoning_effort.
• fix: pass through reasoning_effort for custom/vLLM/Ollama providers (#55276) #55428 [closed] partial — (+13/-1) — n/a: Emits think=True plus effort and widens a legacy gate; closed because fix(custom-provider): forward reasoning_effort at the live profile path (GLM-5.2/ARK) #58156 implemented safer top-level effort without forcing the Ollama-specific enable flag.
• fix: forward reasoning_effort for custom providers (GLM-5.2 on ARK) #57601 [closed] partial — (+21/-2) — n/a: Adds max, removes hardcoded medium effort, and attempts custom forwarding in an unreachable legacy branch; closed after salvage into merged fix(custom-provider): forward reasoning_effort at the live profile path (GLM-5.2/ARK) #58156.
• fix(custom-provider): forward reasoning_effort at the live profile path (GLM-5.2/ARK) #58156 [merged] best fix — (+153/-11) — n/a: Merged recorded best fix for reasoning_effort / thinking_budget silently dropped for custom and zai providers #55276's custom-provider facet, forwarding enabled effort through live CustomProfile; later reports expose incompatibility with heterogeneous endpoints and plain models.
• fix(custom-profile): gate reasoning_effort on known reasoning-capable models (#59660) #59678 duplicate — (+187/-4) — n/a: Gates enabled effort through a model-name allowlist to prevent plain-model 400s; its keep-open review identifies unsafe unbounded o1/o3/o4 matching and missing transport-level coverage.
• fix(ollama-cloud): preserve :cloud tags for local-daemon proxy in model catalog #59819 best fix — (+108/-44) — n/a: Recorded best fix for [Bug]: Ollama Cloud model picker shows local models instead of cloud models #52599, shaping Cloud IDs differently for hosted and local-daemon endpoints; its keep-open review requires endpoint-aware caching to prevent cross-mode contamination.
• fix: gate Ollama think/reasoning_effort on model's declared capabilities #63315 best fix — (+260/-5) — n/a: Recorded best fix for Bug: CustomProfile (Ollama fallback) forwards reasoning_effort to non-reasoning models, causing HTTP 400 'does not support thinking' #59660, using Ollama /api/show capabilities rather than model-name guesses; its high-salvage keep-open review requires probe failure to remain unknown instead of being treated as a successful non-thinking declaration.
• fix(providers): scope think to Ollama endpoints #64286 best fix — (+245/-58) — n/a: Recorded best fix for [Bug]: think=False incorrectly sent to all provider=custom endpoints, not just Ollama #11237, Bug: CustomProfile sends reasoning_effort="none" to remote APIs, causing HTTP 400 #65233, and custom provider: unconditional top-level reasoning_effort is fatal on OpenAI-compatible proxies that reject unknown params #72649 by positively probing /api/tags and withholding both disable fields from generic relays; its keep-open path is orthogonal to fix: gate Ollama think/reasoning_effort on model's declared capabilities #63315's per-model capability gate.
• fix(ollama): emit top-level reasoning_effort=none on /v1/chat/completions (#25758) #64608 [merged] best fix — (+21/-5) — n/a: Merged recorded best fix for Ollama /v1 disablement, emitting top-level reasoning_effort=none alongside think=false; it remains the reference implementation salvaged from fix(ollama): emit top-level reasoning_effort=none on /v1/chat/complet… #29820.
• fix(custom-profile): omit reasoning_effort="none" for remote endpoints #65234 [closed] partial — (+202/-20) — n/a: Uses locality plus a custom reasoning_disable setting to omit unsupported disable fields; closed but relevant as an alternative rejected by review because locality does not establish endpoint capability.
• fix(custom): move reasoning_effort to extra_body for custom providers #72656 best fix — (+19/-31) — n/a: Recorded best fix for the LiteLLM enabled-effort 400 by actually omitting reasoning_effort rather than moving it into SDK extra_body; the keep-open review requires the test contract to match that omission.
• fix(responses): preserve explicit reasoning none #75234 best fix — (+96/-1) — n/a: Recorded best fix for explicit none on OpenAI Responses, emitting reasoning.effort=none only for known GPT-5.6 OpenAI/Codex routes and rejecting unknown relays; the author addressed the keep-open endpoint-capability review with route tests.

Duplicates

#3197 is incorporated into #10782; #6038 and #10740 were salvaged into #10782; #11296, #11362, #11395, #11520, #12488, and #12914 are obsolete-path variants of the #11237 fix now represented on the live path by #64286; #16192 and #16196 are superseded by #19881; #25866 and #29820 overlap, with #29820's provider half salvaged into #64608; #55280, #55428, and #57601 are superseded by #58156; #59678 is an alternative to target #63315; #52613 overlaps #59819 but lacks its endpoint-specific model shaping.

Suggested consolidation

For #63315, author action: distinguish /api/show probe failure/unknown from a successful capability list lacking thinking, preserve existing emission on unknown, suppress controls only on confirmed non-thinking models, and add the failed-probe final-wire regression requested by the explicit keep-open review. Keep #63315 open as the recorded best fix and close #59678 as its weaker model-name-gate duplicate once that change lands; retain #64286 separately for endpoint-level scoping, while #46489, #59819, #72656, and #75234 remain on their documented salvage paths rather than being folded into this model-capability fix.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I59660(["issue #59660 (open)"])
    subgraph Dup59678 ["PRs duplicating each other"]
        P59678["PR #59678 (open)"]
        P63315["PR #63315 (open)"]
    end
    P63315 -->|best fix| I59660
    class I59660 open
    class P59678 open
    class P63315 open
    class P63315 best
    class P63315 target
    click I59660 "https://github.com/NousResearch/hermes-agent/issues/59660"
    click P59678 "https://github.com/NousResearch/hermes-agent/pull/59678"
    click P63315 "https://github.com/NousResearch/hermes-agent/pull/63315"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 37 pull requests and 16 issues in this complex. Each diff was read against this issue; Assessment working set: 331 kB of PR diffs, 187 kB of issue/PR text, 71 kB of discussion (93 comments), 74 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[kshitijk4poor]

Closing — the think half of this PR (the #11237 Mistral/Groq 422) landed via #98000 (built from #97984 by @xxxigm).

The /api/show capabilities gating is a different symptom (non-thinking Ollama models 400-ing on thinking fields) and is better landed as its own PR built against the current profile-plugin architecture — it would be a meaningful follow-up to #98000, and the cached-probe design you built here is a strong starting point for it. Happy to see that re-proposed standalone.