Skip to content

feat(ollama-cloud): add reasoning_effort support (xhigh->max) - #51482

Closed
kshitijk4poor wants to merge 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/29221-ollama-cloud-reasoning-effort
Closed

feat(ollama-cloud): add reasoning_effort support (xhigh->max)#51482
kshitijk4poor wants to merge 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/29221-ollama-cloud-reasoning-effort

Conversation

@kshitijk4poor

@kshitijk4poor kshitijk4poor commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Hermes can now send reasoning_effort to Ollama Cloud's OpenAI-compatible /v1/chat/completions endpoint — but only for models that actually support thinking — unlocking DeepSeek V4's "Max thinking" tier via reasoning_effort: "max". Previously the bare ollama-cloud profile silently dropped all reasoning config.

Salvage of #29221 by @s010mn onto current main (cherry-picked, authorship preserved), plus one follow-up commit from me with three correctness fixes — all verified live against ollama.com.

Changes

  • plugins/model-providers/ollama-cloud/__init__.py (@s010mn + my follow-up): OllamaCloudProfile overrides build_api_kwargs_extras to emit top-level reasoning_effort. xhigh/maxmax; low/medium/high pass through.
  • follow-up (me), all live-verified on deepseek-v4-pro / gemma3 / qwen3-coder:
    1. Capability-gated. Only models whose native /api/show capabilities list contains thinking get reasoning_effort. Resolved once per (model, base_url) in run_agent._supports_reasoning_extra_body via a cached probe (hermes_cli.models.ollama_model_supports_thinking), threaded into the profile as supports_reasoningno live HTTP in the per-request path. Mirrors the existing LM Studio capability pattern. The original ignored the flag and emitted for every model.
    2. Disable actually disables. Ollama Cloud defaults to thinking ON and ignores extra_body.thinking:{disabled} (verified). The only off switch is top-level reasoning_effort: "none". The original returned ({}, {}) on disable → thinking stayed on. Now emits {"reasoning_effort": "none"}.
    3. Omit unrecognized effort. The original forwarded unknown strings verbatim, including minimal (a real Hermes level). Ollama Cloud hard-400s unrecognized values (accepted set: {low, medium, high, max, none}). Now omitted → model default.

Why the core touches

run_agent.py + hermes_cli/models.py gain the capability probe (where reasoning-capability resolution already lives, e.g. LM Studio). The plugin profile only consumes the resolved supports_reasoning flag — it doesn't reach into core.

Validation (live against ollama.com)

model / input result
deepseek-v4-pro /api/show capabilities: [...,"thinking"] → gated ON
gemma3:27b, qwen3-coder:480b no thinking cap → no reasoning_effort
thinking + xhigh top-level reasoning_effort="max"
thinking + enabled:false reasoning_effort="none" (real off switch)
extra_body.thinking:{disabled} ignored (thinking stayed ON)
reasoning_effort:"ultra" HTTP 400 → justifies omitting unknowns
  • 24/24 profile tests pass; 194 provider/transport tests unaffected; ruff clean.
  • E2E verified through real imports + the real transport.

Credit

Salvaged from #29221 by @s010mn. Cherry-picked with authorship preserved; one follow-up commit on top.

Map Hermes xhigh→max to unlock DeepSeek V4's 'Max thinking' tier
through Ollama Cloud's OpenAI-compatible /v1/chat/completions endpoint.
low/medium/high pass through unchanged; disabled/none suppress
reasoning entirely.

Empirically confirmed: reasoning_effort:max produces ~2.5× more
thinking tokens than high on deepseek-v4-pro:cloud (1576 vs 642).
@alt-glitch alt-glitch added type/feature New feature or request comp/plugins Plugin system and bundled plugins provider/ollama Ollama / local models P3 Low — cosmetic, nice to have labels Jun 23, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: salvage of #29221 (@s010mn), part of the ollama-cloud reasoning_effort cluster alongside OPEN #30262 (superset: also adds DeepSeek thinking-mode) and the broader max-effort work in #49355 / #49644. Depends on the AUTHOR_MAP chore #51481 (merge first so check-attribution passes). Maintainer to pick the canonical ollama-cloud implementation from this cluster.

@kshitijk4poor
kshitijk4poor force-pushed the salvage/29221-ollama-cloud-reasoning-effort branch from 08f1064 to 0f36445 Compare June 23, 2026 18:33
… semantics

Three follow-up fixes to the salvaged reasoning_effort support, all verified
live against ollama.com /v1/chat/completions + /api/show on deepseek-v4-pro,
gemma3, and qwen3-coder:

1. Capability-gate on /api/show 'thinking'. The original ignored the
   supports_reasoning flag and emitted reasoning_effort for every model. Now
   gated: only models whose native /api/show capabilities list contains
   'thinking' (deepseek-v4 yes; gemma3 / qwen3-coder no) get reasoning_effort.
   Mirrors the LM Studio pattern — capability resolved once per (model,
   base_url) in run_agent._supports_reasoning_extra_body via a cached probe
   (hermes_cli.models.ollama_model_supports_thinking), threaded into the
   profile hook as supports_reasoning. No live HTTP in the per-request path.

2. Disable actually disables. Ollama Cloud defaults to thinking ON and IGNORES
   the extra_body.thinking:{type:disabled} shape (verified: still returned
   reasoning). The only working off switch is top-level reasoning_effort:'none'.
   The salvaged code returned ({}, {}) for enabled:false / effort:none, leaving
   thinking ON. Now emits {'reasoning_effort': 'none'}.

3. Omit unrecognized effort. The original forwarded any unknown string verbatim
   including 'minimal' (a real Hermes effort level). Ollama Cloud rejects
   unrecognized values with a hard HTTP 400 (accepted set: low/medium/high/
   max/none), so forwarding 'minimal' would break the request. Now omitted.

Core touches (run_agent.py, hermes_cli/models.py) add the capability probe;
the plugin profile only consumes the resolved flag. 24/24 profile tests green;
194 provider/transport tests unaffected.
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #65723 — your follow-up commit was cherry-picked onto current main with your authorship preserved in git history (5d9a72b7c). The base commit by @s010mn had already landed separately (221cd6024), so this salvage carried your three correctness fixes: capability gating, the real reasoning_effort:"none" off switch, and unknown-effort omission.

The live verification against ollama.com (especially discovering that extra_body.thinking:{disabled} is silently ignored) is exactly the kind of E2E rigor we want. Thanks!

@kshitijk4poor
kshitijk4poor deleted the salvage/29221-ollama-cloud-reasoning-effort branch August 5, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/ollama Ollama / local models type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants