Skip to content

fix: pass through reasoning_effort for custom/vLLM/Ollama providers (#55276) - #55428

Closed
Stoltemberg wants to merge 1 commit into
NousResearch:mainfrom
Stoltemberg:fix/reasoning-effort-custom-provider
Closed

fix: pass through reasoning_effort for custom/vLLM/Ollama providers (#55276)#55428
Stoltemberg wants to merge 1 commit into
NousResearch:mainfrom
Stoltemberg:fix/reasoning-effort-custom-provider

Conversation

@Stoltemberg

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #55276 — enables reasoning control for custom providers (vLLM, Ollama, local models) by passing through reasoning_effort config instead of silently dropping it.

Related Issue

Fixes #55276

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Changes Made

1. CustomProfile.build_api_kwargs_extras()

When reasoning is enabled, now sends:

  • think=True (vLLM/Ollama convention for enabling chain-of-thought)
  • reasoning_effort=<level> (pass through low/medium/high/xhigh)

Previously only think=False was sent when reasoning was disabled.

2. _supports_reasoning_extra_body()

Added check for custom/ollama/vllm local providers so reasoning extra_body is not gated out by the OpenRouter domain check.

How to Test

  1. Configure a custom provider with reasoning:
    provider: custom
    base_url: http://localhost:8000/v1
    agent:
      reasoning_effort: high
  2. Verify the API call includes think=True and reasoning_effort=high

Platforms Tested

  • Windows 11

…ousResearch#55276)

Two changes to enable reasoning control for custom providers:

1. CustomProfile.build_api_kwargs_extras(): When reasoning is enabled,
   send think=True (vLLM/Ollama convention) and pass through the
   effort level. Previously only think=False was sent when disabled.

2. _supports_reasoning_extra_body(): Return True for custom/ollama/vllm
   local providers so reasoning extra_body is not gated out by the
   OpenRouter domain check.

Fixes NousResearch#55276.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins provider/ollama Ollama / local models P3 Low — cosmetic, nice to have labels Jun 30, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Competing fix for #55276 alongside open PR #55280. This PR touches both the plugin profile layer (plugins/model-providers/custom/__init__.py) and the core gate (run_agent.py _supports_reasoning_extra_body), using the think=True + reasoning_effort wire format; #55280 fixes only the profile layer using extra_body.reasoning={enabled,effort}, and #20594 addresses the core gate. Same goal, different mechanism/scope — related, not a duplicate; maintainer should pick one wire format.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary\n\nVerdict: LGTM\n\nPasses through reasoning_effort for custom/vLLM/Ollama providers instead of silently dropping it. 2-file change, +13/-1 lines.\n\n### Looks Good\n- Small, focused fix for a real feature gap\n- Sends think=True and reasoning_effort= for local providers\n- Consistent with the existing vLLM/Ollama convention\n---\nReviewed by Hermes Agent

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: LGTM

Passes through reasoning/thinking support for custom/vLLM/Ollama providers. Adds think=True/False and reasoning_effort to extra_body when reasoning_config is set. Also marks custom/vLLM/Ollama/local providers as reasoning-capable in _supports_reasoning_extra_body. Clean fix for a missing passthrough.

Verdict: LGTM -- correct reasoning support extension.

@Stoltemberg

Copy link
Copy Markdown
Contributor Author

Thanks for flagging the competing PR #55280. A quick comparison:

This PR is the more complete fix — it ensures custom/vLLM/Ollama providers are correctly marked as reasoning-capable in the core gate, while #55280 would still require a separate fix for that. Happy to coordinate with @liuhao1024 if they'd like to combine efforts.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for tracing both the profile and core-gate paths, and for coordinating with the related work.

This is now implemented on main through the live provider-profile path:

  • 67df958dbe06bb10ca16b8686c76baf0de3bac03 updates plugins/model-providers/custom/__init__.py:52-60 to pass enabled configured effort through as top-level reasoning_effort, while retaining think: false when reasoning is disabled.
  • agent/transports/chat_completions.py:579-612 supplies reasoning_config to CustomProfile and merges its top-level API kwargs into the outgoing request.
  • tests/plugins/model_providers/test_custom_profile.py:67-98 covers the configured effort wire format, including xhigh/max, and verifies that think=True is not forced on custom endpoints.
  • The implementing commit is included in v2026.7.7.

The current implementation deliberately uses the safer top-level reasoning_effort format rather than enabling the Ollama-specific think=True flag for all custom/vLLM endpoints. This is an automated hermes-sweeper review.

@teknium1 teknium1 closed this Jul 15, 2026
@teknium1 teknium1 added sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have provider/ollama Ollama / local models sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

reasoning_effort / thinking_budget silently dropped for custom and zai providers

4 participants