fix: forward reasoning_config to custom providers (vLLM, Ollama, etc.)

[vominh1919]

Problem

When using provider: custom with a vLLM-served thinking model (e.g. MiniMax-M2.7, DeepSeek-R1, GLM-4.x), the agent silently sends no thinking-budget control on the wire request. This causes:

1. Runaway reasoning: The model spends the entire max_tokens budget inside <think> blocks, producing empty content with finish_reason: "length" — no recovery path for the user.
2. No effort control: Even when the user explicitly configures reasoning_config.effort, it's ignored for custom providers.

Root cause: _supports_reasoning_extra_body() in run_agent.py returns False for any base_url not in its hardcoded allowlist (OpenRouter, Nous, GitHub, LM Studio). Custom/vLLM providers are excluded, so extra_body.reasoning is never emitted — even when the user explicitly opted in via reasoning_config.

Additionally, when reasoning is supported, the effort level was hardcoded to "medium" instead of respecting the user's reasoning_config.effort.

Fix

run_agent.py: Added a "custom" provider check in _supports_reasoning_extra_body() that honors the user's explicit reasoning_config. Returns True when enabled is not explicitly set to False.

agent/transports/chat_completions.py: Changed the reasoning extra_body assembly to use the user's configured effort level from reasoning_config instead of hardcoding "medium". Consistent with how Kimi and TokenHub handle effort levels.

Before vs After

Scenario	Before	After
provider: custom + reasoning_config: {effort: high}	extra_body.reasoning NOT sent	extra_body.reasoning = {enabled: true, effort: "high"}
provider: custom + no reasoning_config	No reasoning sent	No reasoning sent (unchanged)
provider: custom + reasoning_config: {enabled: false}	No reasoning sent	No reasoning sent (unchanged)
OpenRouter + reasoning_config: {effort: low}	Hardcoded "medium"	"low" from user config

Related

Fixes #20576

[lachlancahill]

Would love to see this merged. Currently running into a lot of runaway reasoning. For those using qwen with vllm, i found I experienced this problem less in SGLang.

[aider4ryder]

@vominh1919 Thanks for this fix — I cherry-picked it locally and it does address the silent-drop path for custom providers.

One small concern: the current implementation clamps the forwarded effort to only:

if _e in ("low", "medium", "high"):
    _effort = _e

For custom providers this may be too restrictive. Some OpenAI-compatible / OpenAI-ish backends accept additional effort values, including xhigh or max.

Example: GLM-5.2 behind a custom endpoint accepts reasoning control through OpenAI-compatible request fields, and max is a valid stronger effort level on that backend. Hermes itself also exposes /reasoning xhigh, so with this PR a user setting reasoning_effort: xhigh still gets silently downgraded to medium on custom providers.

I don't think Hermes should necessarily map xhigh -> max inside this PR, because different custom backends may use different vocabularies. But for custom providers specifically, it may be better to either:

• pass through the configured effort string as-is, or
• at least allow xhigh / max in addition to low|medium|high.

Otherwise this fixes the binary on/off drop, but still loses higher-effort settings for custom endpoints that support them.

[aider4ryder]

Small correction after testing the cherry-picked code locally: on current main, provider: custom resolves to CustomProfile, so the provider-profile path is used before the legacy flag path.

I verified the emitted kwargs directly:

profile = get_provider_profile("custom")  # CustomProfile
ct.build_kwargs(
    model="glm-5.2",
    base_url="https://api.z.ai/api/coding/paas/v4",
    reasoning_config={"enabled": True, "effort": "high"},
    supports_reasoning=True,
    provider_profile=profile,
    request_overrides=None,
)["extra_body"]
# => {}

So this PR fixes the legacy/custom gate, but it may not actually affect the current registered CustomProfile path unless CustomProfile.build_api_kwargs_extras() also emits something when supports_reasoning=True / reasoning_config is enabled.

A user-level workaround is to put a static per-custom-provider override in config:

custom_providers:
  - name: z1
    base_url: https://api.z.ai/api/coding/paas/v4
    model: glm-5.2
    extra_body:
      reasoning:
        enabled: true
        effort: high

That works because custom-provider extra_body is passed through request_overrides and merged last, but it is static and does not honor /reasoning changes.

[teknium1]

Thanks for identifying the silent custom-provider reasoning-control gap. This is now implemented on main through the live provider-profile path rather than the legacy transport path this PR changes.

• Automated hermes-sweeper review verified plugins/model-providers/custom/__init__.py:52-60: enabled configured effort is forwarded as top-level reasoning_effort; disabled reasoning emits think: false; xhigh and max are preserved.
• agent/chat_completion_helpers.py:850-888 resolves registered custom to CustomProfile and returns through that profile path before legacy assembly, matching the June 30 discussion's observation.
• agent/transports/chat_completions.py:562-575 passes reasoning_config to the profile and merges its top-level request kwargs.
• Commit 67df958dbe06bb10ca16b8686c76baf0de3bac03 added this implementation and its regression coverage in tests/plugins/model_providers/test_custom_profile.py; it shipped in v2026.7.7.