Skip to content

fix(openrouter): stop silently ignoring thinking effort off - #10991

Merged
jbg merged 2 commits into
mainfrom
fix/openrouter-reasoning-off
Aug 6, 2026
Merged

fix(openrouter): stop silently ignoring thinking effort off#10991
jbg merged 2 commits into
mainfrom
fix/openrouter-reasoning-off

Conversation

@filipkujawa

@filipkujawa filipkujawa commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

GOOSE_THINKING_EFFORT=off was a silent no-op on the OpenRouter path: apply_reasoning_config returned early and sent no reasoning config, so models that reason by default (Kimi K3, DeepSeek-R, GLM) kept thinking at full default effort.
On Terminal-Bench 2.1 that unbounded thinking cost goose 0/6 on cobol-modernization and build-pov-ray; disabling reasoning at the wire level flips them to PASS and halves median output tokens.

Now off on reasoning-capable models sends OpenRouter's documented disable knob {"reasoning": {"enabled": false}}, keeping the lowest-supported-effort clamp the upstream builder already computed for OpenAI-shaped models.
Some endpoints have mandatory reasoning (gpt-5 family, gemini-3.5, gpt-oss, deepseek-r1, some grok) and deterministically reject every disable spelling with a clear 400; rather than maintaining a name list that goes stale, the provider downgrades the request to {"reasoning": {"effort": "low"}} and retries once when the router reports that error.
The 400 is marked permanent so the generic retry loop does not burn attempts on it first.

Unchanged: an explicit reasoning object from OPENROUTER_PARAMETERS still wins (and never triggers the downgrade), and non-reasoning models still get no reasoning field.

Testing

  • Unit tests assert the exact JSON per case: kimi-k3 off (resolved through the canonical registry), the kept gpt-5 clamp, non-reasoning models (including OpenAI-shaped chat variants like gpt-5.1-chat), and OPENROUTER_PARAMETERS precedence under off; a wiremock test drives the provider through the mandatory-reasoning 400 and asserts the downgraded retry.
  • Live wire checks: {"enabled": false} yields 0 reasoning tokens on kimi-k3, glm-4.6, gemini-2.5-flash, grok-4.20, grok-4.3, claude-sonnet-4.5; mandatory endpoints (gpt-5-mini, gpt-5-pro, gemini-3.5-flash, gpt-oss-120b, grok-4.5, deepseek-r1) reject every disable spelling and all accept effort:"low".
  • End-to-end with the patched binary on gemini-3.5-flash: request log shows the disable, the router 400, and the successful effort:"low" retry.
  • Terminal-Bench 2.1 (patched binary, off, K3): the three signature tasks went 9/9 at k=3 vs 0/9 in thinking-on baselines; a 7-task no-regression slice all passed.
  • cargo fmt, cargo clippy -D warnings, cargo test -p goose green.

GOOSE_THINKING_EFFORT=off previously returned early without sending any
reasoning config, so models that reason by default (Kimi K3, DeepSeek-R,
GLM) kept thinking at full default effort.

Off now sends OpenRouter's documented disable knob
{"reasoning": {"enabled": false}}. For OpenAI/xAI-shaped models the
upstream request builder already clamps off to the lowest supported
effort; that clamp is kept as {"reasoning": {"effort": ...}} because
OpenRouter rejects disable requests for mandatory-reasoning endpoints.

An explicit reasoning object from OPENROUTER_PARAMETERS still takes
precedence, and non-reasoning models still get no reasoning field.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 190763d756

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread crates/goose/src/providers/formats/openrouter.rs Outdated
…-reasoning endpoints

OpenRouter deterministically rejects every reasoning disable spelling on
mandatory-reasoning endpoints (gpt-5 family, gemini-3.5, gpt-oss,
deepseek-r1, some grok) with a clear 400. Instead of maintaining
name-shape heuristics for that set, send the disable and retry once with
the lowest effort when the router reports that error. The 400 is marked
permanent so the generic retry loop does not re-attempt it first.
@filipkujawa
filipkujawa requested review from jbg and michaelneale August 5, 2026 23:56
@jbg
jbg added this pull request to the merge queue Aug 6, 2026
Merged via the queue into main with commit 86eec2a Aug 6, 2026
26 checks passed
@jbg
jbg deleted the fix/openrouter-reasoning-off branch August 6, 2026 02:18
lifeizhou-ap added a commit that referenced this pull request Aug 6, 2026
* main: (32 commits)
  fix: keep turn-context in place for OpenAI Responses-stack models so prompt caching works (#10993)
  fix(developer): byte-bound the shell truncation preview (#10992)
  fix(openrouter): stop silently ignoring thinking effort off (#10991)
  fix: dispatch edited queued messages (#10933)
  fix: contain recipe template paths (#10930)
  fix: make shell approval titles faithful (#10986)
  fix: block MCP app form submissions (#10985)
  fix: migrate desktop routing to React Router 8.3.0 (#10971)
  fix: sanitize Bedrock tool errors (#10934)
  fix implicit OAuth during model discovery (#10929)
  fix: update React Router to 7.18.2 (#10967)
  test: early-exit code-exec smoke tests once tool invocation is observed (#10954)
  fix: keep ACP session naming out of live conversations (#10963)
  Bind MCP apps to trusted ownership metadata (#10747)
  tests: add recursion_limit attribute to remaining ACP test files (#10559)
  Sanitize Unicode tags in MCP resources (#10746)
  fix(oauth): preserve RFC 9207 iss from MCP OAuth callback (#10678)
  feat(installer): detect Termux and select musl portable build (#10568)
  feat: add Celeris provider (#10714)
  fix: shell ACP providers on desktop (#10907)
  ...
lifeizhou-ap added a commit that referenced this pull request Aug 6, 2026
* main: (101 commits)
  fix: keep turn-context in place for OpenAI Responses-stack models so prompt caching works (#10993)
  fix(developer): byte-bound the shell truncation preview (#10992)
  fix(openrouter): stop silently ignoring thinking effort off (#10991)
  fix: dispatch edited queued messages (#10933)
  fix: contain recipe template paths (#10930)
  fix: make shell approval titles faithful (#10986)
  fix: block MCP app form submissions (#10985)
  fix: migrate desktop routing to React Router 8.3.0 (#10971)
  fix: sanitize Bedrock tool errors (#10934)
  fix implicit OAuth during model discovery (#10929)
  fix: update React Router to 7.18.2 (#10967)
  test: early-exit code-exec smoke tests once tool invocation is observed (#10954)
  fix: keep ACP session naming out of live conversations (#10963)
  Bind MCP apps to trusted ownership metadata (#10747)
  tests: add recursion_limit attribute to remaining ACP test files (#10559)
  Sanitize Unicode tags in MCP resources (#10746)
  fix(oauth): preserve RFC 9207 iss from MCP OAuth callback (#10678)
  feat(installer): detect Termux and select musl portable build (#10568)
  feat: add Celeris provider (#10714)
  fix: shell ACP providers on desktop (#10907)
  ...
michaelneale added a commit that referenced this pull request Aug 9, 2026
* origin/main: (47 commits)
  chore(deps): bump pypa/gh-action-pypi-publish from 1.14.1 to 1.14.2 (#11037)
  chore(deps): bump actions/setup-java from 5.6.0 to 5.7.0 (#11036)
  chore(deps): bump Jimver/cuda-toolkit from 0.2.35 to 0.2.36 (#11035)
  chore(deps-dev): bump js-yaml from 4.3.0 to 4.3.1 in /documentation (#11052)
  chore(deps-dev): bump @types/yauzl from 2.10.3 to 3.4.0 in /ui (#10975)
  chore(deps-dev): bump electron from 41.0.0 to 41.10.3 in /ui (#10969)
  chore(deps): bump brace-expansion from 1.1.16 to 1.1.18 in /documentation (#10947)
  chore(deps-dev): bump postcss from 8.5.8 to 8.5.23 in /ui (#10946)
  chore(deps): bump aiohttp from 3.14.1 to 3.14.3 in /scripts/provider-error-proxy (#10945)
  chore(deps): bump fast-uri from 3.1.4 to 3.1.5 in /documentation (#10944)
  fix: reject invalid subrecipe content (#10994)
  chore(deps): bump azure/login from 3.0.0 to 3.0.1 (#11033)
  chore(deps): bump pnpm/action-setup from 6.0.9 to 6.0.10 (#11034)
  Cache-safe request assembly: append-only turn context and declared cache semantics (#11022)
  Unrolled agent loop (#9574)
  fix: keep turn-context in place for OpenAI Responses-stack models so prompt caching works (#10993)
  fix(developer): byte-bound the shell truncation preview (#10992)
  fix(openrouter): stop silently ignoring thinking effort off (#10991)
  fix: dispatch edited queued messages (#10933)
  fix: contain recipe template paths (#10930)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants