Skip to content

fix(anthropic): handle per-level reasoning_effort flags without supports_reasoning - #38618

Merged
tin-berri merged 2 commits into
litellm_internal_stagingfrom
fix/reasoning-effort-per-level-flags
Aug 28, 2026
Merged

fix(anthropic): handle per-level reasoning_effort flags without supports_reasoning#38618
tin-berri merged 2 commits into
litellm_internal_stagingfrom
fix/reasoning-effort-per-level-flags

Conversation

@tin-berri

@tin-berri tin-berri commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Problem

gpt-5-search-api declares supports_minimal_reasoning_effort: true but has no supports_reasoning flag. The resolver was treating this as "no reasoning support" and falling back to the chain floor, causing minimal requests to downgrade to low.

Solution

When a model has any per-level flag set (not None), treat supports_reasoning as implicitly True and proceed through the flag resolution path. This allows models like gpt-5-search-api to correctly advertise and accept their declared effort levels.

Behavior change

20 azure deployments (those with max or xhigh per-level flags set) will now forward those efforts instead of degrading to high. This is intended (avoiding unnecessary degradation) but is a latency/cost change; operators should monitor reasoning effort changes in logs if they see bill increases.

Testing

  • ✅ New test verifies per-level flag enables resolution without explicit supports_reasoning
  • ✅ All existing reasoning effort capability tests pass

Fixes the regression identified in PR #38492 follow-up review.


Note

Medium Risk
Changes router-advertised reasoning effort levels for map entries with per-level flags but no supports_reasoning, which can alter forwarded effort (including higher tiers on some Azure deployments) and affect latency/cost.

Overview
Fixes incorrect downgrades when a catalog entry sets per-level supports_*_reasoning_effort flags but omits supports_reasoning (e.g. gpt-5-search-api with supports_minimal_reasoning_effort: true).

resolve_supported_reasoning_efforts now treats any non-None per-level flag as implicit evidence that reasoning is supported, so resolution continues through the flag/opt-in/opt-out path instead of behaving like a mapped non-reasoning model (()) or unknown. supports_reasoning: false is checked first and still returns no efforts, even if per-level flags are present.

Tests cover implicit enablement from a lone per-level flag and that explicit supports_reasoning: false overrides inherited per-level metadata.

Reviewed by Cursor Bugbot for commit 4603619. Bugbot is set up for automated code reviews on this repo. Configure here.

…rts_reasoning

When a model has only per-level flags (e.g. supports_minimal_reasoning_effort: true)
but no explicit supports_reasoning flag, treat it as implicitly reasoning-capable.
This fixes gpt-5-search-api which declares minimal support but was incorrectly
degraded to low/minimal floor due to missing explicit supports_reasoning flag.

Test: verify per-level flag enables resolution path even without supports_reasoning.

Note: This change indirectly causes 20 azure deployments to forward max/xhigh
instead of degrading to high when requested, as these models now correctly
resolve their supported efforts through declared capability flags. This is
intended behavior (avoiding unnecessary degradation) but silent; operators
seeing increased latency/cost should check reasoning effort changes in logs.

Co-Authored-By: Claude <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR corrects reasoning-effort capability resolution when per-level flags are present without a top-level supports_reasoning declaration.

  • Treats declared per-level effort flags as implicit reasoning support.
  • Preserves explicit supports_reasoning: false as the highest-precedence opt-out.
  • Adds regression tests for implicit support and explicit opt-out precedence.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
litellm/router_utils/reasoning_effort_capability.py Correctly infers reasoning support from per-level metadata while returning no efforts immediately for an explicit opt-out.
tests/test_litellm/router_utils/test_reasoning_effort_capability.py Adds focused coverage for per-level implicit support and explicit-false precedence without weakening existing assertions.

Reviews (3): Last reviewed commit: "fix(anthropic): explicit supports_reason..." | Re-trigger Greptile

Comment thread litellm/router_utils/reasoning_effort_capability.py Outdated
Comment thread tests/test_litellm/router_utils/test_reasoning_effort_capability.py Outdated
@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

… flags

Greptile P1: the implicit-True branch bypassed the operator's explicit
supports_reasoning: false escape hatch when per-level flags were present
or inherited through the bare-twin lookup. Return () first on explicit
False, then apply the per-level implication only when the flag is unset.

Also drops a test comment that restated the test name (P2).

Co-Authored-By: Claude <noreply@anthropic.com>
@tin-berri

Copy link
Copy Markdown
Contributor Author

@greptileai can you reply here with score & why

@tin-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Score: 5/5. The change correctly distinguishes explicit supports_reasoning=False, unset reasoning support with no per-level metadata, and implicit support when any per-level flag is present. It preserves existing mapped/off-map unknown semantics, honors inherited bare-model flags, and leaves declared effort lists and Azure none gating intact. The added tests cover both the new implicit-support path and the explicit-false override; I found no blocking issues.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 4603619. Configure here.

@codspeed-hq

codspeed-hq Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing fix/reasoning-effort-per-level-flags (4603619) with litellm_internal_staging (98c5233)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (3300fc3) during the generation of this report, so 98c5233 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@tin-berri

Copy link
Copy Markdown
Contributor Author

@greptileai review pls

@tin-berri
tin-berri enabled auto-merge (squash) August 28, 2026 04:43
Comment thread litellm/router_utils/reasoning_effort_capability.py
@tin-berri
tin-berri disabled auto-merge August 28, 2026 04:46
@tin-berri

Copy link
Copy Markdown
Contributor Author

@greptileai based on the response / comment, can you give me a score now!

@tin-berri
tin-berri merged commit b3322e3 into litellm_internal_staging Aug 28, 2026
81 checks passed
@tin-berri
tin-berri deleted the fix/reasoning-effort-per-level-flags branch August 28, 2026 05:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants