fix(anthropic,vertex): three reasoning_effort bugs surfaced by PR #27039 QA matrix by mateo-berri · Pull Request #27053 · BerriAI/litellm

mateo-berri · 2026-05-02T15:53:21Z

Relevant issues

Follow-up to the QA matrix on PR #27039 (fix(anthropic,bedrock): omit thinking/output_config when reasoning_effort="none"). The QA comment listed six pre-existing bugs found while sweeping the full provider × generation × effort matrix end-to-end on a proxy. Two of those (#4 and #5 in the QA list) are already addressed by open PR #27050, and #1 is already addressed by open PR #27040 (fix(bedrock): forward output_config.effort for adaptive-thinking Claude).

This PR closes the remaining three: bugs #2, #3, and #6 from that matrix.

Linear ticket

Resolves LIT-2758 (follow-ups)

Pre-Submission checklist

I have added tests in tests/test_litellm/
My PR passes all unit tests on the touched modules (tests/test_litellm/llms/anthropic/chat/, tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/, tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py — 299 passed)
PR scope is isolated to the three bugs called out in the PR fix(anthropic,bedrock): omit thinking/output_config when reasoning_effort="none" #27039 QA comment

Type

🐛 Bug Fix

Changes

Three independent commits, one per bug from the PR #27039 QA matrix.

1. `fix(vertex,anthropic): forward output_config.effort on Vertex AI Claude 4.6+` (QA bug #2)

Direct :rawPredict curls against us-east5 confirmed Vertex AI Claude 4.6+ accepts output_config.effort with the same per-model enum as Anthropic direct (opus-4-7: low|medium|high|xhigh|max; opus-4-6 / sonnet-4-6: low|medium|high|max). LiteLLM was unconditionally stripping effort in sanitize_vertex_anthropic_output_params, so the reasoning_effort knob never reached Vertex even though Vertex accepted it server-side. The strip was hiding a knob Vertex actually supports — the comment in output_params_utils.py was stale.

Empty VERTEX_UNSUPPORTED_OUTPUT_CONFIG_KEYS and rely on the existing per-model gating in AnthropicConfig._apply_output_config (driven by supports_xhigh_reasoning_effort / supports_max_reasoning_effort flags from the model cost map).
Update the existing Vertex tests — the prior "stripped" assertions are flipped to "forwarded" with explicit 4.6+ model IDs.
Refresh the now-stale comments in vertex_ai_partner_models/anthropic/transformation.py and experimental_pass_through/transformation.py.

2. `fix(anthropic): bump minimal reasoning_effort budget_tokens to 1024 (Anthropic API minimum)` (QA bug #6)

Anthropic's Messages API enforces thinking.enabled.budget_tokens >= 1024 and 400s otherwise (exact error: thinking.enabled.budget_tokens: Input should be greater than or equal to 1024). LiteLLM's DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET=128 leaked through AnthropicConfig._map_reasoning_effort on the pre-4.6 (budget_tokens) thinking path, so reasoning_effort="minimal" 400'd on Anthropic direct, Azure AI Anthropic, Bedrock Invoke, and Vertex AI Anthropic. Only Bedrock Converse silently clamped to 1024 in converse_transformation.py, masking the failure on that one route.

Add an Anthropic-specific constant DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET_ANTHROPIC (default 1024, env-overridable) and use it from AnthropicConfig._map_reasoning_effort.
Leave the generic DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET=128 alone — it's also used by Gemini's Vertex code where the smaller minimum is correct.

3. `fix(anthropic): validate reasoning_effort uniformly across model generations` (QA bug #3)

Invalid reasoning_effort values (disabled, invalid, "", any non-enum string) were silently mapped to {type: "adaptive"} on Claude 4.6/4.7 because AnthropicConfig._map_reasoning_effort short-circuited on the adaptive-thinking branch before reaching the trailing else: raise ValueError check. The same values raised ValueError(500) on pre-4.6 models, so the failure mode for an invalid effort knob depended on which model generation the caller happened to hit — most starkly visible on Bedrock Converse, where disabled / invalid / "" all returned 200 with thinking={type:"adaptive"} on Sonnet/Opus 4.6 (a silent footgun) and 500 ValueError on Opus 4.5.

Validate up front against the allowed enum {none, minimal, low, medium, high, xhigh, max} at the top of _map_reasoning_effort, before the adaptive-thinking short-circuit.
The raise stays as ValueError for now because open PR fix(reasoning_effort): raise BadRequestError(400) on invalid effort values #27050 separately upgrades the trailing raise to litellm.BadRequestError; once that lands the same upgrade applies uniformly to this guard rather than diverging the two PRs.
Per-model gating of xhigh / max remains in AnthropicConfig._apply_output_config, unchanged.

Tests

tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py

test_invalid_reasoning_effort_is_rejected_uniformly (parametrized over claude-opus-4-6-20250514, claude-sonnet-4-6-20260219, claude-opus-4-7, claude-sonnet-4-5-20250929 × disabled, invalid, "", garbage — 16 cases). Pins QA bug Guarantee format of exceptions #3.
test_minimal_reasoning_effort_emits_at_least_anthropic_min_budget (parametrized over claude-haiku-4-5, claude-sonnet-4-5-20250929, claude-opus-4-5-20251101 — 3 cases). Pins QA bug handle max tokens #6.
Existing test_reasoning_effort_maps_to_budget_thinking_for_non_opus_4_6 updated from ("minimal", 128) to ("minimal", 1024).

tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py

test_vertex_ai_anthropic_output_config_effort_forwarded (replaces the old _dropped test). Pins QA bug Enable model / call timeouts #2 on the output_config={"effort": "high"} only path.
test_vertex_ai_anthropic_output_config_format_plus_effort_forwarded (replaces the old _strips_only_effort test). Pins the mixed format + effort path.
test_vertex_ai_anthropic_output_format_preserved_output_config_effort_forwarded (replaces the old _dropped test). Pins the legacy output_format + new output_config.effort path.
test_sanitize_vertex_anthropic_output_params_unit updated — the effort-only and mixed cases now assert preservation rather than strip.

299 tests pass across tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py, tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/, and tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py.

Out of scope (deliberate)

QA bug Give me consistent exceptions #1 (output_config.effort dropped on every Bedrock route): handled by the already-open PR fix(bedrock): forward output_config.effort for adaptive-thinking Claude #27040 (fix(bedrock): forward output_config.effort for adaptive-thinking Claude). This PR does not touch any Bedrock files to avoid merge conflicts with that work.
QA bug Add support for replicate [code in ticket] #4 (500 APIConnectionError on invalid effort values instead of 400 BadRequestError) and QA bug Update main.py #5 (Anthropic / Azure forwarding effort="" raw): handled by the already-open PR fix(reasoning_effort): raise BadRequestError(400) on invalid effort values #27050 (fix(reasoning_effort): raise BadRequestError(400) on invalid effort values). The new validation guard from this PR raises plain ValueError so it picks up the BadRequestError upgrade automatically once fix(reasoning_effort): raise BadRequestError(400) on invalid effort values #27050 lands.

Slack Thread

…de 4.6+ Vertex AI Claude 4.6+ accepts output_config.effort with the same per-model enum as Anthropic direct (low/medium/high on every 4.6+ model, plus xhigh on opus-4-7 and max on opus-4-6/opus-4-7). Verified end-to-end via direct :rawPredict curls in PR #27039 QA bug #2. LiteLLM was unconditionally stripping 'effort' from output_config in the sanitize_vertex_anthropic_output_params helper, which meant the reasoning_effort knob never reached the wire on Vertex even though Vertex accepted it server-side. The strip was hiding a knob Vertex actually supports — same enum surface as Anthropic direct on 4.6/4.7, contradicting the original comment in output_params_utils.py. Empty VERTEX_UNSUPPORTED_OUTPUT_CONFIG_KEYS and rely on the existing per-model gating in AnthropicConfig._apply_output_config (which already validates xhigh/max via supports_xhigh_reasoning_effort / supports_max_reasoning_effort flags from the model cost map). Update tests to match the new behavior — the prior "stripped" assertions are flipped to "forwarded" with explicit 4.6+ model IDs. Resolves PR #27039 QA bug #2. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

…Anthropic API minimum) Anthropic's Messages API enforces 'thinking.enabled.budget_tokens >= 1024' and 400s otherwise (exact error: 'thinking.enabled.budget_tokens: Input should be greater than or equal to 1024'). LiteLLM's DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET=128 leaked through AnthropicConfig._map_reasoning_effort on the pre-4.6 (budget_tokens) thinking path, so reasoning_effort='minimal' 400'd on: - Anthropic direct - Azure AI Anthropic - Bedrock Invoke (chat + /v1/messages) - Vertex AI Anthropic Only Bedrock Converse silently clamped to 1024 in converse_transformation.py, masking the failure on that one route. Add an Anthropic-specific constant DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET_ANTHROPIC (default 1024, overridable via env var) and use it from AnthropicConfig._map_reasoning_effort. Leave the generic DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET=128 alone — it's also used by Gemini's Vertex code where the smaller minimum is correct. Resolves PR #27039 QA bug #6. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

…rations Invalid reasoning_effort values ('disabled', 'invalid', '', any non-enum string) were silently mapped to '{type: "adaptive"}' on Claude 4.6/4.7 because AnthropicConfig._map_reasoning_effort short-circuited on the adaptive-thinking branch before reaching the trailing 'else: raise ValueError' check. The same values raised ValueError(500) on pre-4.6 models, so the failure mode for an invalid effort knob depended on which model generation the caller happened to hit — visible most starkly on Bedrock Converse, where 'disabled' / 'invalid' / '' all returned 200 with 'thinking={type:"adaptive"}' on Sonnet/Opus 4.6 (a silent footgun) and 500 ValueError on Opus 4.5. Validate up front against the allowed enum {none, minimal, low, medium, high, xhigh, max} so the failure mode is consistent across model generations. Per-model gating of 'xhigh' / 'max' remains in AnthropicConfig._apply_output_config (driven by the model cost map's supports_xhigh_reasoning_effort / supports_max_reasoning_effort flags), unchanged. This raise stays as ValueError because the open PR #27050 separately upgrades the trailing raise to litellm.BadRequestError; once that lands the same upgrade applies uniformly to this guard rather than diverging the two PRs. Resolves PR #27039 QA bug #3. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

CLAassistant · 2026-05-02T15:53:28Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

mateo-berri · 2026-05-02T15:57:14Z

@greptileai

mateo-berri · 2026-05-02T15:59:08Z

bugbot run

greptile-apps · 2026-05-02T15:59:30Z

Greptile Summary

Fixes three pre-existing reasoning_effort bugs surfaced by PR #27039 QA: (1) output_config.effort was stripped before reaching Vertex AI Claude 4.6+, which actually accepts it; (2) reasoning_effort="minimal" emitted a budget below Anthropic's enforced minimum of 1024 on pre-4.6 paths, causing 400 errors; (3) invalid effort strings such as "disabled" or "" were silently coerced to {type:"adaptive"} on Claude 4.6/4.7 while raising ValueError on pre-4.6, making the failure mode model-generation-dependent. All three fixes are well-targeted, unit-tested with good parametrization, and leave unrelated providers (Bedrock, Gemini) untouched.

Confidence Score: 5/5

Safe to merge — all three targeted bug fixes are correct, well-commented, and covered by new unit tests with no logic regressions.

Only finding is a P2 stale constant name in a test comment; all production logic changes are correct and isolated to the described bugs.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/chat/transformation.py	Adds upfront validation of reasoning_effort against _VALID_REASONING_EFFORT_VALUES (bug #3) and changes "minimal" budget from 128 to 1024 (bug #6); logic is correct and well-commented.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/output_params_utils.py	Empties VERTEX_UNSUPPORTED_OUTPUT_CONFIG_KEYS so output_config.effort is forwarded to Vertex 4.6+ (bug #2); scaffold preserved for future unsupported keys. Docstring updated; previous stale "effort" example removed.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py	Comment-only update to reflect that output_config.effort is now forwarded; no logic changes.
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py	Comment-only update to reflect that effort is now forwarded on 4.6+; no logic changes.
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py	Adds 19 new parametrized test cases for bugs #3 and #6, and updates the minimal-budget assertion from 128 to 1024; has a stale constant name in a comment (P2).
tests/test_litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/test_vertex_ai_partner_models_anthropic_transformation.py	Flips four Vertex effort tests from "dropped" to "forwarded" assertions and updates model IDs to claude-opus-4-6; tests are consistent with the bug fix.

_{Reviews (2): Last reviewed commit: "fix(anthropic): move reasoning_effort en..." | Re-trigger Greptile}

The original commit added a new env var DEFAULT_REASONING_EFFORT_MINIMAL_THINKING_BUDGET_ANTHROPIC, but the documentation validation CI (tests/documentation_tests/test_env_keys.py) requires every os.getenv key in the LiteLLM source to have a matching entry in the litellm-docs config_settings.md table — and that file lives in a separate repo. Drop the new env var and reuse the existing DEFAULT_REASONING_EFFORT_LOW_THINKING_BUDGET=1024 constant, which is both the LiteLLM 'low' budget and the Anthropic API minimum (thinking.enabled.budget_tokens >= 1024). Net behavior is identical: reasoning_effort='minimal' on the pre-4.6 budget_tokens path now emits budget_tokens=1024 instead of 128 — clearing the 400 'thinking.enabled.budget_tokens: Input should be greater than or equal to 1024' on every Anthropic-backed provider. The QA comment on PR #27039 explicitly suggested this consolidation ('the current minimal mapping would benefit from a bump to 1024 matching low'). Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e374cb3. Configure here.}

Greptile P2 on PR #27053: the docstring still cited 'effort' as an example of an unsupported key dropped by the sanitizer, but VERTEX_UNSUPPORTED_OUTPUT_CONFIG_KEYS is now empty (Vertex 4.6+ accepts output_config.effort, per direct :rawPredict curls in the PR #27039 QA matrix). Update the docstring to reflect the actual current behavior: the helper is effectively a passthrough plus a defensive non-dict guard, and the filtering scaffold is kept so that adding a new Vertex-unsupported key would be a one-line frozenset edit. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

CI integration tests caught that BaseConfig.get_config() iterates cls.__dict__ to build the outgoing request body — anything not starting with '__', '_abc', or '_is_base_class' that isn't a callable or None gets serialized into the wire payload. The newly-added AnthropicConfig._VALID_REASONING_EFFORT_VALUES ClassVar[frozenset] was slipping past that filter and ending up in JSON requests as a non-serializable field, breaking custom-prompt-management round-tripping and three test_custom_prompt_management.py tests with 'TypeError: Object of type frozenset is not JSON serializable'. Move the enum to a module-level constant. Same validation behavior, but get_config() never sees it. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>

mateo-berri · 2026-05-02T16:14:01Z

@greptileai

mateo-berri · 2026-05-02T16:20:15Z

bugbot run

codecov · 2026-05-02T16:21:15Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit ca5c24c. Configure here.}

mateo-berri · 2026-05-03T12:00:33Z

Superceded by #27074

cursoragent and others added 3 commits May 2, 2026 15:49

cursor Bot reviewed May 2, 2026

View reviewed changes

cursoragent and others added 2 commits May 2, 2026 16:06

cursor Bot reviewed May 2, 2026

View reviewed changes

mateo-berri closed this May 3, 2026

mateo-berri deleted the litellm_fix_qa_bugs_anthropic_reasoning_effort branch May 4, 2026 22:51

Navigation Menu

Uh oh!

Uh oh!

fix(anthropic,vertex): three reasoning_effort bugs surfaced by PR #27039 QA matrix - #27053

fix(anthropic,vertex): three reasoning_effort bugs surfaced by PR #27039 QA matrix#27053
mateo-berri wants to merge 6 commits into
litellm_internal_stagingfrom
litellm_fix_qa_bugs_anthropic_reasoning_effort

mateo-berri commented May 2, 2026

Uh oh!

CLAassistant commented May 2, 2026

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

greptile-apps Bot commented May 2, 2026 •

edited

Loading

Important Files Changed

Uh oh!

cursor Bot left a comment

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

codecov Bot commented May 2, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

mateo-berri commented May 3, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

mateo-berri commented May 2, 2026

Relevant issues

Linear ticket

Pre-Submission checklist

Type

Changes

1. fix(vertex,anthropic): forward output_config.effort on Vertex AI Claude 4.6+ (QA bug #2)

2. fix(anthropic): bump minimal reasoning_effort budget_tokens to 1024 (Anthropic API minimum) (QA bug #6)

3. fix(anthropic): validate reasoning_effort uniformly across model generations (QA bug #3)

Tests

Out of scope (deliberate)

Uh oh!

CLAassistant commented May 2, 2026

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

greptile-apps Bot commented May 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

mateo-berri commented May 2, 2026

Uh oh!

codecov Bot commented May 2, 2026

Codecov Report

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

1. `fix(vertex,anthropic): forward output_config.effort on Vertex AI Claude 4.6+` (QA bug #2)

2. `fix(anthropic): bump minimal reasoning_effort budget_tokens to 1024 (Anthropic API minimum)` (QA bug #6)

3. `fix(anthropic): validate reasoning_effort uniformly across model generations` (QA bug #3)

greptile-apps Bot commented May 2, 2026 •

edited

Loading

mateo-berri commented May 3, 2026 •

edited

Loading