Skip to content

fix(anthropic): strip thinking_blocks for non-Anthropic backends - #32684

Closed
samagana wants to merge 1 commit into
BerriAI:litellm_oss_stagingfrom
samagana:fix/anthropic-thinking-blocks-non-anthropic-backends
Closed

fix(anthropic): strip thinking_blocks for non-Anthropic backends#32684
samagana wants to merge 1 commit into
BerriAI:litellm_oss_stagingfrom
samagana:fix/anthropic-thinking-blocks-non-anthropic-backends

Conversation

@samagana

@samagana samagana commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #27946

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

The Anthropic /v1/messages -> OpenAI chat-completions adapter (translate_anthropic_messages_to_openai) attaches the Anthropic-specific thinking_blocks field to assistant messages unconditionally. Non-Anthropic OpenAI-compatible backends reject that field outright, which breaks any multi-turn conversation once an earlier assistant turn carried reasoning.

This was verified directly against an OpenAI-compatible GLM endpoint, bypassing litellm entirely, to confirm the backend's actual contract rather than litellm's behavior at some commit:

  • an assistant turn carrying thinking_blocks gets 400 invalid_request_error: Extra inputs are not permitted, field: 'messages[1].thinking_blocks'
  • the same turn carrying reasoning_content instead gets 200 OK, and the model consumes it and reasons over the prior turn correctly

That contract is what motivates the fix: convert, rather than merely drop, thinking_blocks for non-Anthropic backends.

I don't have a from-litellm before/after repro to attach a commit hash to, since the failing request never reaches litellm's HTTP layer; it fails on the backend's own schema validation regardless of which litellm commit constructed it. The regression tests added in this PR exercise the actual code path (translate_anthropic_messages_to_openai) that produces the request body, asserting the pre-fix behavior (raw thinking_blocks attached) would fail and the post-fix behavior (converted reasoning_content) passes for a representative set of non-Anthropic and Anthropic model identifiers.

Type

🐛 Bug Fix
✅ Test

Changes

Gate the thinking_blocks attachment on is_anthropic_claude_model or is_bedrock_arn_model, the same pair of checks already used together elsewhere in this file (for example the cache_control path). Anthropic Claude backends keep thinking_blocks and their signed signatures unchanged; that covers anthropic/*, Bedrock *anthropic*, Vertex *claude*, and Bedrock ARNs such as Application Inference Profiles that point at Claude but contain neither "anthropic" nor "claude" in the identifier. Every other backend gets the raw blocks converted to an OpenAI-style reasoning_content string instead, concatenating the unredacted thinking blocks (redacted blocks carry no readable text and are dropped). When the target model is unknown (None), the prior behavior is preserved so no existing caller changes.

This is the complete form of the half-fixes in #27947 and #28258, both of which only add reasoning_content and leave thinking_blocks attached, so they do not resolve the 400.

@greptile-apps

greptile-apps Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a regression in translate_anthropic_messages_to_openai where the Anthropic-specific thinking_blocks field was unconditionally attached to assistant messages, causing non-Anthropic OpenAI-compatible backends to reject multi-turn conversations with a 400 "Extra inputs are not permitted" error.

  • The fix gates thinking_blocks attachment on is_anthropic_claude_model(model) or is_bedrock_arn_model(model), exactly matching the pattern already used for cache_control and thinking-parameter translation in the same file. Anthropic/Claude/Bedrock ARN targets keep the blocks and their signed signatures; all other backends receive a concatenated reasoning_content string built from the unredacted blocks, while redacted blocks (which carry no readable text) are silently dropped.
  • Two parametrized tests verify both code paths — stripping for non-Anthropic models and preservation for Anthropic/Bedrock targets — including correct handling of mixed thinking/redacted-thinking block lists.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to the thinking_blocks gating logic, follows the exact same pattern used for cache_control and thinking parameter translation throughout the file, and is covered by tests that exercise both the strip and preserve paths.

The fix aligns with the established helper pattern (is_anthropic_claude_model + is_bedrock_arn_model) used everywhere else in this file for Anthropic-specific field handling. The model is None fallback preserves prior behavior, the Bedrock ARN detection is tightly scoped to :bedrock: in the service segment, and the test suite covers the representative cases including mixed redacted/unredacted blocks.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py Adds preserve_thinking_blocks gate using is_anthropic_claude_model and is_bedrock_arn_model, matching the established pattern already used for cache_control and thinking parameter translation in this file. Non-Anthropic backends get reasoning_content instead; Bedrock ARNs and Claude backends keep thinking_blocks unchanged.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py Adds two parametrized tests covering: stripping of thinking_blocks and conversion to reasoning_content for non-Anthropic models; preservation of thinking_blocks for Anthropic/Claude/Bedrock ARN models. Also verifies redacted blocks are correctly dropped from the reasoning_content concatenation.

Reviews (3): Last reviewed commit: "fix(anthropic): strip thinking_blocks fo..." | Re-trigger Greptile

@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

The Anthropic /v1/messages -> OpenAI chat-completions pass-through adapter
(translate_anthropic_messages_to_openai) attaches the Anthropic-specific
thinking_blocks field to assistant messages unconditionally. Non-Anthropic
OpenAI-compatible backends reject it: on multi-turn conversations, models
like GLM behind an OpenAI-compatible endpoint fail with

    400 invalid_request_error: Extra inputs are not permitted,
    field: 'messages[1].thinking_blocks'

This breaks any multi-turn conversation once an earlier assistant turn
carried reasoning.

Verified directly against an OpenAI-compatible GLM endpoint (bypassing
litellm):
- assistant turn with thinking_blocks   -> 400 (field rejected)
- assistant turn with reasoning_content -> 200 OK (the model consumes it and
  reasons over the prior turn)

So the fix is to convert, not just drop: for non-Anthropic backends, strip the
raw thinking_blocks and set the OpenAI-style reasoning_content string
(concatenating the unredacted thinking blocks; redacted blocks carry no
readable text and are dropped).

Gate the thinking_blocks attachment on is_anthropic_claude_model or
is_bedrock_arn_model, the same pair of checks already used together elsewhere
in this file (e.g. for cache_control). Anthropic Claude backends (anthropic/*,
bedrock *anthropic*, vertex *claude*, and Bedrock ARNs such as Application
Inference Profiles that point at Claude) keep thinking_blocks and their signed
signatures unchanged. Everyone else gets reasoning_content instead. When the
target model is unknown (None) the prior behaviour is preserved (blocks
kept), so no existing caller changes.

This is the complete form of the half-fixes in BerriAI#27947 and BerriAI#28258, both of
which only add reasoning_content and leave thinking_blocks attached, so they
do not resolve the 400. Closes BerriAI#27946.
@samagana
samagana force-pushed the fix/anthropic-thinking-blocks-non-anthropic-backends branch from ae57ebb to c2fd9b3 Compare July 9, 2026 22:36
@samagana
samagana changed the base branch from main to litellm_oss_staging July 9, 2026 22:36
@CLAassistant

CLAassistant commented Jul 9, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@samagana

samagana commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

@greptile-apps

@samagana samagana closed this Jul 9, 2026
@samagana samagana reopened this Jul 9, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing samagana:fix/anthropic-thinking-blocks-non-anthropic-backends (ae57ebb) with main (f824783)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_oss_staging (6d796d0) during the generation of this report, so main (f824783) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@mateo-berri

Copy link
Copy Markdown
Contributor

Thanks for the contribution. #38275 landed the strip at the Together transformation, and #27946 is closed, so closing this as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants