fix(anthropic): drop and self-heal empty thinking blocks on /v1/messages

[tin-berri]

TLDR

Problem this solves:

• Mixed-provider auto-routers break Claude Code sessions with Anthropic 400s
• A bridged reasoning turn can emit {"type": "thinking", "thinking": ""}
• Replaying that block in a tool loop gets "each thinking block must contain thinking"
• The existing strip-and-retry only matched signature errors, so nothing recovered

How it solves it:

• The /v1/messages bridge no longer emits thinking blocks with empty text
• The /v1/messages sanitizer now drops empty thinking blocks from replayed history
• The strip-and-retry matcher now also recognizes the empty-thinking 400

User Flow

Before: a developer running Claude Code against an auto-router mixing Anthropic and OSS reasoning tiers sees sessions die mid-task

1. Claude Code sends POST https://litellm-domain/v1/messages and the router places that turn on an OSS reasoning model, which goes straight to parallel tool calls with no reasoning text
2. The streamed response contains a thinking block with empty text, which Claude Code stores as conversation history
3. On a later turn the router places the request on a Claude model, replaying that history
4. The request fails with 400 "messages.1.content.0.thinking: each thinking block must contain thinking" and Claude Code stops working

After: the same session runs to completion

1. Claude Code sends the same POST https://litellm-domain/v1/messages and the OSS turn streams back with no empty thinking block at all
2. Sessions that already carry a poisoned turn are healed too: the replayed empty block is dropped before the request reaches Anthropic, and the call returns 200
3. The later Claude turn succeeds and the tool loop continues

Relevant issues

• Sibling fix for the /v1/chat/completions prompt factory is open community PR fix(anthropic): drop thinking blocks with empty thinking text, not just missing signature #38049 (disjoint files)
• The /v1/responses adapter already got the same producer fix in fix(anthropic): stop emitting empty thinking blocks on the Responses adapter #36033

Linear ticket

Resolves LIT-6357

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Shared setup: a local proxy on :4357 with two deployments: claude-tier is anthropic/claude-haiku-4-5-20251001 served through a live gateway upstream (real Anthropic account, real spend), oss-tier is hosted_vllm/stub-reasoner served by a deterministic local upstream that streams a vLLM-style reasoning response going straight to parallel tool calls with no reasoning text (the shape the customer's OSS tier produces)

Poisoned-history payload (what Claude Code replays after a bridged turn):

{
  "model": "claude-tier",
  "max_tokens": 2048,
  "thinking": {"type": "enabled", "budget_tokens": 1024},
  "tools": [{"name": "get_weather", "description": "Get weather for a city", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}],
  "messages": [
    {"role": "user", "content": "Weather in Paris and London?"},
    {"role": "assistant", "content": [
      {"type": "thinking", "thinking": "", "signature": ""},
      {"type": "tool_use", "id": "toolu_01A", "name": "get_weather", "input": {"city": "Paris"}},
      {"type": "tool_use", "id": "toolu_01B", "name": "get_weather", "input": {"city": "London"}}
    ]},
    {"role": "user", "content": [
      {"type": "tool_result", "tool_use_id": "toolu_01A", "content": "18C sunny"},
      {"type": "tool_result", "tool_use_id": "toolu_01B", "content": "14C rain"}
    ]}
  ]
}

Before (3002994)

Replayed empty thinking block, non-streaming

1. curl -s -X POST http://127.0.0.1:4357/v1/messages -H "Authorization: Bearer $KEY" -H "content-type: application/json" -d @poisoned.json
2. HTTP 400: litellm.BadRequestError: AnthropicException - ... "messages.1.content.0.thinking: each thinking block must contain thinking"

Replayed empty thinking block, streaming

1. Same payload with "stream": true
2. HTTP 400 with the same error text

Bridged reasoning turn produces the poison

1. curl -s -X POST http://127.0.0.1:4357/v1/messages ... -d '{"model": "oss-tier", "stream": true, ...}' where the upstream streams an empty thinking entry then two tool calls
2. The SSE stream contains content_block_start {"type": "thinking", "thinking": ""} immediately followed by content_block_stop with no delta: the block Claude Code will store and replay

After (133c069)

Replayed empty thinking block, non-streaming

1. Same curl, same payload
2. HTTP 200 with a normal assistant answer (the empty block is dropped before the request reaches Anthropic, no retry round-trip burned)

Replayed empty thinking block, streaming

1. Same payload with "stream": true
2. HTTP 200, normal SSE stream

Bridged reasoning turn produces the poison

1. Same stub-backed streaming call
2. The SSE stream opens the two tool_use blocks directly with no thinking block at all; a stub scenario with real reasoning text still streams its thinking block and deltas unchanged

Type

🐛 Bug Fix
✅ Test

Caveats (if any)

Medium

• An all-whitespace reasoning stream still emits a whitespace-only thinking block
  • Healed at ingestion on the next turn (verified live, whitespace replay returns 200)

Low

• A pathological stream (text, then a contentless thinking entry, then tools) can still open an empty block on the mid-stream transition path; no known provider emits that shape, and the sanitizer plus retry cover the replay
• The retry matcher rename touches only internal call sites (grep-verified, no external users)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches message sanitization, streaming block assembly, and HTTP retry logic on the Anthropic Messages bridge; behavior changes for empty thinking blocks could affect multi-turn tool clients, though coverage is heavily tested.

Overview
Fixes LIT-6357: mixed-provider /v1/messages tool loops were failing when conversation history contained {"type": "thinking", "thinking": ""} (often from a bridged non-Anthropic reasoning turn with no reasoning text).

Ingestion: strip_empty_text_blocks_from_anthropic_messages is renamed to strip_empty_content_blocks_from_anthropic_messages and now removes empty/whitespace-only thinking blocks (via is_empty_thinking_block) in addition to empty text blocks, on the native Messages handler path.

Producer: The pass-through adapter no longer emits empty thinking blocks in non-streaming responses, and AnthropicStreamWrapper treats deltas whose thinking_blocks are all empty as blank so it does not open a poison thinking block before tool calls.

Retry: is_anthropic_invalid_thinking_signature_error is renamed to is_anthropic_invalid_thinking_block_error and also matches Anthropic’s “each thinking block must contain thinking” 400 so strip-and-retry can still run.

^{Reviewed by Cursor Bugbot for commit 133c069. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

Confidence score: 4/5

This is a focused and well-tested fix for the reported /v1/messages flow:

• The producer path skips empty and whitespace-only thinking blocks while preserving real thinking, signatures, redacted thinking, and tool calls (streaming_iterator.py:1031, transformation.py:1264).
• The ingestion sanitizer removes poisoned empty thinking blocks before dispatch without mutating the caller’s messages, while retaining tool-use and redacted-thinking content (common_utils.py:1034, handler.py:242).
• The retry matcher now covers the empty-thinking Anthropic 400 while preserving the existing invalid-signature recovery (common_utils.py:974, transformation.py:159).
• Tests cover sync and async streaming, signed/unsigned empty blocks, whitespace, later real thinking, sanitizer behavior, and existing text-block behavior.

I’m not giving 5/5 because the PR itself notes an uncovered pathological mid-stream transition where an empty thinking block can still be opened, and the required CI status is currently pending. The sanitizer and retry path mitigate that case, so this is still a strong 4/5 for the stated customer flow.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 2152456. Configure here.}

[greptile-apps]

Greptile Summary

This PR filters empty thinking blocks from Anthropic Messages history and bridged responses, and broadens recovery for related provider errors

• Sanitizes replayed empty text and thinking content before dispatch
• Suppresses empty thinking blocks in streaming and non-streaming bridge output
• Extends the retry matcher and adds focused regression coverage

Confidence Score: 4/5

The signed-first-chunk regression should be fixed before merging because it can poison later reasoning-history replay

The sanitizer and non-streaming producer changes are sound, but the streaming blank check skips real provider signature content before translation

Files Needing Attention: litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py, litellm/llms/anthropic/common_utils.py

Important Files Changed

Filename	Overview
litellm/llms/anthropic/common_utils.py	Adds shared empty-thinking detection, request sanitization, and broader retry matching; the implementation is functionally focused but over-documented
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py	Suppresses empty initial thinking chunks but also drops meaningful signatures when a signed block arrives before thinking text
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py	Prevents non-streaming bridge responses from emitting empty thinking blocks while preserving non-empty and redacted blocks
litellm/llms/anthropic/experimental_pass_through/messages/handler.py	Applies the expanded sanitizer consistently across synchronous and asynchronous Anthropic Messages dispatch
litellm/llms/base_llm/anthropic_messages/transformation.py	Extends the bounded strip-and-retry path to Anthropic empty-thinking validation errors

Comments Outside Diff (1)

1. litellm/llms/anthropic/common_utils.py, line 1040-1067 (link)

   Filtering logic is over-documented

   This docstring duplicates error details, scenarios, and behavior already captured by names and tests, increasing maintenance cost when the logic changes

   Context Used: CLAUDE.md (source)

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "fix(anthropic): drop and self-heal empty..." | Re-trigger Greptile}

[greptile-apps]

Signed chunks are dropped

When a stream starts with empty thinking and a signature, this check skips the chunk, dropping the signature needed for later replay

Knowledge Base Used:

• Model request execution
• Provider adapters and capabilities

[tin-berri]

The classifier captures the skipped chunk's signature into the pending block start, so it survives; new tests pin both carry and discard paths

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tin-berri]

Addressed the P1 with pinning tests showing the signature already survives the skip. @greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

Confidence score: 5/5

The previous P1 is addressed. The new sync and async pinning tests verify that a signature on an initially blank thinking chunk is preserved when real thinking follows, while also verifying it is discarded when the next block is text. This confirms the blank-chunk suppression does not lose valid signature data or attach it to the wrong content block.

The rest of the fix remains well covered: empty thinking blocks are suppressed in streaming and non-streaming bridge output, replayed empty blocks are sanitized, existing invalid-signature recovery is preserved, and the empty-thinking 400 matcher is exercised. The latest e2e status is also passing, so I’m raising the score from 4/5 to 5/5.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 133c069. Configure here.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit6357_empty_thinking (133c069) with litellm_internal_staging (bd7e9c1)}