Skip to content

fix(anthropic): drop thinking blocks with empty thinking text, not just missing signature - #38049

Open
shoemoney wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
shoemoney:fix/drop-empty-thinking-blocks
Open

shoemoney wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
shoemoney:fix/drop-empty-thinking-blocks

Conversation

@shoemoney

@shoemoney shoemoney commented Aug 24, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • A thinking block with a valid-looking signature but empty thinking text is not dropped
  • Anthropic rejects it with a 400 even though the signature is present
  • This crashes completion()/acompletion() for anthropic/* and vertex_ai/claude-*

How it solves it:

  • _is_unsignable_thinking_block() now also returns True when thinking is missing, not a string, or empty after .strip()
  • Signature check runs first and is unchanged; redacted_thinking blocks are untouched

The sequential-mode branch of anthropic_messages_pt() already guards this case with len(thinking_block) > 0 and the comment "don't pass empty text blocks. anthropic api raises errors." But _drop_unsignable_thinking_blocks(), the filter used to build thinking_blocks earlier in the same function and the only guard on the anthropic_messages_pt() -> AnthropicConfig.transform_request() path, calls _is_unsignable_thinking_block() alone. The primary path was left unguarded. #36033 (Responses streaming adapter) and #27850 (Bedrock Converse) fixed the same class of bug elsewhere; this is the same fix on _is_unsignable_thinking_block().

User Flow

Before: a developer whose app replays assistant history through anthropic/claude-* or vertex_ai/claude-* gets a 400 whenever that history contains a thinking_blocks entry with an empty thinking field (for example, forwarded from a non-Anthropic reasoning provider's turn that had no summary text)

  1. Client sends a multi-turn completion() call where messages[1]["thinking_blocks"] = [{"type": "thinking", "thinking": "", "signature": "sig_abc123"}]
  2. The block is kept because it has a non-empty signature; only the signature was ever checked
  3. The outbound Anthropic request contains {"type": "thinking", "thinking": "", "signature": "sig_abc123"}
  4. Anthropic returns 400 messages.N.content.M.thinking: each thinking block must contain thinking and the call raises

After: the same history no longer reaches Anthropic with an empty thinking block

  1. Same client call, same history
  2. The block is dropped because thinking is empty, independent of the signature
  3. The outbound request has no thinking block for that turn
  4. The call proceeds instead of raising a 400

Relevant issues

None filed. Found by code reading in prompt_templates/factory.py and reproduced with a unit test (below).

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally: uv run pytest tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py -v, 102 passed (97 pre-existing + 5 new)
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This is a pure request-transform bug, so the proof is a unit test calling anthropic_messages_pt() directly with the malformed history shape and asserting the outbound content list. No live API key and no mocks; the function does no I/O.

Before (f005afa)

Empty-but-signed thinking block

  1. anthropic_messages_pt() called with messages[1]["thinking_blocks"] = [{"type": "thinking", "thinking": "", "signature": "sig_abc123_looks_valid"}]
  2. pytest -k test_anthropic_messages_pt_drops_empty_but_signed_thinking_block -> FAILED: assert 'thinking' not in ['thinking', 'text'] (the empty block was kept)

Whitespace-only thinking text

  1. _is_unsignable_thinking_block({"type": "thinking", "thinking": " \n\t ", "signature": "sig_abc123_looks_valid"})
  2. pytest -k test_is_unsignable_thinking_block_treats_whitespace_only_as_empty -> FAILED: assert False is True (the function said the block was signable)

After (9fa8d06)

Empty-but-signed thinking block

  1. Same call as above
  2. pytest -k test_anthropic_messages_pt_drops_empty_but_signed_thinking_block -> PASSED, thinking type is absent from the outbound content list

Whitespace-only thinking text

  1. Same call as above
  2. pytest -k test_is_unsignable_thinking_block_treats_whitespace_only_as_empty -> PASSED, function returns True

Full file: uv run pytest tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py -v -> 102 passed, 0 failed

Also ran ruff format --check and ruff check on the two changed files. Did not run the full make test-unit suite (left to CI per CONTRIBUTING.md) or any live Anthropic/Vertex calls.

Type

Bug Fix
Test

Caveats (if any)

Low

  • The emptiness check uses .strip(), so whitespace-only text counts as empty. The sibling len(thinking_block) > 0 check is always combined with and not _is_unsignable_thinking_block(m), so the two agree in practice; this one is the stricter of the two because _drop_unsignable_thinking_blocks() calls it standalone.
  • Related, not a duplicate: open PR fix(utils): keep nested thinking when dropping top-level thinking #37624 fixes a 400 with the same error string via apply_additional_drop_params(). It does not touch this function.

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR extends Anthropic request shaping to discard signed thinking blocks whose text is missing, non-string, empty, or whitespace-only, preventing malformed replayed history from reaching Anthropic-compatible endpoints.

  • Preserves non-empty signed and redacted thinking blocks.
  • Adds focused regression coverage for empty, whitespace-only, unsigned, valid signed, and redacted blocks.
  • The functional change appears sound, with only excessive explanatory comments requiring cleanup.

Confidence Score: 4/5

The PR appears safe to merge after the non-blocking excess-commentary issue is cleaned up.

The stricter predicate consistently drops malformed empty thinking blocks while preserving valid signed and redacted blocks, and the added tests exercise the relevant request transformation without network access.

Files Needing Attention: litellm/litellm_core_utils/prompt_templates/factory.py; tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py

Important Files Changed

Filename Overview
litellm/litellm_core_utils/prompt_templates/factory.py Correctly strengthens thinking-block validation, but adds substantially more explanatory commentary than the straightforward logic needs.
tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py Adds meaningful regression cases in the mapped test file, though the tests contain unnecessarily extensive docstrings.

Reviews (1): Last reviewed commit: "fix(anthropic): drop thinking blocks wit..." | Re-trigger Greptile

Comment on lines +2337 to +2345

Anthropic also rejects a `thinking` block whose `thinking` text is empty or
whitespace-only ("each thinking block must contain thinking"), regardless of
signature. This shape reaches us when a caller replays a `thinking_blocks`
history item that originated from a non-Anthropic reasoning provider (e.g. an
OpenAI Responses-API turn with no summary text) through this Anthropic-shaped
request path (`/v1/chat/completions` -> anthropic/vertex_ai's claude models),
which is the same failure the Anthropic Responses-bridge adapter guards
against (see PR #36033) for its own separate content-block path.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Excessive predicate commentary

The extended request-history explanation around this straightforward signature-and-text predicate, along with similarly extensive test docstrings, duplicates implementation rationale and makes the behavior harder to scan and maintain. Keep the commentary focused on the non-obvious provider constraint.

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing shoemoney:fix/drop-empty-thinking-blocks (dfba491) with litellm_internal_staging (e5da593)

Open in CodSpeed

@shoemoney

Copy link
Copy Markdown
Author

code-quality is failing repo-wide right now, not on this diff. This branch touches only litellm/litellm_core_utils/prompt_templates/factory.py and its test file, no workflow files.

The job exits on a workflow startup invariant check:

ERROR: Workflow startup invariants violated:
  - .github/workflows/test-unit.yml: job `unit` gives pytest 20m but caps the job at 55m.
    Setup can use up to 35m plus 5m of runner overhead, so the job deadline would preempt
    pytest; raise job-timeout-minutes to at least 60.

Same failure on the latest runs for #38055, #38053 and #38051. Leaving it to whoever owns that workflow config rather than touching it from here.

@shoemoney

Copy link
Copy Markdown
Author

CI note: code-quality / check_workflow_startup_safety failure is pre-existing on main and unrelated to this PR's changed files. All 3 shards cap timeout at 55m but need 60m (35m setup + 20m pytest + 5m overhead) on both base f005afa and head; staging already fixed to 60m and passes locally, and this PR only touches factory.py. No code fix required from this PR; rebase/label will clear it.

…st missing signature 🧠🚫

_is_unsignable_thinking_block() only checked block["signature"], so a
thinking block with a valid-looking signature but empty (or
whitespace-only) thinking text sailed through _drop_unsignable_thinking_blocks
and into anthropic_messages_pt(). Anthropic rejects that with:

  400 messages.N.content.M.thinking: each thinking block must contain thinking

This is reachable whenever a thinking_blocks history item gets replayed
through this Anthropic-shaped request path (e.g. a non-Anthropic reasoning
turn with no summary text), the same class of bug PR BerriAI#36033 fixed on the
Responses adapter's own separate code path.

Now the signature check runs first (unsigned blocks are still dropped, same
as before), then an additional check drops the block if `thinking` is
missing, not a string, or strips to empty. redacted_thinking blocks are
untouched since they don't have type == "thinking".
@shoemoney
shoemoney force-pushed the fix/drop-empty-thinking-blocks branch from ceacb95 to dfba491 Compare September 10, 2026 09:50
@yuneng-berri
yuneng-berri deleted the branch BerriAI:litellm_internal_staging September 13, 2026 04:44
@yuneng-berri yuneng-berri reopened this Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants