fix(anthropic_adapter): preserve first chunk on content-block transitions

[dkssudgo112]

Relevant issues

Fixes #25214

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added testing in tests/test_litellm/ — new regression suite tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_first_chunk_on_block_transition.py plus updates to test_parallel_tool_calls.py
• My PR passes all unit tests in the affected directory (tests/test_litellm/llms/anthropic/experimental_pass_through/: 179 passed)
• My PR's scope is as isolated as possible — it only fixes the first-chunk-drop bug on content-block transitions in AnthropicStreamWrapper
• I have requested a Greptile review by commenting @greptileai — will do after the PR is open

Type

🐛 Bug Fix
✅ Test

Changes

Problem

The /v1/messages endpoint's AnthropicStreamWrapper silently dropped the trigger chunk of every new content block whenever a block transition was detected (text → thinking, thinking → text, text → tool_use). On the wire, the wrapper emitted content_block_stop → content_block_start and then returned early, discarding the processed_chunk computed from the triggering delta.

The old comment claimed "the content_block_start already carries the relevant information", but _translate_streaming_openai_chunk_to_anthropic_content_block() actually returns an empty body for text transitions:

elif choice.delta.content is not None and len(choice.delta.content) > 0:
    return "text", TextBlock(type="text", text="")   # ← empty!

So the first characters of every new text/thinking block were permanently lost.

Symptoms (reproduced on main @ v1.83.1)

Using /v1/messages with stream=true against a Bedrock Converse reasoning model (minimax.minimax-m2.5, moonshotai.kimi-k2.5, Claude extended thinking):

• Responses start mid-sentence (leading characters missing).
• If the model emits the text as a single Bedrock chunk, the text block is streamed with zero content_block_delta events — clients like claude -p (Claude Code CLI) see an empty response.
• Non-streaming on the same deployment returns the full text correctly, so it's strictly a stream-translation regression.

Raw SSE diff (prompt: Respond with exactly: 안녕하세요, 저는 MiniMax입니다.)

Non-streaming (correct):

\n\n안녕하세요, 저는 MiniMax입니다.

Streaming on main:

event: content_block_start  index=2 content_block={"type": "text", "text": ""}
event: content_block_delta  delta={"type": "text_delta", "text": "녕하세요,"}        ← leading "\n\n안" lost
event: content_block_delta  delta={"type": "text_delta", "text": " 저는 MiniMax입니다."}
event: content_block_stop

Streaming with this PR:

event: content_block_start  index=2 content_block={"type": "text", "text": ""}
event: content_block_delta  delta={"type": "text_delta", "text": "\n\n안녕하세요, 저는 MiniMax입니다."}  ← complete
event: content_block_stop

Fix

In litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py, after emitting the synthetic content_block_stop → content_block_start pair on a detected block transition, also enqueue processed_chunk whenever it is a non-empty content_block_delta. A tiny helper _trigger_delta_has_content() inspects the four Anthropic delta variants (text, thinking, partial_json, signature) so that empty tool_use openers (whose tool name is already carried by content_block_start) are intentionally skipped and existing tool-call test expectations are preserved.

Applied to both the sync __next__ path and the async __anext__ path.

Tests

• New: tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_first_chunk_on_block_transition.py
  • Sync and async regression tests that drive a text → thinking → text sequence through AnthropicStreamWrapper with a mocked ModelResponseStream. They assert both the concatenated content of each block and the event ordering (content_block_start immediately followed by the trigger chunk's delta).
  • Verified they fail on main and pass with this PR (both sync and async).
• Updated: tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py::test_anthropic_stream_wrapper_interleaved_tool_calls_and_text
  • The previous expected sequence was missing the two content_block_delta events for the interleaved text chunks (the wrapper was dropping them). The test now expects those deltas and also asserts the text content round-trips verbatim.
• The full tests/test_litellm/llms/anthropic/experimental_pass_through/ suite (179 tests) is green with the fix.

Scope

• Only touches one production file (streaming_iterator.py) and the /v1/messages experimental pass-through streaming path.
• No behaviour change for /chat/completions, non-streaming /v1/messages, or for providers that don't transition between content-block types.
• No API surface / public contract changes — downstream clients that already handled content_block_delta events keep working, and clients that previously saw truncated or empty text now receive the full content.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 6, 2026 9:20am

[dkssudgo112]

@greptileai please review

[greptile-apps]

Greptile Summary

This PR fixes a silent content-loss bug in AnthropicStreamWrapper where the trigger chunk of every content-block transition (text→thinking, thinking→text, text→tool_use) was discarded. Because content_block_start carries only an empty block body for text/thinking transitions, the first characters of every new block were permanently lost for Bedrock Converse reasoning providers (MiniMax, Kimi, Claude extended thinking).

• Root cause: In both __next__ and __anext__, after emitting content_block_stop → content_block_start on a detected block transition, processed_chunk was silently thrown away instead of being queued.
• Fix: New module-level helper _trigger_delta_has_content() inspects the four Anthropic delta variants (text, thinking, partial_json, signature) and conditionally re-enqueues the trigger delta after content_block_start in both sync and async paths.
• Tool-use openers correctly skipped: Tool-opener trigger chunks carry arguments="" (empty partial_json), so they evaluate as falsy and are not emitted twice — the tool name is already carried by content_block_start.
• Tests: New dedicated regression file test_first_chunk_on_block_transition.py covers sync/async content preservation and event ordering; test_parallel_tool_calls.py strengthened to assert the previously-dropped text deltas are now emitted correctly.

Confidence Score: 5/5

Safe to merge — minimal, symmetric fix across sync/async paths fully covered by new and updated tests

All findings are P2 or lower. The fix is logically correct and well-scoped. Tests explicitly verify the previously-broken behavior and pass with the fix applied. Tool-use paths are verified unaffected by existing tests.

No files require special attention

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py	Adds _trigger_delta_has_content helper and correctly re-enqueues trigger delta after content_block_stop/start pair in both sync and async paths
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_first_chunk_on_block_transition.py	New regression test file with sync, async, and event-ordering tests using in-memory mocks; no network calls
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py	Updated interleaved test to expect the two previously-dropped trigger content_block_delta events and adds text content round-trip assertions

Sequence Diagram

sequenceDiagram
    participant S as Upstream Stream
    participant W as AnthropicStreamWrapper
    participant C as Client

    Note over W,C: Before fix — trigger chunk dropped
    S->>W: chunk (first thinking/text chunk)
    W->>C: content_block_stop
    W->>C: content_block_start (empty body)
    Note over W: processed_chunk discarded

    Note over W,C: After fix — trigger chunk preserved
    S->>W: chunk (first thinking/text chunk)
    W->>C: content_block_stop
    W->>C: content_block_start (empty body)
    W->>C: content_block_delta (trigger chunk content)
    S->>W: subsequent chunk
    W->>C: content_block_delta

Loading

_{Reviews (1): Last reviewed commit: "fix(anthropic_adapter): preserve first c..." | Re-trigger Greptile}

[collinstraka]

We're hitting this bug in production — LiteLLM proxying to Vertex AI with Claude Opus/Sonnet extended thinking models. The thinking→tool_use content block transition intermittently drops tool_use argument chunks, which causes downstream clients (Claude Code in our case) to receive empty/invalid tool inputs and enter retry loops.

The failure is non-deterministic and consistent with a chunk-boundary race condition. Disabling extended thinking eliminates it, confirming the transition logic as the root cause.

The Greptile review came back 5/5 confidence and the fix is small and symmetric across both paths. Would appreciate a human reviewer picking this up — it's a meaningful quality-of-life fix for anyone streaming extended thinking models through /v1/messages.

[icsy7867]

Any insight on if this can get merged soon?

[tomaskir]

Confirming this still in 1.87.1 - and it's not Bedrock-specific. The same first-chunk drop happens for OpenAI-compatible reasoning backends (vLLM / SGLang reasoning parsers - DeepSeek-R1, Qwen3-reasoning, gpt-oss) served through the /v1/messages adapter. Symptom in Claude Code: the visible answer starts mid-word, and occasionally a text block streams with zero content_block_deltas so CC shows an empty response. stream=false is unaffected.

Where it bites is together with #29533 / #29600: once that change emits a real thinking block for reasoning_content, every reasoning turn produces a genuine thinking → text transition - so this drop eats the first token(s) of the answer on every response. In practice #25216 is a prerequisite for #29533 being usable from Claude Code.

I tested the same fix independently (re-queue the trigger chunk whenever its delta carries content - text/thinking/signature/non-empty tool args - and skip empty tool openers), so the approach here looks right to me. It just needs a rebase; it's currently showing conflicted against main.

One caveat for reviewers: this fixes the transition drop, but not the case where a single upstream chunk carries both reasoning_content and content. vLLM emits exactly that when </think> lands mid-delta (DeltaMessage(reasoning=…, content=…)); _translate_streaming_openai_chunk_to_anthropic then prioritizes reasoning and drops the content - and with this PR that chunk re-queues a thinking_delta into the just-opened text block.

That one's in the translator rather than the wrapper, reported in #27492 and isn't addressed here. With this PR the combined chunk re-queues a thinking_delta into the new text block. Full reasoning-model streaming needs both fixes.

[Sameerlite]

@dkssudgo112 can you rebase this? Thank you!