fix: content_block_delta type mismatch when reasoning and content arrive in one chunk

[streber42]

Relevant issues

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Proxy config: self-hosted vLLM backend with a reasoning-capable Qwen model behind an openai/ deployment, Claude Code pointed at the proxy's /v1/messages endpoint

Captured live from the failing deployment before the fix. The vLLM reasoning parser flushed the tail of reasoning_content bundled with the first character of the answer in one streaming chunk. AnthropicStreamWrapper opened the new content block as text (block-type detection prefers content) but emitted a thinking_delta into it (delta emission prefers reasoning), producing this SSE sequence:

content_block_stop  {"index": 1}
content_block_start {"index": 2, "content_block": {"type": "text", "text": ""}}
content_block_delta {"index": 2, "delta": {"type": "thinking_delta", "thinking": "\n"}}

Claude Code (a strict Anthropic SSE client) rejects this as "Content block is not a thinking block". Because the corrupted assistant turn gets replayed as conversation history, every subsequent turn in the same context fails the same way until the context is cleared.

After the fix, hitting the same backend with a prompt that forces a reasoning/answer transition, and reducing the captured event stream to (index -> declared block type) then checking every delta against its block's declared type:

$ curl -sN http://localhost:4000/v1/messages \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "my-reasoning-model", "max_tokens": 1500, "stream": true,
         "thinking": {"type": "enabled", "budget_tokens": 1024},
         "messages": [{"role": "user", "content": "..."}]}' \
  | grep -oE "event: [a-z_]+" | sort | uniq -c

    462 event: content_block_delta
      3 event: content_block_start
      3 event: content_block_stop
      1 event: message_delta
      1 event: message_start
      1 event: message_stop

block types opened: {0: "text", 1: "thinking", 2: "text"} (empty leading text block, then thinking, then the answer), zero delta/block-type mismatches across all 462 deltas

Type

🐛 Bug Fix

Changes

_CombinedChunkSplitter previously only split a chunk that combined response content with a finish_reason (needed for fake-streamed providers like Vertex AI Gemma :predict, which collapse the whole response into one chunk). It did not split a chunk that combined reasoning_content/thinking_blocks with content in the same delta, which some vLLM reasoning parsers do at the reasoning/answer boundary.

That combined shape hit two functions that disagree on which payload is dominant: _should_start_new_content_block (content-block-type detection) checks content before reasoning_content and picks text, while translate_streaming_openai_response_to_anthropic (delta emission) checks reasoning_content first and picks thinking_delta. The mismatch means a thinking_delta lands on a block the client was told is text.

Generalized _CombinedChunkSplitter to split on any combination of {reasoning, content/tool_calls, finish_reason} rather than only {content, finish_reason}, so each downstream chunk carries exactly one payload and both functions agree on its type. Existing content+finish_reason behavior is unchanged; a chunk carrying reasoning+content+finish_reason now splits into three single-payload chunks instead of two.

Added a regression test (test_streaming_iterator_reasoning_content_chunk.py) that replays the captured chunk shape (reasoning tail bundled with content) through the full AnthropicStreamWrapper and asserts no delta ever targets a block of the wrong declared type, and that both payloads survive as separate deltas. Updated the existing combined-chunk splitter test that asserted the old (incorrect) two-way split behavior for a reasoning+content+finish chunk to assert the corrected three-way split.

[streber42]

@greptileai

[greptile-apps]

Greptile Summary

This PR fixes a thinking_delta/text block-type mismatch that corrupted streaming responses from vLLM reasoning backends: when the reasoning parser flushed the tail of reasoning_content bundled with the first token of content in one chunk, _should_start_new_content_block and translate_streaming_openai_response_to_anthropic disagreed on which payload was dominant, causing strict Anthropic SSE clients to reject the response.

• Generalizes _CombinedChunkSplitter._split to produce one chunk per semantic payload (reasoning → content/tool_calls → finish_reason) instead of always exactly two, correctly handling all combinations of the three payload types.
• Extracts _has_reasoning and _has_content_or_tool_calls static helpers used consistently by both _is_combined (detection) and _split (production), eliminating the disagreement between the two sites.
• Adds a regression test (test_streaming_iterator_reasoning_content_chunk.py) that drives the full AnthropicStreamWrapper end-to-end and asserts zero block-type/delta-type mismatches, plus updates the existing splitter test to assert the corrected three-way split.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to the splitter that pre-processes chunks before the stream wrapper, existing fake-stream behavior is preserved, and the new regression test drives the exact failure scenario end-to-end.

The fix is well-contained: _is_combined and _split use the same two helper methods, so detection and production stay in sync. The invariant that _split always returns at least one chunk is maintained. Existing tests for the content+finish_reason two-way split still pass unchanged. The new test covers the previously-uncovered reasoning+content boundary shape and verifies no delta type ever lands on a block of the wrong declared type.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py	Generalized _CombinedChunkSplitter._split to produce one chunk per payload (reasoning, content/tool_calls, finish_reason) instead of always two; helper statics extracted for reuse; logic is consistent between _is_combined (detection) and _split (production)
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_streaming_iterator_combined_chunk.py	Existing test updated to assert the corrected three-way split; new assertions are strictly stronger than the old ones and all prior passing test cases remain unchanged
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_streaming_iterator_reasoning_content_chunk.py	New regression test replays the captured vLLM boundary-chunk shape through the full AnthropicStreamWrapper, asserts zero delta/block-type mismatches, and verifies both payloads survive as separate deltas; no real network calls

_{Reviews (1): Last reviewed commit: "fix: content_block_delta type mismatch w..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes a thinking_delta/text block-type mismatch that occurred when a vLLM reasoning parser flushed the tail of reasoning_content bundled with the first token of content in a single streaming chunk. _CombinedChunkSplitter._split is generalized from a two-way content+finish split to a three-way reasoning / content / finish split, ensuring each downstream chunk carries exactly one semantic payload.

• _CombinedChunkSplitter gains two new static helpers (_has_reasoning, _has_content_or_tool_calls) and rewrites _is_combined / _split to count all three payload types; a reasoning+content+finish chunk now produces three single-payload chunks in the correct order.
• Existing test updated to assert the corrected three-way split with more granular per-chunk assertions.
• New regression test replays the captured vLLM boundary chunk through the full AnthropicStreamWrapper and verifies zero delta/block-type mismatches and that both payloads survive.

Confidence Score: 4/5

Safe to merge; the change is tightly scoped to the splitter pre-processing layer and does not touch the main translation or state-machine logic in AnthropicStreamWrapper.

The logic change is small and well-isolated: only _CombinedChunkSplitter._is_combined and _split are rewritten, and the three-way split correctly mirrors what the downstream pair already expects. Tests cover both split mechanics and end-to-end SSE ordering. The two findings are a stale inline comment and a bare dict subscript in the test helper that produces an unhelpful KeyError for unknown delta types — neither affects correctness.

No files require special attention; the existing-test update correctly strengthens rather than weakens coverage.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py	Generalized _CombinedChunkSplitter._split to handle reasoning+content co-arrival at the vLLM boundary, correctly producing 3 single-payload chunks instead of 2; minor stale inline comment in __init__
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_streaming_iterator_combined_chunk.py	Existing combined-chunk test updated to expect correct 3-way split for reasoning+content+finish chunks; new assertions are more granular and accurate than before
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_streaming_iterator_reasoning_content_chunk.py	New regression test exercising the full AnthropicStreamWrapper with a boundary chunk that bundles reasoning tail + first answer token; no real network calls

Comments Outside Diff (1)

1. litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py, line 208-211 (link)

   The __init__ comment still describes the old (narrower) behavior of _CombinedChunkSplitter — splitting only content+finish_reason pairs into two chunks. It no longer reflects the generalized logic that handles reasoning+content co-arrival and can produce up to three chunks.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (2): Last reviewed commit: "fix: content_block_delta type mismatch w..." | Re-trigger Greptile}

[greptile-apps]

The helper uses a bare dict subscript for delta_type lookup, so any delta type not in the four-entry map (e.g. compaction_delta) raises an unhelpful KeyError rather than a descriptive assertion failure. Using .get() with a fallback and an explicit assert makes failures much easier to diagnose if a new event type slips into this test's stream.

Suggested change

	expected_block_type = {
	"text_delta": "text",
	"thinking_delta": "thinking",
	"signature_delta": "thinking",
	"input_json_delta": "tool_use",
	}[delta_type]
	expected_block_type = {
	"text_delta": "text",
	"thinking_delta": "thinking",
	"signature_delta": "thinking",
	"input_json_delta": "tool_use",
	}.get(delta_type)
	assert expected_block_type is not None, (
	f"Unknown delta type {delta_type!r}; extend the map above if intentional"
	)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[streber42]

Closing: litellm_internal_staging's _CombinedChunkSplitter already generalizes to reasoning+content chunks, so this specific fix is now redundant.