fix(anthropic): correct content block type detection when content and reasoning_content share a chunk - #31495
Conversation
Greptile SummaryThis PR fixes Anthropic pass-through streaming block detection for mixed reasoning and text chunks. The main changes are:
Confidence Score: 5/5The changes are narrowly scoped to Anthropic pass-through streaming block classification and include focused test coverage for the mixed reasoning/text scenario. The implementation directly addresses the ordering issue described in the streaming content block classifier, and the accompanying tests cover both the direct classifier behavior and wrapper-level streaming output.
What T-Rex did
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
6c2e81e to
afd1508
Compare
…_blocks in block classifier Cherry-pick onto updated upstream (force-pushed litellm_internal_staging).
…th reasoning_content Cover the new elif branch added in the parent fix to satisfy codecov/patch coverage requirements.
…n block classifier Replace the negative 'not hasattr(thinking_blocks)' guard with a natural specific-before-general ordering: thinking_blocks (carries content) is checked first, then reasoning_content (empty placeholder), then text. Functionally equivalent but easier to read, and mirrors the independent-if ordering already used by the non-streaming _translate_openai_content_to_anthropic.
5d05371 to
204f6a5
Compare
|
@Sameerlite this is ready for review, would appreciate a look. Thanks! |
This ValueError is pre-existing upstream logic — Anthropic SSE doesn't put |
|
@Sameerlite Hey bro |
Problem
When an OpenAI streaming chunk carries both
delta.contentandreasoning_content(orthinking_blocks),_translate_streaming_openai_chunk_to_anthropic_content_block()returns"text"instead of"thinking". This is because theelif content > 0check preceded both reasoning-related checks in the elif chain, so mixed chunks always hit the text branch first.As a result,
thinking_deltaevents are emitted into a text content block (SSE block type mismatch), causing thinking content to leak into the user-visible text output that Claude Code processes.Root cause
In
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py, method_translate_streaming_openai_chunk_to_anthropic_content_block, the elif chain orderedcontentbefore both reasoning branches:A duplicate
reasoning_contentcheck at the very bottom of the chain was also unreachable for the same reason.Fix
Reordered the elif chain to check thinking/reasoning branches before text:
This ensures:
thinking_blocks(the richest signal) open athinkingblock containing the actual thinking text and signaturereasoning_content(e.g. vLLM/SGLang reasoning backends) open athinkingplaceholder block"text"as beforeThe now-dead duplicate
reasoning_contentcheck was also removed.Related
streaming_iterator.py(complementary fix)Pre-Submission checklist
@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewScreenshots / Proof of Fix
Integration test via
AnthropicStreamWrapperwith 3 streaming chunks (mixed content+reasoning, pure text, finish_reason):Before fix:
No thinking block was created —
thinking_deltaleaked into a text content block.After fix:
Correct block separation: thinking in its own
thinkingblock (index 1), text in its owntextblock (index 2).Type
🐛 Bug Fix
Changes
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py: reorder elif chain in_translate_streaming_openai_chunk_to_anthropic_content_block— priority order is nowthinking_blocks→reasoning_content→content; remove now-dead duplicatereasoning_contentchecktests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py: add integration testtest_text_not_dropped_when_reasoning_content_shares_chunk; add direct unit testtest_content_block_type_for_mixed_reasoning_and_content; add thinking_deltas assertion