Skip to content

fix(anthropic): correct content block type detection when content and reasoning_content share a chunk - #31495

Open
Yan233th wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
Yan233th:fix/streaming-text-dropped-with-reasoning
Open

fix(anthropic): correct content block type detection when content and reasoning_content share a chunk#31495
Yan233th wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
Yan233th:fix/streaming-text-dropped-with-reasoning

Conversation

@Yan233th

@Yan233th Yan233th commented Jun 27, 2026

Copy link
Copy Markdown

Problem

When an OpenAI streaming chunk carries both delta.content and reasoning_content (or thinking_blocks), _translate_streaming_openai_chunk_to_anthropic_content_block() returns "text" instead of "thinking". This is because the elif content > 0 check preceded both reasoning-related checks in the elif chain, so mixed chunks always hit the text branch first.

As a result, thinking_delta events are emitted into a text content block (SSE block type mismatch), causing thinking content to leak into the user-visible text output that Claude Code processes.

Root cause

In litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py, method _translate_streaming_openai_chunk_to_anthropic_content_block, the elif chain ordered content before both reasoning branches:

elif choice.delta.content is not None and len(choice.delta.content) > 0:
    return "text", TextBlock(type="text", text="")       # caught mixed chunks

# ... never reached when content was also present:
elif isinstance(choice, StreamingChoices) and hasattr(choice.delta, "thinking_blocks"):
    ...
    return "thinking", ...

elif isinstance(choice, StreamingChoices) and getattr(choice.delta, "reasoning_content", None):
    return "thinking", ...

A duplicate reasoning_content check at the very bottom of the chain was also unreachable for the same reason.

Fix

Reordered the elif chain to check thinking/reasoning branches before text:

tool_use → thinking_blocks → reasoning_content → content → text (fallback)

This ensures:

  • Chunks with thinking_blocks (the richest signal) open a thinking block containing the actual thinking text and signature
  • Chunks with only reasoning_content (e.g. vLLM/SGLang reasoning backends) open a thinking placeholder block
  • Text-only chunks fall through to "text" as before

The now-dead duplicate reasoning_content check was also removed.

Related

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Integration test via AnthropicStreamWrapper with 3 streaming chunks (mixed content+reasoning, pure text, finish_reason):

Before fix:

Block types: ['text']

No thinking block was created — thinking_delta leaked into a text content block.

After fix:

Block types: ['text', 'thinking', 'text']

Correct block separation: thinking in its own thinking block (index 1), text in its own text block (index 2).

Type

🐛 Bug Fix

Changes

  • litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py: reorder elif chain in _translate_streaming_openai_chunk_to_anthropic_content_block — priority order is now thinking_blocksreasoning_contentcontent; remove now-dead duplicate reasoning_content check
  • tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py: add integration test test_text_not_dropped_when_reasoning_content_shares_chunk; add direct unit test test_content_block_type_for_mixed_reasoning_and_content; add thinking_deltas assertion

@CLAassistant

CLAassistant commented Jun 27, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@greptile-apps

greptile-apps Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes Anthropic pass-through streaming block detection for mixed reasoning and text chunks. The main changes are:

  • Reasoning and thinking signals are checked before text content when opening streaming content blocks.
  • Mixed content plus reasoning_content chunks now open a thinking block instead of a text block.
  • Tests cover the direct block classifier and the wrapper-level streaming output.

Confidence Score: 5/5

The changes are narrowly scoped to Anthropic pass-through streaming block classification and include focused test coverage for the mixed reasoning/text scenario.

The implementation directly addresses the ordering issue described in the streaming content block classifier, and the accompanying tests cover both the direct classifier behavior and wrapper-level streaming output.

T-Rex T-Rex Logs

What T-Rex did

  • Ran the mixed streaming reasoning repro test and confirmed both runs exited with code 0.
  • Verified the after-state shows block_types: text, thinking, text with thinking_delta and text_delta emitted in separate blocks.
  • Ran the classifier-priority repro test and observed the before-state shows mixed content in a single text block, while the after-state separates content+reasoning_content with thinking and content-only as text.
  • Observed that content+thinking_blocks containing both thinking and a signature raised a ValueError in the after-state.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. General comment

    P1 content+thinking_blocks with thinking and signature raises instead of returning a thinking content block

    • Bug
      • The requested contract case constructs a streaming delta with non-empty content plus a thinking_blocks entry containing both thinking text and signature. On head commit 204f6a5, _translate_streaming_openai_chunk_to_anthropic_content_block raises ValueError: Both thinkingandsignature in a single streaming chunk isn't supported. The validation objective expected this mixed thinking_blocks+content case to return block_type: thinking and a content block preserving the actual thinking/signature fields.
    • Cause
      • In the changed thinking_blocks branch of the classifier, the helper prioritizes thinking_blocks before content, but then explicitly rejects a single thinking block containing both thinking and signature before returning the Anthropic thinking block.
    • Fix
      • If this contract is intended, adjust the thinking_blocks classifier path to return a thinking content block that preserves both fields for the block-start classification case, or update the contract/tests to avoid constructing a single thinking block with both fields when that shape is unsupported.

    T-Rex Ran code and verified through T-Rex

Reviews (8): Last reviewed commit: "refactor(anthropic): order thinking_bloc..." | Re-trigger Greptile

@codecov

codecov Bot commented Jun 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Yan233th
Yan233th force-pushed the fix/streaming-text-dropped-with-reasoning branch from 6c2e81e to afd1508 Compare June 27, 2026 10:26
@Yan233th

Copy link
Copy Markdown
Author

@greptileai

@Yan233th

Copy link
Copy Markdown
Author

@greptileai

Yan233th added 4 commits June 28, 2026 04:57
…_blocks in block classifier

Cherry-pick onto updated upstream (force-pushed litellm_internal_staging).
…th reasoning_content

Cover the new elif branch added in the parent fix to satisfy codecov/patch
coverage requirements.
…n block classifier

Replace the negative 'not hasattr(thinking_blocks)' guard with a natural
specific-before-general ordering: thinking_blocks (carries content) is
checked first, then reasoning_content (empty placeholder), then text.
Functionally equivalent but easier to read, and mirrors the independent-if
ordering already used by the non-streaming _translate_openai_content_to_anthropic.
@Yan233th
Yan233th force-pushed the fix/streaming-text-dropped-with-reasoning branch from 5d05371 to 204f6a5 Compare June 28, 2026 08:05
@Yan233th

Copy link
Copy Markdown
Author

@greptileai

@Yan233th

Yan233th commented Jun 28, 2026

Copy link
Copy Markdown
Author

@Sameerlite this is ready for review, would appreciate a look. Thanks!

@Yan233th

Copy link
Copy Markdown
Author

Greptile Summary

This PR fixes Anthropic pass-through streaming block detection for mixed reasoning and text chunks. The main changes are:

  • Reasoning and thinking signals are checked before text content when opening streaming content blocks.
  • Mixed content plus reasoning_content chunks now open a thinking block instead of a text block.
  • Tests cover the direct block classifier and the wrapper-level streaming output.

Confidence Score: 5/5

The changes are narrowly scoped to Anthropic pass-through streaming block classification and include focused test coverage for the mixed reasoning/text scenario.

The implementation directly addresses the ordering issue described in the streaming content block classifier, and the accompanying tests cover both the direct classifier behavior and wrapper-level streaming output.

T-Rex T-Rex Logs

What T-Rex did

  • Ran the mixed streaming reasoning repro test and confirmed both runs exited with code 0.
  • Verified the after-state shows block_types: text, thinking, text with thinking_delta and text_delta emitted in separate blocks.
  • Ran the classifier-priority repro test and observed the before-state shows mixed content in a single text block, while the after-state separates content+reasoning_content with thinking and content-only as text.
  • Observed that content+thinking_blocks containing both thinking and a signature raised a ValueError in the after-state.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. General comment
    P1 content+thinking_blocks with thinking and signature raises instead of returning a thinking content block

    • Bug

      • The requested contract case constructs a streaming delta with non-empty content plus a thinking_blocks entry containing both thinking text and signature. On head commit 204f6a5, _translate_streaming_openai_chunk_to_anthropic_content_block raises ValueError: Both thinkingandsignature in a single streaming chunk isn't supported. The validation objective expected this mixed thinking_blocks+content case to return block_type: thinking and a content block preserving the actual thinking/signature fields.
    • Cause

      • In the changed thinking_blocks branch of the classifier, the helper prioritizes thinking_blocks before content, but then explicitly rejects a single thinking block containing both thinking and signature before returning the Anthropic thinking block.
    • Fix

      • If this contract is intended, adjust the thinking_blocks classifier path to return a thinking content block that preserves both fields for the block-start classification case, or update the contract/tests to avoid constructing a single thinking block with both fields when that shape is unsupported.

    T-Rex Ran code and verified through T-Rex

Reviews (8): Last reviewed commit: "refactor(anthropic): order thinking_bloc..." | Re-trigger Greptile

This ValueError is pre-existing upstream logic — Anthropic SSE doesn't put thinking and signature in the same chunk. It was previously masked by the content bug; now that the ordering is fixed, it surfaces correctly. Not a regression introduced by this PR.

@Yan233th

Yan233th commented Jul 3, 2026

Copy link
Copy Markdown
Author

@Sameerlite Hey bro

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants