Skip to content

fix(anthropic): preserve text delta when opening stream block - #30145

Closed
its-amann wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
its-amann:preserve-anthropic-text-deltas
Closed

fix(anthropic): preserve text delta when opening stream block#30145
its-amann wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
its-amann:preserve-anthropic-text-deltas

Conversation

@its-amann

@its-amann its-amann commented Jun 10, 2026

Copy link
Copy Markdown

Relevant issues

Fixes #30014

Summary

Anthropic stream wrapping was dropping the first non empty delta that opened a new content block unless that delta was a tool argument. In the /v1/messages to /v1/chat/completions path, that can make the first text chunk after a block transition disappear from the client stream.

This keeps the existing tool argument behavior and also queues non empty text, thinking, signature, and tool argument deltas when they trigger a content block transition. I added sync and async regression coverage for switching from thinking output back to text output.

Checklist

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have requested a Greptile review and received a Confidence Score of at least 4/5 before requesting a maintainer review

Tests

  • uv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py::TestAnthropicStreamWrapperTextDeltas -q
  • uv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py::TestAnthropicStreamWrapperToolArgs tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py::TestAnthropicStreamWrapperTextDeltas -q
  • uv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py -q
  • uv run ruff check litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py
  • git diff --cached --check

@CLAassistant

CLAassistant commented Jun 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@greptile-apps

greptile-apps Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a bug where the first non-empty delta on a newly opened content block was silently dropped during Anthropic stream wrapping, unless that delta was input_json_delta. The fix generalizes the existing "queue delta at block transition" logic to cover text_delta, thinking_delta, and signature_delta in addition to input_json_delta, and extracts the guard into a shared helper used by both the sync and async paths.

  • _should_queue_block_transition_delta replaces two identical inline conditions in __next__ and __anext__, making the extension one-line rather than requiring matched edits in two places.
  • New sync/async regression tests cover thinking→text and text→thinking transitions, and the existing interleaved-tool-calls test is strengthened with explicit assertions on the text delta values that were previously dropped.

Confidence Score: 5/5

Safe to merge — the change is isolated to the block-transition queueing logic and is fully covered by new regression tests in both sync and async paths.

The fix is a narrow, well-scoped generalization of an existing guard: the old code only preserved input_json_delta at block transitions; the new helper extends that to text_delta, thinking_delta, and signature_delta as well. Both sync and async paths are updated symmetrically. New regression tests cover both directions (thinking→text and text→thinking) in sync and async flavors, and the existing interleaved-tool-calls test is now stricter rather than looser.

No files require special attention.

Important Files Changed

Filename Overview
litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py Adds _should_queue_block_transition_delta helper and wires it into both the sync and async block-transition paths; replaces the prior input_json_delta-only guard cleanly.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py Adds TestAnthropicStreamWrapperTextDeltas class with sync/async tests for both the thinking→text and text→thinking block transitions; also unit-tests the helper's edge cases directly.
tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py Strengthens the interleaved-tool-calls-and-text test by adding the two text content_block_delta events that were previously dropped to expected_types and asserting their text values.

Reviews (2): Last reviewed commit: "test(anthropic): cover block transition ..." | Re-trigger Greptile

@Sameerlite

Copy link
Copy Markdown
Contributor

Thanks for the fix to preserve the opening delta! Two small things:

  • New commits have landed since Greptile last reviewed, so I'm re-triggering a fresh review (below).
  • Could you paste the captured test output (or a short before/after stream transcript)? Listing the commands is helpful, but seeing the result confirms it.

@greptileai

Once that's in we'll take another look!

@Sameerlite Sameerlite closed this Jun 11, 2026
@its-amann

Copy link
Copy Markdown
Author

Thanks for taking a look, and sorry I missed this before the PR was closed.

I see #30024 has since landed and closed #30014, so this PR is duplicate now. For completeness, here is the captured output from the final branch, since you asked for it:

$ uv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py --cov=litellm --cov-report=xml -q
83 passed in 48.48s

$ uv run diff-cover coverage.xml --compare-branch origin/litellm_internal_staging --include 'litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py'
streaming_iterator.py (100%)
Total: 14 lines
Missing: 0 lines
Coverage: 100%

$ uv run --no-sync mypy litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py
Success: no issues found in 1 source file

$ uv run ruff check litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py
All checks passed

$ uv run black --check litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py
3 files would be left unchanged

The behavior covered on this branch was that the transition chunk is emitted as a content_block_delta after content_block_start for both thinking to text and text to thinking transitions, instead of being swallowed. No action needed here if #30024 is the accepted fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Anthropic /v1/messages streaming adapter drops the first non-empty text delta when proxying to /v1/chat/completions

3 participants