fix(anthropic): preserve text delta when opening stream block - #30145
fix(anthropic): preserve text delta when opening stream block#30145its-amann wants to merge 3 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes a bug where the first non-empty delta on a newly opened content block was silently dropped during Anthropic stream wrapping, unless that delta was
Confidence Score: 5/5Safe to merge — the change is isolated to the block-transition queueing logic and is fully covered by new regression tests in both sync and async paths. The fix is a narrow, well-scoped generalization of an existing guard: the old code only preserved No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.py | Adds _should_queue_block_transition_delta helper and wires it into both the sync and async block-transition paths; replaces the prior input_json_delta-only guard cleanly. |
| tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py | Adds TestAnthropicStreamWrapperTextDeltas class with sync/async tests for both the thinking→text and text→thinking block transitions; also unit-tests the helper's edge cases directly. |
| tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_parallel_tool_calls.py | Strengthens the interleaved-tool-calls-and-text test by adding the two text content_block_delta events that were previously dropped to expected_types and asserting their text values. |
Reviews (2): Last reviewed commit: "test(anthropic): cover block transition ..." | Re-trigger Greptile
|
Thanks for the fix to preserve the opening delta! Two small things:
Once that's in we'll take another look! |
|
Thanks for taking a look, and sorry I missed this before the PR was closed. I see #30024 has since landed and closed #30014, so this PR is duplicate now. For completeness, here is the captured output from the final branch, since you asked for it: The behavior covered on this branch was that the transition chunk is emitted as a |
Relevant issues
Fixes #30014
Summary
Anthropic stream wrapping was dropping the first non empty delta that opened a new content block unless that delta was a tool argument. In the
/v1/messagesto/v1/chat/completionspath, that can make the first text chunk after a block transition disappear from the client stream.This keeps the existing tool argument behavior and also queues non empty text, thinking, signature, and tool argument deltas when they trigger a content block transition. I added sync and async regression coverage for switching from thinking output back to text output.
Checklist
Tests
uv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py::TestAnthropicStreamWrapperTextDeltas -quv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py::TestAnthropicStreamWrapperToolArgs tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py::TestAnthropicStreamWrapperTextDeltas -quv run pytest tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py -quv run ruff check litellm/llms/anthropic/experimental_pass_through/adapters/streaming_iterator.pygit diff --cached --check