fix(streaming): preserve unsigned thinking blocks in stream_chunk_builder - #33035
Conversation
…lder ChunkProcessor.get_combined_thinking_content only emitted a thinking block when both accumulated text AND a signature were present. The per-block signature arrives only at the end of a thinking block, so any block whose signature never lands (a stream truncated mid-thinking by max_tokens or cancellation, or a provider that emits thinking text without per-block signatures) had all of its accumulated reasoning text silently discarded, and thinking_blocks came back as None. signature is optional on ChatCompletionThinkingBlock, and the Anthropic streaming handler itself emits thinking blocks with text and no signature, so the assembled response must keep the text. Emit the block whenever text exists and attach the signature only when present. The send-side prompt transforms already drop unsignable blocks before replaying them, so this does not risk invalid replay. Signed and redacted paths are unchanged; adds a regression test.
Greptile SummaryThis PR fixes a silent data-loss bug in
Confidence Score: 5/5Safe to merge — the change is minimal, well-scoped, and strictly additive: callers that previously received unsigned thinking blocks as None will now receive the accumulated text instead, with no impact on signed or redacted paths. The one-line condition change in _flush_thinking_block is straightforward, the TypedDict already declares signature as Optional[str], and the production caller in main.py uses the same chunk-filtering pattern that the new test exercises. No existing test expectations were weakened. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/litellm_core_utils/streaming_chunk_builder_utils.py | Fixes silent data loss in _flush_thinking_block by emitting unsigned thinking blocks; signature is attached only when present, matching the Optional[str] typing on ChatCompletionThinkingBlock. |
| tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py | Adds a focused regression test for the unsigned-thinking-block bug; mirrors the same chunk-filtering pattern used by the production caller in main.py and by the existing interleaved-block test. |
Reviews (1): Last reviewed commit: "fix(streaming): preserve unsigned thinki..." | Re-trigger Greptile
|
Heads up: the red |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Relevant issues
Fixes #33034
Pre-Submission checklist
Screenshots / Proof of Fix
Reasoning-content data loss during streamed-response assembly, reproducible offline (no keys/network).
Before (base
3d63eda) — unsigned thinking text is dropped:After (this PR) — text preserved; signed/redacted paths unchanged:
New regression test (fails on base, passes here) + existing thinking tests still green:
Type
🐛 Bug Fix
Changes
ChunkProcessor.get_combined_thinking_contentrequired both accumulated text and a signature to emit a thinking block, so a block whose signature never arrived (truncated stream, or a provider without per-block signatures) lost all its reasoning text. Emit the block whenever text exists and attach the signature only when present.signatureisOptionalonChatCompletionThinkingBlock, and the send-side prompt transforms already drop unsignable blocks before replaying them, so this does not risk invalid replay. Signed and redacted paths are unchanged.cc @ishaan-jaff @krrish-berri-2 — isolated data-loss fix in stream assembly with a regression test. Thanks!