fix(responses): preserve streaming output in Langfuse

[yunhungo]

TLDR

Problem this solves:

• Streaming Responses output is missing from Langfuse
• Empty terminal output discards preceding completed items

How it solves it:

• Recover output from preceding completed stream items
• Unwrap completion events before Langfuse output serialization

User Flow

Before: a developer streams a Responses API call successfully, but Langfuse records no generation output

1. They send POST http://localhost:4000/v1/responses with "stream": true
2. Their app receives text deltas and the terminal response.completed event
3. They open the matching Langfuse generation and see token usage, but Output is undefined

After: the same streamed call records its assistant output in Langfuse

1. They send POST http://localhost:4000/v1/responses with "stream": true
2. Their app receives text deltas and the terminal response.completed event
3. They open the matching Langfuse generation and see the assistant message in Output

Relevant issues

Fixes #36361

Related to #25429 and #31332

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proof is pending while this PR remains a draft. The host used for the original reproduction is currently unreachable from the test runner, so no after screenshot is claimed here

Type

Bug Fix

Changes

The Responses streaming iterator now retains post-hook output_item.done and output_text.done data. When a provider sends an empty terminal output, the iterator rebuilds it with the existing shared SSE recovery helpers before success callbacks run

The Langfuse OTEL integration now reads output from the nested Responses object when the callback result is a response.completed event

Regression coverage exercises sync output-item SSE recovery, async text-only SSE recovery, and the Langfuse OTEL wrapper. Both affected test files pass with 40 tests

Ruff checks, the strict-rule budget, and the type-discipline budget pass. The complete basedpyright gate cannot provision locally because the repository Rust dependency graph requires rustc 1.94.1 while the runner has 1.89.0

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing yunhungo:litellm_fix_langfuse_streaming_output (399e9fb) with litellm_internal_staging (f6b9518)}

[greptile-apps]

Greptile Summary

The PR reconstructs empty terminal Responses output from preceding streaming events and unwraps completed-event payloads for Langfuse OTEL serialization.

• Accumulates output-item and output-text completion events in synchronous and asynchronous Responses iterators.
• Restores accumulated items before completion callbacks and cache persistence.
• Reads Langfuse output from the nested response carried by response.completed.
• Adds regression coverage for synchronous, asynchronous, and Langfuse output recovery.

Confidence Score: 4/5

The PR needs a compatibility fix before merging because a valid type-changing streaming hook can now terminate the stream.

The recovery behavior works for normal Pydantic stream events, but both iterator paths unconditionally serialize the post-hook result even though the hook contract permits arbitrary replacement values.

Files Needing Attention: litellm/responses/streaming_iterator.py

Important Files Changed

Filename	Overview
litellm/responses/streaming_iterator.py	Adds terminal-output recovery, but the new unconditional model_dump call breaks hooks that validly return non-Pydantic replacement chunks.
litellm/integrations/langfuse/langfuse_otel.py	Correctly unwraps a completed Responses event while retaining fallback behavior for direct response objects.
tests/test_litellm/responses/test_streaming_iterator.py	Covers sync item recovery and async text-only recovery, but does not exercise a type-changing streaming hook.
tests/test_litellm/integrations/test_langfuse_otel.py	Adds focused coverage proving nested completed-event output is serialized for Langfuse.

_{Reviews (1): Last reviewed commit: "fix(responses): preserve streaming outpu..." | Re-trigger Greptile}

[greptile-apps]

Hook replacements break accumulation

When a streaming deployment hook returns a replacement such as a plain dictionary, _accumulate_streamed_output unconditionally calls model_dump() on it, causing an AttributeError that terminates the stream and invokes failure handling instead of returning the modified chunk.

Knowledge Base Used: Logging & Observability Integrations