fix(responses): preserve combined reasoning and content

[weddle]

TLDR

Problem this solves:

• Combined reasoning and text chunks drop incremental visible text
• Reasoning deltas can arrive before their summary part opens

How it solves it:

• Expand each upstream chunk into all ordered Responses events
• Share the event queue across sync and async iteration
• Open the reasoning summary part before streaming its deltas

User Flow

Before: a developer's stream omits text and opens reasoning incompletely

1. They send POST https://litellm-domain/v1/responses with stream: true, reasoning.effort: high, and a required marker in the input
2. They receive HTTP 200 and a reasoning item, then response.reasoning_summary_text.delta without response.reasoning_summary_part.added
3. They concatenate response.output_text.delta values and receive A begin middle end, while response.output_text.done contains MARKER_ALPHA begin middle end

After: the same stream opens reasoning and preserves all visible text

1. They send POST https://litellm-domain/v1/responses with stream: true, reasoning.effort: high, and the same required marker in the input
2. They receive HTTP 200, a reasoning item, response.reasoning_summary_part.added, then response.reasoning_summary_text.delta
3. They concatenate response.output_text.delta values and receive MARKER_ALPHA begin middle end, which matches response.output_text.done

Relevant issues

Fixes #36327

Related to #27492, #25216, #30014, #32777, and #33938

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If this PR is delayed, I will ask the LiteLLM team in Slack #pr-review

Screenshots / Proof of Fix

The proxy hostname and response IDs are redacted. Both deployments used the same model route, request, and provider credentials. All requests used the live DeepInfra API without mocks

Request used for every proxy trial

curl -N -sS -w '\nHTTP %{http_code}\n' \
  "$LITELLM_BASE_URL/v1/responses" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' \
  --data-binary @- <<'JSON'
{
  "model": "deepseek-v4-flash",
  "input": "Reason carefully, then answer with exactly this single line and no other visible text: MARKER_ALPHA begin middle end",
  "stream": true,
  "reasoning": {"effort": "high"}
}
JSON

Before

Source revision 065bf83f0cac68d052b2a7f5ca8e3a1b764a2ec1, LiteLLM 1.94.1

A direct DeepInfra control streamed the complete answer. Its final Chat Completions chunk contained both payloads

{
  "delta": {
    "content": "MARKER_ALPHA begin middle end",
    "reasoning_content": " reasoning visible."
  },
  "finish_reason": "stop"
}

The proxy lost the text prefix in three of four trials. The terminal done event remained complete in all trials

trial 1 response.output_text.delta: A begin middle end
trial 1 response.output_text.done:  MARKER_ALPHA begin middle end

trial 2 response.output_text.delta: ALPHA begin middle end
trial 2 response.output_text.done:  MARKER_ALPHA begin middle end

trial 3 response.output_text.delta: A begin middle end
trial 3 response.output_text.done:  MARKER_ALPHA begin middle end

trial 4 response.output_text.delta: MARKER_ALPHA begin middle end
trial 4 response.output_text.done:  MARKER_ALPHA begin middle end

After

The live provider proof was captured at original fix commit ecf52e0feee53f010d918b5a01ec2c6f4ad8cfda. The deployed lifecycle repair commit 43dd8496c306f433035a1e652546addef9f4aeac preserves that behavior while correcting event ordering

Three live proxy trials returned HTTP 200. Each trial emitted reasoning before text. Each incremental text stream contained the complete marker and matched the terminal done event

trial 1 HTTP 200
response.output_text.delta: MARKER_ALPHA begin middle end
response.output_text.done:  MARKER_ALPHA begin middle end
reasoning preceded text:   true

trial 2 HTTP 200
response.output_text.delta: MARKER_ALPHA begin middle end
response.output_text.done:  MARKER_ALPHA begin middle end
reasoning preceded text:   true

trial 3 HTTP 200
response.output_text.delta: MARKER_ALPHA begin middle end
response.output_text.done:  MARKER_ALPHA begin middle end
reasoning preceded text:   true

Type

Bug Fix

Changes

• Replace the one-event chunk translation with ordered event expansion
• Queue all events from each chunk before reading the next chunk
• End the reasoning item before the message item starts
• Allocate canonical output indexes and contiguous emitted sequence numbers
• Include emitted sequence numbers in serialized Pydantic output
• Emit the missing reasoning summary part lifecycle event identified in PR fix(responses): complete chat bridge reasoning lifecycle #32777 by @mxl
• Use a unique reasoning-tool fixture ID to prevent cross-test cache collisions
• Open empty message lifecycles before tool-only completion
• Use the same event expansion in synchronous and asynchronous iteration
• Add synchronous and asynchronous tests for combined and separate chunks
• Add lifecycle coverage for tool-only and reasoning-to-tool streams
• Keep existing content-only streaming behavior under test

Focused verification at original PR commit ecf52e0feee53f010d918b5a01ec2c6f4ad8cfda:

uv run pytest tests/test_litellm/responses/litellm_completion_transformation/test_reasoning_content_transformation.py -q
9 passed

Focused verification at current head 9d109c717ef05ed6bdff8ebb7d6937e35f05ea48:

.venv/bin/pytest tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py -k TestEnsureOutputItemContentPartAdded -q
4 passed

.venv/bin/pytest tests/test_litellm/responses/litellm_completion_transformation/test_reasoning_content_transformation.py -q
11 passed

.venv/bin/pytest tests/test_litellm/responses/litellm_completion_transformation/test_reasoning_content_transformation.py tests/test_litellm/responses/litellm_completion_transformation/test_function_call_output_normalization.py -q
13 passed

.venv/bin/pytest tests/test_litellm/responses/litellm_completion_transformation -q
133 passed

make check
All checks passed

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

❌ Patch coverage is 92.35294% with 13 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...lm_completion_transformation/streaming_iterator.py	92.35%	13 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing weddle:litellm_fix_responses_combined_reasoning_content (9d109c7) with litellm_internal_staging (f6b9518)}

[greptile-apps]

Greptile Summary

The PR expands each chat-completion chunk into ordered Responses API events shared by synchronous and asynchronous iteration.

• Allocates distinct output indexes for reasoning, message, and tool items.
• Emits the reasoning-summary part lifecycle before its text deltas.
• Preserves reasoning and visible text from combined upstream chunks.
• Adds lifecycle and ordering coverage for mixed-content and tool streams.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Introduces shared ordered event expansion, centralized sequence numbering, distinct output-index allocation, and complete reasoning-summary lifecycle handling.
tests/test_litellm/responses/litellm_completion_transformation/test_reasoning_content_transformation.py	Adds sync and async coverage for mixed reasoning/content streams, lifecycle ordering, sequence numbers, and completed-output alignment.
tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py	Adds coverage ensuring reasoning items use the summary-part lifecycle and emit it only once.
tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py	Extends tool-stream lifecycle coverage for the revised event-expansion behavior.

_{Reviews (6): Last reviewed commit: "test(responses): isolate reasoning tool ..." | Re-trigger Greptile}

[weddle]

@greptileai Please re-review the current head for lifecycle ordering, output indexes, and emitted sequence numbers.

[weddle]

@greptileai Please re-review current head. Sequence numbers now serialize through Pydantic output, with mapped sync, async, and tool-stream regressions

[weddle]

@greptileai Please re-review current head, including the reasoning summary part lifecycle event credited to PR #32777

[weddle]

The failing responses-caching-types shard reproduces known global state contamination; both reported normalization tests pass locally. Please rerun that job

[weddle]

@greptileai Please re-review current head, including the reasoning summary part lifecycle event credited to PR #32777

[weddle]

@greptileai Please re-review current head, including the reasoning summary part lifecycle event credited to PR #32777