fix(responses): recover empty completed output from stream events

[jourdant]

Replaces #32718, which was closed after it was initially opened against litellm_internal_staging and received review feedback for unrelated files. This replacement targets litellm_oss_staging directly and contains only the intended Responses streaming fix.

Relevant issues

Fixes #25429

Related to #26179 and #26309

Related implementation: #31332

Linear ticket

N/A

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

ChatGPT subscription Responses streams can return assistant content through response.output_item.done and response.output_text.done, followed by a terminal response.completed event whose response.output is empty

Before this change, a non-streaming Chat Completions request reached the terminal event with output=[] and failed during the Responses-to-Chat-Completions conversion:

HTTP 500

litellm.APIConnectionError: APIConnectionError: ChatgptException -
Unknown items in responses API response: []

After this change, the same provider and proxy route return a normal Chat Completions response:

curl http://127.0.0.1:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  --data '{
    "model": "oai-gpt-5.6-sol",
    "messages": [
      {
        "role": "system",
        "content": "Reply with exactly ok."
      },
      {
        "role": "user",
        "content": "Do it."
      }
    ],
    "max_tokens": 16
  }'

Sanitized result:

{
  "http_status": 200,
  "id_present": true,
  "model": "oai-gpt-5.6-sol",
  "content": "ok",
  "error": null
}

Type

🐛 Bug Fix

Changes

BaseResponsesAPIStreamingIterator now retains completed output items while processing the stream and uses them to populate an empty response.completed.output

Recovery uses transformed stream events rather than raw provider items so existing ID, container, and encrypted-content normalization is preserved

The recovery path supports both output-item completion events and providers that emit only output-text completion events. Existing non-empty terminal output remains authoritative

Raw and transformed indexes must both be non-negative integers. Missing, negative, boolean, string, floating-point, or Pydantic-coerced indexes are rejected so malformed events cannot overwrite valid recovered items

Failed and incomplete terminal responses are not rewritten, and genuinely unsupported output item types continue through the existing diagnostic path

Regression coverage was added to tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py for empty completed output recovery, text-only recovery, transformed event IDs, dictionary and Pydantic terminal responses, deterministic output-index ordering, authoritative terminal output, terminal failure behavior, and malformed indexes

Local validation:

18 passed
make pre-commit: passed
Ruff strict delta gate: passed
Type-discipline delta gate: passed
Basedpyright delta gate: passed
Patch coverage: 100% (28 changed production lines, 0 missing)

[greptile-apps]

Greptile Summary

This PR fixes a 500 error that occurred when a ChatGPT-subscription provider emitted content through response.output_item.done / response.output_text.done events but delivered an empty output list in the terminal response.completed event, causing the Responses-to-Chat-Completions conversion to fail with "Unknown items in responses API response: []".

• BaseResponsesAPIStreamingIterator now accumulates completed output items (_streamed_output_items) and text-only synthetic items (_streamed_text_only_items) while processing the stream, then backfills an empty response.completed.output from those accumulators before setting self.completed_response.
• The new litellm/responses/sse_output_recovery.py module centralizes record_output_item_chunk and record_output_text_chunk helpers; both defend against malformed indexes (non-int, negative, missing) so that invalid events cannot corrupt recovered state.
• Recovery is gated exclusively on RESPONSE_COMPLETED with a non-empty recovered_output; RESPONSE_INCOMPLETE and RESPONSE_FAILED terminal events are intentionally left unmodified.

Confidence Score: 5/5

Safe to merge — the recovery path is narrowly scoped to RESPONSE_COMPLETED events with empty output, leaving all other terminal states and non-empty authoritative output untouched.

The change is well-bounded: it only mutates the response object when the terminal event is RESPONSE_COMPLETED and carries an empty output list, and that only happens after all streamed items have been accumulated. Index validation (type-exact int, non-negative, raw-vs-transformed match) prevents malformed events from injecting garbage. The new tests cover every meaningful branch: item-done recovery, text-only recovery, dict terminal response, deterministic sort order, malformed-index rejection, and the two non-backfill guards for INCOMPLETE and FAILED. No existing assertions were weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/responses/streaming_iterator.py	Adds per-stream accumulators and a recovery path that backfills empty RESPONSE_COMPLETED output from streamed OUTPUT_ITEM_DONE / OUTPUT_TEXT_DONE events; only triggers for RESPONSE_COMPLETED, correctly leaving INCOMPLETE/FAILED unmodified.
litellm/responses/sse_output_recovery.py	New shared helpers for recording OUTPUT_ITEM_DONE and OUTPUT_TEXT_DONE chunks into recovery dicts; logic is sound with appropriate guards for missing/negative/non-int indexes.
tests/llm_responses_api_testing/test_base_responses_api_streaming_iterator.py	Adds 7 new focused tests covering output-item recovery, text-only recovery, authoritative terminal output, dict terminal response, index ordering, malformed-index rejection, and non-backfill for INCOMPLETE/FAILED; existing test modifications are purely cosmetic formatting changes.

_{Reviews (2): Last reviewed commit: "revert(responses): keep cost tracking un..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 35.71429% with 18 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/responses/streaming_iterator.py	35.71%	18 Missing ⚠️

📢 Thoughts on this report? Let us know!

[jourdant]

@greptileai The typed-event test fix is pushed. Please re-review the latest commit. The outside-diff cost-tracking rename was intentionally left unchanged to keep this PR isolated.

[steveonjava]

Hit this one for real on our end. ChatGPT's Responses API (gpt-5.6-sol/terra/luna over Codex OAuth) sends response.completed.output: [] even after the content already streamed through via output_item.done/output_text.done, and every completion 500s with Unknown items in responses API response: [] until this backfill runs. Ported the diff onto our vendored 1.95.0 and ran it against responses() directly, the completion() bridge, and the full proxy over HTTP. Nothing regressed on models that already worked. Would be great to see this merged.

[jourdant]

@steveonjava I'm glad to see it's validated working for you! I've been struggling to get visibility on my PRs (this one included). I do need to sort the code coverage etc