fix(chatgpt): accumulate response.output_item.done for non-streaming responses

[andafterall]

Summary

Fix empty output: [] returned by the ChatGPT subscription provider (chatgpt/* models) on /v1/responses and /v1/chat/completions (which bridges through Responses).

The bug

Calls succeed (status: completed, usage reports correct token counts), but response.output is always empty:

curl -s -X POST http://127.0.0.1:4000/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{"model":"chatgpt/gpt-5.4","input":[{"role":"user","content":"hi"}]}'
# { "status": "completed", "usage": {"input_tokens":17,"output_tokens":6,...}, "output": [] }

/v1/chat/completions fails with:

litellm.APIConnectionError: ChatgptException - Unknown items in responses API response: [].

Root cause

litellm/llms/chatgpt/responses/transformation.py::transform_response_api_response reads the terminal response.completed SSE event and builds ResponsesAPIResponse from its response field. This works for the standard OpenAI Responses API where that event carries the fully assembled output.

The ChatGPT Codex backend (chatgpt.com/backend-api/codex/responses) behaves differently: it streams output items via response.output_item.done events and emits response.completed with an empty response.output — the terminal event only carries metadata (id/status/usage). The current code ignores the output_item.done events, so nothing makes it into the assembled non-streaming response.

The upstream Codex CLI client handles this by accumulating items while iterating the stream — see codex-rs/core/src/client.rs's map_response_stream, which maintains items_added: Vec<ResponseItem> and pushes each OutputItemDone item into it before shipping the terminal Completed event.

Fix

• Accumulate items from response.output_item.done events into a list while parsing the SSE body.
• When response.completed arrives and its response.output is empty/missing, fall back to the accumulator.
• When response.completed already carries a populated output (standard OpenAI behavior), it wins — backward compatible.
• Minor refactor: extract response-object construction into a small _build_completed_response static helper, keeping the main loop readable.

Tests

Two new cases in tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py:

• test_chatgpt_accumulates_output_item_done_when_completed_output_empty — reproduces the ChatGPT Codex backend SSE sequence (reasoning item + message item via output_item.done, terminal response.completed with empty output) and asserts all items are recovered and output_text reflects the message.
• test_chatgpt_prefers_nonempty_completed_output_over_accumulated — ensures standard OpenAI Responses behavior is untouched when the terminal event does carry output.

All 27 tests in tests/test_litellm/llms/chatgpt/ pass locally. Ruff + black clean on the changed files.

Manual verification

Patched ChatGPTResponsesAPIConfig.transform_response_api_response via monkey-patch against a live local proxy (1.83.10) authenticated with a real ChatGPT Plus subscription:

Endpoint	Model	Before	After
/v1/responses	chatgpt/gpt-5.4	output: []	"hello world"
/v1/responses	chatgpt/gpt-5.4-mini	output: []	text + reasoning item
/v1/responses	chatgpt/gpt-5.3-codex-spark	output: []	text + reasoning item
/v1/chat/completions	chatgpt/gpt-5.4-mini	500 Unknown items in responses API response: []	"chat ok"

Test plan

• Unit tests pass (pytest tests/test_litellm/llms/chatgpt/)
• ruff check clean on changed files
• black --check clean on changed files
• Manual repro against live ChatGPT Plus subscription on three models and both endpoints
• CLA signed

[greptile-apps]

Greptile Summary

This PR fixes an empty output: [] bug in the ChatGPT Codex backend (chatgpt/* provider) for both /v1/responses and /v1/chat/completions. The Codex backend streams content via response.output_item.done SSE events and sends a terminal response.completed with an empty output, unlike the standard OpenAI Responses API which populates output in the terminal event. The fix accumulates items from output_item.done events and uses them as a fallback when the terminal response.completed carries an empty/missing output, matching the approach in the upstream Codex CLI reference implementation.

Confidence Score: 5/5

Safe to merge — focused bug fix, fully backward-compatible, and no P0/P1 findings.

The change is narrow and well-contained: it adds an accumulator list local to each transform_response_api_response call and a small static helper. The fallback logic (not output and accumulated) correctly handles all empty/missing output cases while leaving the standard OpenAI path untouched. Both new tests are mock-only and cover the two critical branches. No existing tests were modified.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/chatgpt/responses/transformation.py	Adds output_item.done accumulation with clean extraction into a static _build_completed_response helper; fallback logic is correct and backward-compatible.
tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py	Two new mock-only unit tests added: one for the ChatGPT Codex accumulation path, one to verify existing OpenAI behavior is unchanged; no real network calls.

Sequence Diagram

sequenceDiagram
    participant C as LiteLLM Client
    participant T as ChatGPTResponsesAPIConfig
    participant B as ChatGPT Codex Backend

    C->>B: POST /responses (stream=true)
    B-->>T: SSE: response.output_item.done (reasoning item)
    Note over T: accumulated_output_items.append(reasoning_item)
    B-->>T: SSE: response.output_item.done (message item)
    Note over T: accumulated_output_items.append(message_item)
    B-->>T: SSE: response.completed (output=[])
    Note over T: _build_completed_response():
  output=[] is falsy → use accumulator
    T-->>C: ResponsesAPIResponse(output=[reasoning, message])

    Note over C,B: Standard OpenAI path (unchanged)
    B-->>T: SSE: response.completed (output=[item1, item2])
    Note over T: output is truthy → use as-is, ignore accumulator
    T-->>C: ResponsesAPIResponse(output=[item1, item2])

Loading

_{Reviews (1): Last reviewed commit: "fix(chatgpt): accumulate response.output..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 88.88889% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/chatgpt/responses/transformation.py	88.88%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.