fix(responses_bridge): map incomplete responses to finish_reason length instead of 500

[mateo-berri]

TLDR

Problem this solves:

• Chat completions through the Responses bridge return 500 when reasoning consumes the whole token budget
• Truncated bridge streams ended with finish_reason "stop" and no usage

How it solves it:

• An incomplete response with no message now yields one empty assistant message
• finish_reason maps from the incomplete reason: "length", or "content_filter" for filtered
• Truncated partial text keeps its content and gets finish_reason "length"
• Streaming handles the response.incomplete terminal event like response.completed, with the mapped finish_reason and usage

User Flow

Before: a developer whose app calls a bridged reasoning model with a small completion budget gets a 500 whenever reasoning uses up the whole budget

1. They send POST https://litellm-domain/v1/chat/completions with "model": "bridge/gpt-5.6-sol" (a openai/responses/... deployment), "reasoning_effort": "high", "max_completion_tokens": 16
2. The response is HTTP 500 with "litellm.APIConnectionError: ... gpt-5.6-sol unable to complete request: max_output_tokens", so their app surfaces a gateway error to the end user
3. The same request with "stream": true returns 200, but the only meaningful chunk carries "finish_reason": "stop" and no usage, so the app believes the model finished normally and records zero spend
4. POST https://litellm-domain/v1/responses with the same model and "max_output_tokens": 16 returns 200 with "status": "incomplete", so only the chat completions surface is broken

After: the same request behaves like OpenAI's native chat completions on truncation

1. They send the same POST https://litellm-domain/v1/chat/completions with "reasoning_effort": "high", "max_completion_tokens": 16
2. The response is HTTP 200 with one choice: an assistant message with "content": "", "finish_reason": "length", and usage showing prompt 37 / completion 16 / reasoning 16 tokens
3. The same request with "stream": true returns a terminal chunk with "finish_reason": "length", and with "stream_options": {"include_usage": true} the final chunk carries the real usage
4. When the truncation reason is a content filter, the choice comes back with "finish_reason": "content_filter" instead

Relevant issues

Linear ticket

Resolves LIT-5886

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Two proxies, same config, same prompt, real OpenAI calls with no mocks. Before is the merge base cb4eb82, after is 5c89490 on this branch. All four cases were re-run at the current tip 16bba15 and came back identical, so the proof stands at the head, not just at the commit it was first captured on

Shared config:

model_list:
  - model_name: bridge/gpt-5.6-sol
    litellm_params:
      model: openai/responses/gpt-5.6-sol
      api_key: os.environ/OPENAI_API_KEY
general_settings:
  master_key: sk-1234

Prompt used in every case: "Determine the last ten decimal digits of 7 raised to the power 123456789. Think through modular arithmetic carefully, then return only those ten digits." Every request caps output at 16 tokens, which is what makes the upstream response come back incomplete

Before (cb4eb82, port 52361)

Non-streaming /v1/chat/completions

1. curl -sS -i http://localhost:52361/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"bridge/gpt-5.6-sol","messages":[{"role":"user","content":"<prompt>"}],"reasoning_effort":"high","max_completion_tokens":16}'
2. Observed HTTP/1.1 500 Internal Server Error with body {"error":{"message":"litellm.APIConnectionError: APIConnectionError: OpenAIException - gpt-5.6-sol unable to complete request: max_output_tokens. ...","code":"500"}}, no usage anywhere, and x-litellm-response-cost: 0
3. The proxy log carries ValueError: gpt-5.6-sol unable to complete request: max_output_tokens, raised at transformation.py:768

Streaming /v1/chat/completions

1. Same request with "stream": true and "stream_options":{"include_usage":true}
2. Observed HTTP 200 whose whole stream is {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}, then a second empty-delta chunk, then [DONE]
3. No usage chunk arrives even though the request asked for one, and every cost header reads 0.0

Non-streaming /v1/messages

1. curl -sS -i http://localhost:52361/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"bridge/gpt-5.6-sol","max_tokens":16,"messages":[{"role":"user","content":"<prompt>"}]}'
2. Observed HTTP 200 with "content":[],"stop_reason":"max_tokens" and "usage":{"input_tokens":37,"output_tokens":16}, billed at x-litellm-response-cost: 0.000665

Streaming /v1/messages

1. Same request with "stream": true
2. Observed message_start, then message_delta carrying {"stop_reason": "max_tokens"} with usage 37 in / 16 out, then message_stop

After (5c89490, port 35416)

Non-streaming /v1/chat/completions

1. Same curl, against the fixed proxy
2. Observed HTTP/1.1 200 OK with "choices":[{"finish_reason":"length","index":0,"message":{"content":"","role":"assistant","reasoning_items":[{"type":"reasoning","id":"rs_0785b87a...","encrypted_content":"gAAAAABqh4bd...","summary":[]}]}}] and "usage":{"completion_tokens":16,"prompt_tokens":37,"total_tokens":53,"completion_tokens_details":{"reasoning_tokens":16},...}
3. The call is billed at x-litellm-response-cost: 0.000665, where the 500 billed nothing

Streaming /v1/chat/completions

1. Same request with "stream": true and "stream_options":{"include_usage":true}
2. Observed the terminal choice chunk {"choices":[{"index":0,"delta":{},"finish_reason":"length"}]}
3. The final chunk carries the real usage {"completion_tokens":16,"prompt_tokens":37,"total_tokens":53,"completion_tokens_details":{"reasoning_tokens":16},...}, then [DONE]

Non-streaming /v1/messages

1. Same curl as the before leg
2. Observed HTTP 200 with "content":[],"stop_reason":"max_tokens", usage 37 in / 16 out, cost 0.000665, unchanged from before

Streaming /v1/messages

1. Same request with "stream": true
2. Observed the same message_start / message_delta with "stop_reason": "max_tokens" and usage 37 / 16 / message_stop, unchanged

The Anthropic shape already reported the truncation honestly at the merge base, so the fix lands only on the OpenAI chat completions shape and /v1/messages stays where it was

Re-run at the tip (16bba15, port 55643)

The same four cases, driven again against a proxy booted from the current head with PYTHONPATH pinned to that worktree and the loaded litellm.__file__ asserted to sit inside it:

1. Non-streaming /v1/chat/completions: HTTP/1.1 200 OK, "finish_reason":"length", reasoning_items present, "usage":{"completion_tokens":16,"prompt_tokens":37,"total_tokens":53,"completion_tokens_details":{"reasoning_tokens":16}}, billed x-litellm-response-cost: 0.000665
2. Streaming /v1/chat/completions: terminal chunk {"choices":[{"index":0,"delta":{},"finish_reason":"length"}]} followed by the real usage chunk and [DONE]
3. Non-streaming /v1/messages: HTTP 200 with "content":[],"stop_reason":"max_tokens", usage 37 in / 16 out, cost 0.000665
4. Streaming /v1/messages: message_start, message_delta carrying {"stop_reason": "max_tokens"} with usage 37 / 16, message_stop

Surprises

• Truncated calls now bill; the 500 billed nothing (causes)
• Streamed chunks omit the reasoning items non-streaming returns (leaves alone)
• Cost headers read zero on every streamed call (leaves alone)
• /v1/messages already reported max_tokens before the fix (leaves alone)
• message_start echoes responses/gpt-5.6-sol, not the alias (leaves alone)

Type

🐛 Bug Fix

Caveats (if any)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes how truncated/incomplete Responses API results are mapped into chat completions, including finish_reason and usage/billing. Wrong mapping would mis-signal truncation or still 500, but the change is isolated to the bridge transformer.

Overview
Chat completions through the Responses bridge no longer 500 when a reasoning model burns the whole token budget. Incomplete Responses API results now look like native OpenAI truncation: HTTP 200, finish_reason length (or content_filter), and real usage.

If there is no message (reasoning-only output), the bridge emits one empty assistant choice and still round-trips reasoning_items. Partial text is kept and only the finish reason is overridden. Streaming treats response.incomplete like response.completed, so the terminal chunk carries the mapped finish reason and usage instead of stop with no spend. Unknown empty completed responses still raise.

^{Reviewed by Cursor Bugbot for commit 16bba15. Bugbot is set up for automated code reviews on this repo. Configure here.}

• 16bba15 passes /live-pr-risk

[greptile-apps]

Greptile Summary

The PR converts incomplete Responses bridge results into valid chat completions instead of errors and preserves truncation metadata and usage.

• Maps incomplete reasons to length or content_filter for non-streaming responses.
• Emits terminal finish reasons, reasoning items, and usage for incomplete streams.
• Adds typed regression-test helpers and coverage for empty, partial-text, filtered, and streamed incomplete responses.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/completion_extras/litellm_responses_transformation/transformation.py	Adds incomplete-response finish-reason mapping and terminal handling for both non-streaming and streaming bridge responses.
tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py	Adds typed helpers and focused regression coverage; the previously reported missing annotations are now present.

_{Reviews (4): Last reviewed commit: "require an incomplete reason before over..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 97.72727% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...litellm_responses_transformation/transformation.py	97.72%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_responses_bridge_incomplete_500 (16bba15) with litellm_internal_staging (e07a712)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (02e67cd) during the generation of this report, so e07a712 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 7b25ee1. Configure here.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 16bba15. Configure here.}

[tin-berri]

Solid fix — converts the 500-on-truncated-reasoning bug into a proper finish_reason (length/content_filter) for both streaming and non-streaming paths, and cleans up duplicated reasoning-item-extraction logic into one shared helper used by both. The response_is_incomplete check (status=="incomplete" OR incomplete_details.reason set) correctly preserves the old raise-on-genuinely-empty-and-not-incomplete behavior — good regression guard in test_transform_response_zero_choices_not_incomplete_still_raises. Boundary case (completed status + null-reason incomplete_details staying "stop") is explicitly tested too. Good coverage across reasoning-only, content-filter, partial-text-override, and all three streaming response.incomplete event shapes. CI green. Approved.