fix(responses): mint Responses API item IDs in the completion bridge by mateo-berri · Pull Request #37946 · BerriAI/litellm

mateo-berri · 2026-08-22T17:40:45Z

TLDR

Problem this solves:

Bridged /v1/responses output items carry chatcmpl-* IDs
Native OpenAI rejects replayed history, expecting an msg prefix
Streaming events say msg_*, the final snapshot says chatcmpl-*
Image generation items use chatcmpl-*_img_N instead of ig_*
Reasoning IDs come from a salted hash, unstable across processes
Streaming reasoning deltas mint a fresh item ID on every chunk

How it solves it:

Mint msg_* IDs for bridged message output items
Mint ig_* for image generation calls, rs_* for reasoning
Reuse the streamed item IDs in the response.completed snapshot
Stop seeding the streamed item ID from the upstream chunk ID
Keep one reasoning item ID for the whole streamed response

User Flow

Before: a developer moving a conversation from a bridged provider to native OpenAI gets a 400 the moment they send the assistant turn back, so the handoff is impossible.

They send POST https://litellm-domain/v1/responses with "model": "claude-bridged" and one user message, "Say the single word: apple"
They get 200 back with "status": "completed", and the assistant output item carries "id": "chatcmpl-fb0c33ee-62e3-4db8-9a25-2d7c7cbd44ae"
They continue the same conversation on OpenAI: POST https://litellm-domain/v1/responses with "model": "gpt-native", replaying the user turn, the assistant item exactly as received, and a new user message, "Now say the single word: banana"
They get 400 Invalid 'input[1].id': 'chatcmpl-fb0c33ee-62e3-4db8-9a25-2d7c7cbd44ae'. Expected an ID that begins with 'msg'., so the turn never reaches the model
They retry step 1 with "stream": true, and watch the streaming events hand back "item_id": "msg_a3d55c84-96f5-4757-8770-b77092e6c85a"
The final response.completed event contradicts those events, carrying "id": "chatcmpl-5762b1bc-c73d-43bc-b3fe-0faecc35696f" on the same assistant item
Replaying that completed snapshot at step 3 fails with the same 400, so streaming clients are stuck too
Their only workaround is to strip the ID off every assistant item before replaying it

After: the same handoff goes through, and the streaming events and the final snapshot agree on one ID.

They send POST https://litellm-domain/v1/responses with "model": "claude-bridged" and one user message, "Say the single word: apple"
They get 200 back with "status": "completed", and the assistant output item carries "id": "msg_a8802041-8bc4-4f50-b132-4174136a877b"
They continue the same conversation on OpenAI: POST https://litellm-domain/v1/responses with "model": "gpt-native", replaying the user turn, the assistant item exactly as received, and a new user message, "Now say the single word: banana"
They get 200 back with "status": "completed" and the model answers "banana"
They retry step 1 with "stream": true, and watch the streaming events hand back "item_id": "msg_a6028496-1fc5-4bb8-bc9e-52c1e26c959a"
The final response.completed event carries that same "id": "msg_a6028496-1fc5-4bb8-bc9e-52c1e26c959a" on the assistant item
Replaying that completed snapshot at step 3 returns 200, so streaming clients hand off too
No workaround is needed; assistant items replay exactly as received

Relevant issues

Fixes #27333

Based on #27426, which was auto-closed unmerged. This picks up its msg_* minting and extends it to the response.completed snapshot so streaming and non-streaming agree. Note that #27426 minted img_* for image generation calls; OpenAI rejects that and expects ig_*, which is what this PR mints.

Linear ticket

Resolves LIT-6005

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup. One proxy at the merge base and one at this branch, same config, both hitting real Anthropic and real OpenAI with no mocks and real spend:

model_list:
  - model_name: claude-bridged
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: gpt-native
    litellm_params:
      model: openai/gpt-5-mini
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: sk-issue27333

litellm_settings:
  drop_params: true

The reporter's flow, through the OpenAI Agents SDK

The issue describes an agent handing off to an OpenAI Responses agent inside one runner execution, so the proof runs that, not a curl approximation. Two agents against the same proxy, the second one fed result.to_input_list() from the first:

claude_agent = Agent(name="claude-leg", instructions="Answer with a single lowercase word and nothing else.",
                     model=OpenAIResponsesModel(model="claude-bridged", openai_client=CLIENT))
openai_agent = Agent(name="openai-leg", instructions="Answer with a single lowercase word and nothing else.",
                     model=OpenAIResponsesModel(model="gpt-native", openai_client=CLIENT))

first = await Runner.run(claude_agent, "Say the single word: apple")
history = first.to_input_list()
history.append({"role": "user", "content": "Now say the single word: banana"})
second = await Runner.run(openai_agent, history)

At the merge base the handoff dies on the assistant item the first agent produced:

$ python agent_handoff.py    # proxy booted from 7a1afa1c40
step 1 (claude-bridged) reply: apple
step 1 history handed to the next agent:
[
  {"content": "Say the single word: apple", "role": "user"},
  {"id": "chatcmpl-9c728174-d81c-40da-994b-3ddbfc80280a",
   "content": [{"annotations": [], "text": "apple", "type": "output_text"}],
   "role": "assistant", "status": "completed", "type": "message"}
]
step 2 (gpt-native) FAILED
BadRequestError: Error code: 400 - Invalid 'input[1].id': 'chatcmpl-9c728174-d81c-40da-994b-3ddbfc80280a'. Expected an ID that begins with 'msg'.

At this branch the same script runs straight through:

$ python agent_handoff.py    # proxy booted from 6a55683cd0
step 1 (claude-bridged) reply: apple
step 1 history handed to the next agent:
[
  {"content": "Say the single word: apple", "role": "user"},
  {"id": "msg_5a3b7032-71da-4228-a33c-c77e35fdc59a",
   "content": [{"annotations": [], "text": "apple", "type": "output_text"}],
   "role": "assistant", "status": "completed", "type": "message"}
]
step 2 (gpt-native) reply: banana

The same script also passes against this branch merged into the target branch, so the fix survives the merge.

The rest of this section is the same handoff at the HTTP level, plus the streaming case.

Before (`7a1afa1`)

Cross-provider replay, non-streaming

Send a bridged turn to Anthropic and read the ID on the assistant output item:

$ curl -sS -o resp.json -w '%{http_code}\n' http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"claude-bridged","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]}]}'
200

$ jq -c '{status, output: [.output[] | {type, id}]}' resp.json
{"status":"completed","output":[{"type":"message","id":"chatcmpl-fb0c33ee-62e3-4db8-9a25-2d7c7cbd44ae"}]}

Replay that assistant item into native OpenAI:

$ curl -sS -o resp.json -w '%{http_code}\n' http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"gpt-native","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]},{"type":"message","id":"chatcmpl-fb0c33ee-62e3-4db8-9a25-2d7c7cbd44ae","status":"completed","role":"assistant","content":[{"type":"output_text","text":"apple","annotations":[]}]},{"type":"message","role":"user","content":[{"type":"input_text","text":"Now say the single word: banana"}]}]}'
400

$ jq -r '.error.message' resp.json | head -4
litellm.BadRequestError: OpenAIException - {
  "error": {
    "message": "Invalid 'input[1].id': 'chatcmpl-fb0c33ee-62e3-4db8-9a25-2d7c7cbd44ae'. Expected an ID that begins with 'msg'.",

Cross-provider replay, streaming

Send the same bridged turn with "stream": true and compare the incremental events against the final snapshot:

$ curl -sSN http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"claude-bridged","stream":true,"input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]}]}' > stream.txt

$ grep '^data: ' stream.txt | sed 's/^data: //' | grep -v '^\[DONE\]$' > events.jsonl

$ jq -r 'select(.item_id != null) | .item_id' events.jsonl | sort -u
msg_a3d55c84-96f5-4757-8770-b77092e6c85a

$ jq -r 'select(.type=="response.completed") | .response.output[] | select(.type=="message") | .id' events.jsonl
chatcmpl-5762b1bc-c73d-43bc-b3fe-0faecc35696f

Replay the response.completed item into native OpenAI:

$ curl -sS -o resp.json -w '%{http_code}\n' http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"gpt-native","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]},{"type":"message","id":"chatcmpl-5762b1bc-c73d-43bc-b3fe-0faecc35696f","status":"completed","role":"assistant","content":[{"type":"output_text","text":"apple","annotations":[]}]},{"type":"message","role":"user","content":[{"type":"input_text","text":"Now say the single word: banana"}]}]}'
400

$ jq -r '.error.message' resp.json | head -4
litellm.BadRequestError: OpenAIException - {
  "error": {
    "message": "Invalid 'input[1].id': 'chatcmpl-5762b1bc-c73d-43bc-b3fe-0faecc35696f'. Expected an ID that begins with 'msg'.",

After (`6a55683`)

Cross-provider replay, non-streaming

Send a bridged turn to Anthropic and read the ID on the assistant output item:

$ curl -sS -o resp.json -w '%{http_code}\n' http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"claude-bridged","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]}]}'
200

$ jq -c '{status, output: [.output[] | {type, id}]}' resp.json
{"status":"completed","output":[{"type":"message","id":"msg_a8802041-8bc4-4f50-b132-4174136a877b"}]}

Replay that assistant item into native OpenAI:

$ curl -sS -o resp.json -w '%{http_code}\n' http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"gpt-native","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]},{"type":"message","id":"msg_a8802041-8bc4-4f50-b132-4174136a877b","status":"completed","role":"assistant","content":[{"type":"output_text","text":"apple","annotations":[]}]},{"type":"message","role":"user","content":[{"type":"input_text","text":"Now say the single word: banana"}]}]}'
200

$ jq -r '"status: " + .status + "\nreply: " + ([.output[] | select(.type=="message") | .content[0].text] | join(" "))' resp.json
status: completed
reply: banana

Cross-provider replay, streaming

Send the same bridged turn with "stream": true and compare the incremental events against the final snapshot:

$ curl -sSN http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"claude-bridged","stream":true,"input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]}]}' > stream.txt

$ grep '^data: ' stream.txt | sed 's/^data: //' | grep -v '^\[DONE\]$' > events.jsonl

$ jq -r 'select(.item_id != null) | .item_id' events.jsonl | sort -u
msg_a6028496-1fc5-4bb8-bc9e-52c1e26c959a

$ jq -r 'select(.type=="response.completed") | .response.output[] | select(.type=="message") | .id' events.jsonl
msg_a6028496-1fc5-4bb8-bc9e-52c1e26c959a

Replay the response.completed item into native OpenAI:

$ curl -sS -o resp.json -w '%{http_code}\n' http://localhost:4000/v1/responses \
    -H 'Authorization: Bearer sk-issue27333' -H 'Content-Type: application/json' \
    -d '{"model":"gpt-native","input":[{"type":"message","role":"user","content":[{"type":"input_text","text":"Say the single word: apple"}]},{"type":"message","id":"msg_a6028496-1fc5-4bb8-bc9e-52c1e26c959a","status":"completed","role":"assistant","content":[{"type":"output_text","text":"apple","annotations":[]}]},{"type":"message","role":"user","content":[{"type":"input_text","text":"Now say the single word: banana"}]}]}'
200

$ jq -r '"status: " + .status + "\nreply: " + ([.output[] | select(.type=="message") | .content[0].text] | join(" "))' resp.json
status: completed
reply: banana

Prefix expectations were confirmed straight against https://api.openai.com/v1/responses, outside LiteLLM: a message item accepts msg_ in both dashed-UUID and hex form, an image generation call is rejected with "Expected an ID that begins with 'ig'" when sent img_*, and accepted on prefix when sent ig_*.

Type

🐛 Bug Fix

Caveats (if any)

Item IDs are now random per response, not derived upstream
History captured before this change still carries chatcmpl-*
Top-level response ID unchanged; only output item IDs move

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
6a55683 passes /live-pr-risk

Note

Medium Risk
Changes public output-item IDs that clients persist and replay. Existing stored chatcmpl-* history still fails OpenAI prefix checks; new IDs are random per response rather than derived from upstream.

Overview
Bridged /v1/responses output items now mint OpenAI-valid IDs (msg_*, rs_*, ig_*) instead of leaking upstream chatcmpl-* IDs, so replayed history is accepted by native OpenAI.

Streaming no longer seeds the message item ID from the chat-completion chunk ID. Reasoning deltas share one cached rs_* ID instead of hashing each chunk. The response.completed snapshot copies those streamed IDs onto the matching output items so incremental events and the final snapshot agree.

Image generation calls use unique ig_* UUIDs (not chatcmpl-*_img_N). Top-level response IDs are unchanged.

^{Reviewed by Cursor Bugbot for commit 6a55683. Bugbot is set up for automated code reviews on this repo. Configure here.}

The Chat Completions -> Responses bridge stamped the upstream chatcmpl-* ID onto message output items, so replaying bridged history into native OpenAI Responses failed with "Expected an ID that begins with 'msg'". Image generation calls were minted as chatcmpl-*_img_N instead of ig_*, and reasoning items used a salted hash() that is not stable across processes. Streaming minted msg_* for its incremental events but rebuilt the response.completed snapshot through the same broken transform, so the snapshot contradicted the events it had just sent and streaming clients hit the same 400. The snapshot now reuses the IDs already streamed. Fixes #27333

codecov · 2026-08-22T17:44:55Z

Codecov Report

❌ Patch coverage is 85.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...lm_completion_transformation/streaming_iterator.py	83.33%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

greptile-apps · 2026-08-22T17:45:02Z

Greptile Summary

The PR mints Responses API-compatible IDs for bridged message, reasoning, and image-generation items while keeping streamed events consistent with the completed snapshot

Replaces chat-completion-derived item IDs with type-specific UUID-based IDs
Reuses message and reasoning IDs across streaming deltas and final output
Adds regression coverage for prefixes, uniqueness, replay compatibility, and streaming consistency

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Caches Responses-compatible item IDs and immutably aligns the completed snapshot with streamed message and reasoning events
litellm/responses/litellm_completion_transformation/transformation.py	Generates UUID-based msg_, rs_, and ig_ identifiers for bridged output items
tests/test_litellm/responses/litellm_completion_transformation/test_image_generation_output.py	Verifies image-generation item identifiers use the required prefix and remain unique
tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py	Covers bridged item prefixes and consistency between incremental streaming events and completed snapshots

_{Reviews (4): Last reviewed commit: "refactor: drop the unused response argum..." | Re-trigger Greptile}

… mutating items Align the response.completed item IDs by copying each output item rather than writing to it in place, and move the regression cases into the existing completion-response and image-generation test modules.

mateo-berri · 2026-08-22T18:17:10Z

@greptileai

Write the fallback reasoning item id back to the cache so the reasoning-done path and the completed snapshot cannot drift apart, and cover the shared delta id and the snapshot alignment with tests.

mateo-berri · 2026-08-22T18:43:13Z

@greptileai

codspeed-hq · 2026-08-22T19:25:09Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_fix_27333_bridge_item_ids (6a55683) with litellm_internal_staging (490c9f9)¹}

No successful run was found on litellm_internal_staging (9349b22) during the generation of this report, so 9036da5 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

…ctor The image generation item ID no longer comes from the chat completion response, so the extractor does not need it.

mateo-berri · 2026-08-22T21:22:22Z

@greptileai

mateo-berri · 2026-08-22T21:26:29Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6a55683. Configure here.}

greptile-apps Bot reviewed Aug 22, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/streaming_iterator.py Outdated

fix: keep one reasoning item id across a bridged stream

6d2b7db

Write the fallback reasoning item id back to the cache so the reasoning-done path and the completed snapshot cannot drift apart, and cover the shared delta id and the snapshot alignment with tests.

refactor: drop the unused response argument from the image item extra…

6a55683

…ctor The image generation item ID no longer comes from the chat completion response, so the extractor does not need it.

cursor Bot reviewed Aug 22, 2026

View reviewed changes

mateo-berri enabled auto-merge August 22, 2026 21:32

yucheng-berri approved these changes Aug 22, 2026

View reviewed changes

mateo-berri merged commit 98dfb78 into litellm_internal_staging Aug 22, 2026
72 checks passed

mateo-berri deleted the litellm_fix_27333_bridge_item_ids branch August 22, 2026 21:46

github-actions Bot mentioned this pull request Aug 22, 2026

🦞 OpenClaw 生态日报 2026-08-23 junlinzhao327-oss/big_model_radar#418

Open

Navigation Menu

Uh oh!

Uh oh!

fix(responses): mint Responses API item IDs in the completion bridge - #37946

fix(responses): mint Responses API item IDs in the completion bridge#37946
mateo-berri merged 4 commits into
litellm_internal_stagingfrom
litellm_fix_27333_bridge_item_ids

mateo-berri commented Aug 22, 2026 •

edited by cursor Bot

Loading

Uh oh!

codecov Bot commented Aug 22, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Aug 22, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

codspeed-hq Bot commented Aug 22, 2026 •

edited

Loading

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

mateo-berri commented Aug 22, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

The reporter's flow, through the OpenAI Agents SDK

Before (7a1afa1)

Cross-provider replay, non-streaming

Cross-provider replay, streaming

After (6a55683)

Cross-provider replay, non-streaming

Cross-provider replay, streaming

Type

Caveats (if any)

Final Attestation

Uh oh!

codecov Bot commented Aug 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented Aug 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

codspeed-hq Bot commented Aug 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

mateo-berri commented Aug 22, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mateo-berri commented Aug 22, 2026 •

edited by cursor Bot

Loading

Before (`7a1afa1`)

After (`6a55683`)

codecov Bot commented Aug 22, 2026 •

edited

Loading

greptile-apps Bot commented Aug 22, 2026 •

edited

Loading

codspeed-hq Bot commented Aug 22, 2026 •

edited

Loading