Skip to content

fix(litellm): preserve Responses streaming errors - #40

Closed
zoroyihan7 wants to merge 1 commit into
mainfrom
fix/litellm-responses-error-events
Closed

zoroyihan7 wants to merge 1 commit into
mainfrom
fix/litellm-responses-error-events

Conversation

@zoroyihan7

@zoroyihan7 zoroyihan7 commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator

An upstream exception after a native /v1/responses stream starts was emitted as an untyped {"error": ...} frame. Responses clients could discard the error and report a missing completion event. Emit response.failed with the upstream error details, the client-visible response ID, and an increasing sequence number. Keep Chat Completions and Cursor conversion formats, as well as retry and fallback policies, unchanged.

Apply the patch to the existing pinned LiteLLM 1.99.0 image only after checking upstream source hashes and compiling the patched files. Include the patch files in both Docker and Kaniko build contexts, and allow an existing push credential Secret to select the Kaniko backend.

Validation:

  • 14 unit tests, including response identity, terminal-event timing, and rejected source versions.
  • 13 HTTP integration cases through the built LiteLLM image and a synthetic upstream, covering Responses failures, text and tool output, Chat Completions, and Cursor conversion.
  • Codex CLI 0.153.2 against the synthetic proxy: rate limits and server errors retain their cause, and a normal response completes.
  • Build-context packaging and existing-Secret release checks pass; unit and packaging checks are included in CI.

Upstream implementation: BerriAI/litellm#40243

@zoroyihan7 zoroyihan7 closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant