Repository navigation
fix(sse): surface OpenRouter mid-stream error chunks instead of a false empty success - #8210
Merged
diegosouzapw merged 3 commits intoJul 23, 2026
Conversation
…se empty success OpenRouter (and similar OpenAI-compatible aggregators) can send an HTTP 200 SSE stream whose body carries a chat.completion.chunk with empty choices and a top-level error object instead of any delta -- e.g. the underlying provider hitting its own capacity limit mid-request. The Responses-API response translator's `!chunk.choices?.length` branch treated this exactly like a legitimate trailing-usage/no-op chunk, so the stream silently ended with response.completed / error: null / output: [] -- a false "successful but empty" response masking a real 502 provider_unavailable failure. Live repro: request 1784726796287-a45bb3 (OpenClaw via the default combo, nvidia/nemotron-3-ultra-550b-a55b:free via OpenRouter) got HTTP 200 with 0 tokens in/out after Nvidia returned "Worker local total request limit reached (33/32)" mid-stream; the client saw an empty completed response with no indication anything failed. Mirrors the Gemini mid-stream error fix (cheaper-inference#4177): set state.upstreamError from the in-band error chunk so stream.ts's existing upstreamError handling takes over (sendCompleted emits status: "failed" with a real error object). Co-Authored-By: Markus Hartung <markus.hartream@gmail.com>
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
…8210) Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
# Conflicts: # config/quality/file-size-baseline.json
Collaborator
|
Thanks, @hartmark! 🙏 Merged — surfaces OpenRouter-style mid-stream aggregator errors (HTTP 200 SSE with empty choices + top-level error) as a real upstream error instead of a false empty-success, so combo fallback kicks in. Rebased onto the current release tip and rebaselined openai-responses.ts for the +26 own-growth. Test 2/2 green. |
Merged
HouMinXi
pushed a commit
to HouMinXi/OmniRoute
that referenced
this pull request
Aug 2, 2026
…se empty success (cheaper-inference#8210) * fix(sse): surface OpenRouter mid-stream error chunks instead of a false empty success OpenRouter (and similar OpenAI-compatible aggregators) can send an HTTP 200 SSE stream whose body carries a chat.completion.chunk with empty choices and a top-level error object instead of any delta -- e.g. the underlying provider hitting its own capacity limit mid-request. The Responses-API response translator's `!chunk.choices?.length` branch treated this exactly like a legitimate trailing-usage/no-op chunk, so the stream silently ended with response.completed / error: null / output: [] -- a false "successful but empty" response masking a real 502 provider_unavailable failure. Live repro: request 1784726796287-a45bb3 (OpenClaw via the default combo, nvidia/nemotron-3-ultra-550b-a55b:free via OpenRouter) got HTTP 200 with 0 tokens in/out after Nvidia returned "Worker local total request limit reached (33/32)" mid-stream; the client saw an empty completed response with no indication anything failed. Mirrors the Gemini mid-stream error fix (cheaper-inference#4177): set state.upstreamError from the in-band error chunk so stream.ts's existing upstreamError handling takes over (sendCompleted emits status: "failed" with a real error object). Co-Authored-By: Markus Hartung <markus.hartream@gmail.com> * chore(quality): file-size baseline for own-growth (cheaper-inference#8210) Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Markus Hartung <markus.hartream@gmail.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…se empty success (cheaper-inference#8210) * fix(sse): surface OpenRouter mid-stream error chunks instead of a false empty success OpenRouter (and similar OpenAI-compatible aggregators) can send an HTTP 200 SSE stream whose body carries a chat.completion.chunk with empty choices and a top-level error object instead of any delta -- e.g. the underlying provider hitting its own capacity limit mid-request. The Responses-API response translator's `!chunk.choices?.length` branch treated this exactly like a legitimate trailing-usage/no-op chunk, so the stream silently ended with response.completed / error: null / output: [] -- a false "successful but empty" response masking a real 502 provider_unavailable failure. Live repro: request 1784726796287-a45bb3 (OpenClaw via the default combo, nvidia/nemotron-3-ultra-550b-a55b:free via OpenRouter) got HTTP 200 with 0 tokens in/out after Nvidia returned "Worker local total request limit reached (33/32)" mid-stream; the client saw an empty completed response with no indication anything failed. Mirrors the Gemini mid-stream error fix (cheaper-inference#4177): set state.upstreamError from the in-band error chunk so stream.ts's existing upstreamError handling takes over (sendCompleted emits status: "failed" with a real error object). Co-Authored-By: Markus Hartung <markus.hartream@gmail.com> * chore(quality): file-size baseline for own-growth (cheaper-inference#8210) Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: Markus Hartung <markus.hartream@gmail.com> Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenRouter (and similar OpenAI-compatible aggregators) can send an HTTP
200SSE stream whose body carries achat.completion.chunkwith emptychoicesand a top-levelerrorobject instead of any delta — e.g. the underlying provider hitting its own capacity limit mid-request. The Responses-API response translator's!chunk.choices?.lengthbranch treated this exactly like a legitimate trailing-usage/no-op chunk, so the stream silently ended withresponse.completed/error: null/output: []— a false "successful but empty" response masking a real502 provider_unavailablefailure.Live repro: a real production request (OpenRouter,
nvidia/nemotron-3-ultra-550b-a55b:free) got HTTP 200 with 0 tokens in/out after Nvidia returned"Worker local total request limit reached (33/32)"mid-stream; the client saw an empty completed response with no indication anything failed, which surfaced downstream as a confusing agent-side "couldn't generate a response" failure with zero diagnostic signal.Mirrors the existing Gemini mid-stream error fix (
#4177): setsstate.upstreamErrorfrom the in-band error chunk sostream.ts's existingupstreamErrorhandling takes over (sendCompletedemitsstatus: "failed"with a real error object instead of a fabricated empty success).Test plan
tests/unit/openrouter-midstream-error-chunk.test.ts(new, 2 tests) — TDD: confirmed RED before the fix (chunk silently dropped,state.upstreamErrornever set), GREEN after. Covers the502case and a429rate-limit-coded case.