Skip to content

fix(sse): combo failover for OpenAI streams truncated without finish_reason - #7568

Merged
diegosouzapw merged 1 commit into
release/v3.8.49from
fix/7285-combo-finish-reason
Jul 17, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.49from
fix/7285-combo-finish-reason

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Closes #7285

Root cause

validateResponseQuality() (open-sse/services/combo/validateQuality.ts::parseAccumulatedSse()) only recognizes Claude SSE lifecycle events (message_start, content_block_*, message_stop, message_delta.delta.stop_reason). hasOpenAICompatibleStreamValue() (open-sse/utils/streamHelpers.ts) only inspects delta.content/reasoning_content/reasoning_text/tool_calls — it never looks at choice.finish_reason.

Result: an OpenAI-shape upstream that returns HTTP 200, emits a role-only delta (or other non-content bytes), and then closes the connection without ever sending a chunk carrying finish_reason (and without data: [DONE]) falls through every existing failover branch and lands on the generic "replay buffered bytes" path — {valid: true, clonedResponse}. The truncated stream is forwarded to the client as a success instead of triggering combo failover to a sibling target. auto/*:free suffixes amplify exposure because they narrow the candidate pool to free-tier providers, the population most likely to truncate mid-stream.

Fix

Added OpenAI-shape lifecycle tracking (hasChoicePayload / hasTerminalMarker) parallel to the existing Claude tracking in validateQuality.ts:

  • New helpers in streamHelpers.ts: isOpenAIChoicesPayload() and hasOpenAIFinishReason().
  • parseAccumulatedSse() now marks hasChoicePayload on any choices[] chunk, and hasTerminalMarker when a choice carries finish_reason != null or the data: [DONE] sentinel appears.
  • At stream end: if an OpenAI-shape chunk was seen but the stream closed with no terminal marker and no recognized content, the response is marked invalid ({valid: false}) so combo failover retries a sibling target.

Healthy OpenAI streams are unaffected: a stream carrying real content (or finish_reason) exits the bounded peek loop early via the existing foundContent branch, before ever reaching the new check — preserving the #3399/#3685 pass-through contract (a slow-but-healthy stream must not be misclassified as truncated). STREAM_RECOVERY_ENABLED stays untouched (default false) — this fix lives entirely in validateResponseQuality(), independent of streamRecovery.ts's retry path.

TDD proof (Hard Rule #18)

New regression test: tests/unit/combo-streaming-openai-no-finish-reason-7285.test.ts

RED (before fix):

✖ #7285 RED: OpenAI-shape stream with role-only delta and NO finish_reason should fail over but currently passes as valid
  AssertionError: expected failover (valid:false)
  true !== false
✔ #7285 control: a healthy OpenAI stream ending with finish_reason still passes through (#3399/#3685 no-regression)
tests 2 / pass 1 / fail 1

GREEN (after fix):

✔ #7285 RED: OpenAI-shape stream with role-only delta and NO finish_reason should fail over but currently passes as valid
✔ #7285 control: a healthy OpenAI stream ending with finish_reason still passes through (#3399/#3685 no-regression)
tests 2 / pass 2 / fail 0

Gates run (all green)

  • node --import tsx/esm --test tests/unit/combo-streaming-openai-no-finish-reason-7285.test.ts — 2/2 pass
  • Existing touched-area suites (all pass, no regressions):
    • tests/unit/combo-quality-validator-reasoning.test.ts (12/12)
    • tests/unit/combo-round-robin-streaming-lock-3811.test.ts (1/1)
    • tests/unit/combo-stream-readiness-fallback.test.ts (11/11)
    • tests/unit/combo-streaming-empty-content-failover.test.ts (6/6, Streaming combo returns "[Proxy Error] The upstream API returned an empty response" instead of failing over #3685 contract)
    • tests/unit/empty-response-hardening.test.ts (12/12)
    • tests/unit/masked-200-exhaustion-fallback-6427.test.ts (3/3)
    • tests/unit/streamHelpers.test.ts (20/20)
    • tests/unit/streaming-empty-content-block-1382.test.ts (2/2)
    • tests/unit/validate-response-quality.test.ts (8/8)
    • tests/unit/gemini-cli-ansi-sanitization.test.ts (5/5)
    • tests/unit/correctness/sse-parser.property.test.ts (4/4)
  • node scripts/check/check-file-size.mjs — OK (no growth on frozen files, new test file untouched by cap)
  • node scripts/check/check-complexity.mjs — OK (2054 ≤ baseline 2056)
  • node scripts/check/check-cognitive-complexity.mjs — OK (889 ≤ baseline 890)
  • npm run typecheck:core — clean, exit 0
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files> — clean, exit 0

npm run typecheck:noimplicit:core surfaces pre-existing implicit-any errors in unrelated files (open-sse/services/combo.ts, open-sse/utils/usageTracking.ts, src/shared/services/cliRuntime.ts) — none in the files this PR touches; not introduced by this change.

Scope

Diff strictly scoped to the issue: open-sse/services/combo/validateQuality.ts, open-sse/utils/streamHelpers.ts, the new regression test, and a changelog fragment. No drive-by refactors.

…reason (#7285)

validateResponseQuality() only recognized Claude SSE lifecycle events
(message_start/content_block_*/message_stop/message_delta.stop_reason).
An OpenAI-shape stream (choices[].delta) that emits some bytes (e.g. a
role-only delta) and then closes without ever carrying finish_reason
(and without a data: [DONE] sentinel) fell through to the generic
replay branch and was forwarded to the client as a success instead of
triggering combo failover.

Adds OpenAI-shape lifecycle tracking (hasChoicePayload/hasTerminalMarker)
parallel to the existing Claude tracking: when an OpenAI-shape chunk was
seen but the stream ends without finish_reason or [DONE], and no
recognized content was found, mark the response invalid so combo
failover retries a sibling target. Healthy OpenAI streams (finish_reason
present, or real content found) are unaffected — they exit the peek
loop before reaching this check, preserving the #3399/#3685
pass-through contract.

Regression test: tests/unit/combo-streaming-openai-no-finish-reason-7285.test.ts

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes issue #7285 by introducing tracking for OpenAI-shape SSE streams to detect when they are truncated without a finish_reason or [DONE] sentinel, allowing combo failover to trigger correctly. The changes include new lifecycle flags, helper functions, and comprehensive unit tests. The review feedback suggests adding defensive checks in the new helper functions to prevent potential runtime TypeError crashes when handling malformed or null payloads.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +327 to +334
export function isOpenAIChoicesPayload(parsed: Record<string, unknown>): boolean {
return Array.isArray(parsed.choices);
}

export function hasOpenAIFinishReason(parsed: Record<string, unknown>): boolean {
if (!Array.isArray(parsed.choices)) return false;
return parsed.choices.some((choice) => isRecord(choice) && choice.finish_reason != null);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To prevent potential runtime TypeError crashes when parsing malformed or unexpected SSE payloads (such as data: null which parses to null), we should defensively check that parsed is a non-null object before accessing its properties.

Suggested change
export function isOpenAIChoicesPayload(parsed: Record<string, unknown>): boolean {
return Array.isArray(parsed.choices);
}
export function hasOpenAIFinishReason(parsed: Record<string, unknown>): boolean {
if (!Array.isArray(parsed.choices)) return false;
return parsed.choices.some((choice) => isRecord(choice) && choice.finish_reason != null);
}
export function isOpenAIChoicesPayload(parsed: Record<string, unknown>): boolean {
return !!parsed && Array.isArray(parsed.choices);
}
export function hasOpenAIFinishReason(parsed: Record<string, unknown>): boolean {
if (!parsed || !Array.isArray(parsed.choices)) return false;
return parsed.choices.some((choice) => isRecord(choice) && choice.finish_reason != null);
}

@diegosouzapw
diegosouzapw merged commit a1299d2 into release/v3.8.49 Jul 17, 2026
20 checks passed
@diegosouzapw
diegosouzapw deleted the fix/7285-combo-finish-reason branch July 17, 2026 08:32
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…reason (diegosouzapw#7285) (diegosouzapw#7568)

validateResponseQuality() only recognized Claude SSE lifecycle events
(message_start/content_block_*/message_stop/message_delta.stop_reason).
An OpenAI-shape stream (choices[].delta) that emits some bytes (e.g. a
role-only delta) and then closes without ever carrying finish_reason
(and without a data: [DONE] sentinel) fell through to the generic
replay branch and was forwarded to the client as a success instead of
triggering combo failover.

Adds OpenAI-shape lifecycle tracking (hasChoicePayload/hasTerminalMarker)
parallel to the existing Claude tracking: when an OpenAI-shape chunk was
seen but the stream ends without finish_reason or [DONE], and no
recognized content was found, mark the response invalid so combo
failover retries a sibling target. Healthy OpenAI streams (finish_reason
present, or real content found) are unaffected — they exit the peek
loop before reaching this check, preserving the diegosouzapw#3399/diegosouzapw#3685
pass-through contract.

Regression test: tests/unit/combo-streaming-openai-no-finish-reason-7285.test.ts
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…reason (diegosouzapw#7285) (diegosouzapw#7568)

validateResponseQuality() only recognized Claude SSE lifecycle events
(message_start/content_block_*/message_stop/message_delta.stop_reason).
An OpenAI-shape stream (choices[].delta) that emits some bytes (e.g. a
role-only delta) and then closes without ever carrying finish_reason
(and without a data: [DONE] sentinel) fell through to the generic
replay branch and was forwarded to the client as a success instead of
triggering combo failover.

Adds OpenAI-shape lifecycle tracking (hasChoicePayload/hasTerminalMarker)
parallel to the existing Claude tracking: when an OpenAI-shape chunk was
seen but the stream ends without finish_reason or [DONE], and no
recognized content was found, mark the response invalid so combo
failover retries a sibling target. Healthy OpenAI streams (finish_reason
present, or real content found) are unaffected — they exit the peek
loop before reaching this check, preserving the diegosouzapw#3399/diegosouzapw#3685
pass-through contract.

Regression test: tests/unit/combo-streaming-openai-no-finish-reason-7285.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(routing): combo failover misses OpenAI-shape streams that end without finish_reason — truncated response forwarded as success (auto/*:free)

1 participant