Skip to content

fix(sse): recognize bare delta.reasoning in combo streaming quality-gate peek (#13620) - #13806

Merged
diegosouzapw merged 2 commits into
release/v3.8.51from
fix/13620-combo-reasoning-only-sse-burst
Sep 16, 2026
Merged

diegosouzapw merged 2 commits into
release/v3.8.51from
fix/13620-combo-reasoning-only-sse-burst

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Closes #13620

Root cause (short)

hasOpenAICompatibleStreamValue() in open-sse/utils/streamHelpers.ts — the sole
content-detection predicate for OpenAI-shape SSE chunks — recognized content,
reasoning_content, reasoning_text, and tool_calls, but not bare delta.reasoning
(the field Ollama Cloud emits for reasoning-capable models like deepseek-v4.1-flash,
glm-5.3-flash, glm-5.3).

This predicate is the only exit condition for the combo quality-gate's bounded SSE peek
(open-sse/services/combo/validateQuality.ts), which every combo dispatch path runs
before forwarding a stream to the client. Because bare reasoning chunks never
satisfied it, the peek loop kept reading and buffering every reasoning delta until the
first real content delta arrived, then replayed the entire buffered prefix in one
synchronous burst — the "dead silence, then a burst" the reporter measured (196s
silence, then 14,051 reasoning deltas in 1.24s). A stream that finished with only
reasoning deltas (no content/tool_calls) was also misclassified as "streaming openai terminated with empty completion" and failed over unnecessarily.

Non-combo (direct) requests were unaffected because they stream via
ensureStreamReadiness(), whose accept-list already includes bare reasoning.

Fix

Swap the two hand-rolled reasoning_content/reasoning_text checks in
hasOpenAICompatibleStreamValue() for the already-imported hasAnyReasoningSignal()
helper (open-sse/utils/reasoningFields.ts), which is already used correctly a few
lines below in hasValuableContent(). This aligns the streaming predicate with the
non-streaming validator, hasValuableContent(), and ensureStreamReadiness(), all of
which already treat bare reasoning as real content. One-line, low-risk change —
hasAnyReasoningSignal() already guards on non-empty strings, so no new
false-positive risk from empty reasoning: "" deltas.

Regression test

tests/unit/issue-13620-combo-reasoning-only-sse-burst.test.ts — 3 tests:

  1. isKnownNonClaudeStreamPayload() recognizes bare delta.reasoning (root cause).
  2. The peek stops buffering at the first reasoning delta instead of buffering through
    to the first content delta (behavioral, counts physical stream chunks pulled).
  3. A reasoning-only stream that terminates cleanly (finish_reason/[DONE]) is
    valid: true, not rejected as an empty completion (secondary failure mode).

RED (current release tip, before the fix):

✖ #13620 root cause: isKnownNonClaudeStreamPayload must recognize bare delta.reasoning (Ollama Cloud shape)
  AssertionError: false !== true
✖ #13620 behavioral: peek must stop buffering at the first reasoning delta, not wait for content
  AssertionError: expected the peek to stop at the first reasoning chunk (1 chunk pulled);
  actual: 5 chunks were pulled (buffered) before the peek recognized any content
✖ #13620 secondary failure mode: reasoning-only stream that terminates cleanly must not be rejected as an empty completion
  AssertionError: expected valid ... — got reason: streaming openai terminated with empty completion
ℹ tests 3 | pass 0 | fail 3

GREEN (after the fix):

✔ #13620 root cause: isKnownNonClaudeStreamPayload must recognize bare delta.reasoning (Ollama Cloud shape)
✔ #13620 behavioral: peek must stop buffering at the first reasoning delta, not wait for content
✔ #13620 secondary failure mode: reasoning-only stream that terminates cleanly must not be rejected as an empty completion
ℹ tests 3 | pass 3 | fail 0

Note: the "behavioral" test's threshold is <= 2 physical chunks pulled rather than
exactly 1. The WHATWG ReadableStream spec itself speculatively pulls one extra
chunk into its internal queue right after a reader.read() resolves (default
highWaterMark backpressure refill), independent of application code — confirmed with
a standalone repro against a plain ReadableStream/Response. That prefetch is
unrelated to the bug: the buggy code pulled 4-5 chunks (multiple logical reads inside
the peek loop), while the fixed code pulls at most 2 (one logical read + the engine's
own prefetch).

Gates run

  • npx eslint --suppressions-location config/quality/eslint-suppressions.json open-sse/utils/streamHelpers.ts tests/unit/issue-13620-combo-reasoning-only-sse-burst.test.ts — clean, exit 0.
  • npm run check:open-sse-typecheck — openSseTypecheckErrors=0, OK (all within frozen baseline).
  • node scripts/check/check-file-size.mjs — no violation on touched files (1 pre-existing violation on open-sse/utils/stream.ts, a file this PR does not touch — confirmed identical line count on origin/release/v3.8.51).
  • node scripts/check/check-complexity.mjs — OK, 2824 violations vs baseline 3218 (no new violation).
  • node scripts/check/check-cognitive-complexity.mjs — OK, 1276 violations vs baseline 1437 (no new violation).
  • node scripts/check/check-test-discovery.mjs — OK, new test file discovered.
  • DATA_DIR=$(mktemp -d) node --import tsx/esm --test --test-force-exit tests/unit/issue-13620-combo-reasoning-only-sse-burst.test.ts — 3/3 pass (RED→GREEN above).

Existing tests aligned

None needed to change. Ran the existing coverage for the touched predicate/function to confirm no regression, all green:

  • tests/unit/combo-quality-validator-reasoning.test.ts — 16/16 pass (non-streaming reasoning acceptance, unaffected by this streaming-only change).
  • tests/unit/combo-streaming-empty-completion-with-finish-reason-10404.test.ts — 2/2 pass (no-regression guard for the content-only/tool_calls-only truncation/empty-completion detection this PR shares a code path with).
  • tests/unit/streamHelpers.test.ts — 35/35 pass (hasValuableContent(), unwrapGeminiChunk, appendBoundedText, hasActiveDeltaValue — none touched by this change).

diegosouzapw and others added 2 commits September 15, 2026 19:07
…ate peek (#13620)

hasOpenAICompatibleStreamValue() in open-sse/utils/streamHelpers.ts only
recognized content, reasoning_content, reasoning_text, and tool_calls on an
OpenAI-shape delta — not bare delta.reasoning (the field Ollama Cloud emits
for deepseek-v4.1-flash, glm-5.3-flash, etc). Since this predicate is the
only exit condition for the combo quality-gate's bounded SSE peek
(validateQuality.ts), every reasoning delta was silently buffered until the
first real content delta arrived, then replayed in one synchronous burst —
and a stream that finished with reasoning-only content was misclassified as
"streaming openai terminated with empty completion" and failed over.

Fix: reuse the already-imported hasAnyReasoningSignal() helper (already used
correctly in hasValuableContent() a few lines below) instead of the two
hand-rolled string checks, aligning this predicate with the non-streaming
validator, hasValuableContent(), and ensureStreamReadiness(), which already
treat bare reasoning as real content.

Regression test: tests/unit/issue-13620-combo-reasoning-only-sse-burst.test.ts
@diegosouzapw
diegosouzapw merged commit 9e53fad into release/v3.8.51 Sep 16, 2026
19 of 21 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ate peek (cheaper-inference#13620) (cheaper-inference#13806)

Merged in the 2026-09-16 sweep of the maintainer's own open PRs, at the owner's explicit instruction. No push was made to the PR branch: the merge took the head as the owning session left it (verified OPEN, non-draft and MERGEABLE against the release tip immediately before merging).
@diegosouzapw
diegosouzapw deleted the fix/13620-combo-reasoning-only-sse-burst branch October 18, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(resilience): Combo quality-gate buffers reasoning-only SSE chunks — no client-visible output until the first content delta

1 participant