Skip to content

fix(sse): stop an empty Claude stream from emptying the whole combo - #14314

Merged
diegosouzapw merged 3 commits into
diegosouzapw:release/v3.8.51from
fouadSalkini:fix/claude-passthrough-empty-response
Sep 24, 2026
Merged

diegosouzapw merged 3 commits into
diegosouzapw:release/v3.8.51from
fouadSalkini:fix/claude-passthrough-empty-response

Conversation

@fouadSalkini

@fouadSalkini fouadSalkini commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ base-red inherited: #13866

Symptom

A client with four Claude accounts repeatedly got:

429 Service temporarily unavailable: all targets were skipped by pre-dispatch filters reset after 50m 0s

Two of those four accounts were healthy and not in cooldown. Clearing the cooldowns changed nothing — the cooldowns were never the cause.

What was actually happening

account A → 502 "Claude returned an empty response (no content block)"
account B → 502  same
account C → 429  genuine rate limit
account D → 429  genuine rate limit
→ 429 "all targets were skipped by pre-dispatch filters"

The "reset after 50m" is the rate-limited accounts' quota window, which is what made this look like a cooldown problem.

Measured on 2026-09-20/21: claude-opus-5 succeeded 78.9% of the time overall (521 attempts), but only 35.8% on the native Claude Code passthrough keys — 57.8% returned the empty-response 502 (63 of 109 on one key). It tracks the request shape, not the account.

Three compounding defects

1. safeguards is forwarded and rejected. Anthropic rejects unknown top-level fields outright — 400 safeguards: Extra inputs are not permitted. Newer Claude Code builds send safeguards; the pure-passthrough path forwards the body verbatim, so every such request 400s. Stripped, following the existing top_p precedent a few lines above. The field is client-side only and the request is rejected whole rather than degraded, so dropping it is strictly better than failing.

2. A synthesized 502 caused a 60s per-model lockout on healthy accounts. When the rejection arrives mid-stream the stream ends with no content block, and OmniRoute synthesizes a 502 (stream.ts → emitClaudeEmptyStreamErrorAndAbort, code empty_response). markAccountUnavailable treats any status >= 500 on a per-model-quota provider as an upstream per-model server error and locked claude-opus-5 for 60s — on an account that was fine. With the other two accounts genuinely rate-limited, all four targets vanished and the combo returned the pre-dispatch 429.

A bare 500 was already exempt as "intermittent and NOT model-specific". This 502 is not even an upstream status, so it is now exempt too. 502/503/504 from a real upstream keep the existing lockout path (#6216).

3. The real error was masked. Every in-stream failure collapsed to "streaming upstream error", and the client only ever saw "Claude returned an empty response (no content block)". That masking hid both rejections above for days. The upstream error type and message now flow into the failure reason and the call log.

Tests

New tests/unit/claude-passthrough-empty-response.test.ts (7 tests), each TDD-verified against the unfixed code:

  • safeguards is listed and stripped while real payload survives
  • the synthesized empty-response 502 is recognised; a genuine upstream 502, an empty message, and 503/429 with the same text are not
  • the upstream error type and message reach the failure reason — isolated check with only validateQuality.ts reverted fails with reason must name the upstream error type, got: streaming upstream error
check result
new suite 7/7
quality-validation-benign-error, combo-quality-validator-reasoning, combo-quality-tiny-budget-probe, anthropic-thinking-signature-recovery 31/31
combo-provider-cooldown-sibling (pins the 500 carve-out contract) 9/9
combo-model-lockout-honors-reset-1308, combo-provider-cooldown, gemini-deprecated-model-lockout 18/18
typecheck:core clean
eslint (changed files) clean (one pre-existing suppressed endpointPath warning, present unpatched)
prettier --check clean

The tool_addition half of this incident is a separate one-line fix on #13994, which owns the inline-tools code.

A client with four Claude accounts kept getting

  429 Service temporarily unavailable: all targets were skipped by
      pre-dispatch filters

while two of those accounts were healthy and not in cooldown. Clearing the
cooldowns changed nothing, because the cooldowns were never the problem.

Three defects compounded:

1. Anthropic rejects unknown top-level fields outright — `400 safeguards:
   Extra inputs are not permitted`. Newer Claude Code builds send
   `safeguards`, and the pure-passthrough path forwarded it verbatim, so
   every such request 400'd. Strip it, exactly as top_p is already stripped.

2. When that rejection arrived mid-stream, the stream ended with no content
   block and OmniRoute SYNTHESIZED a 502 (stream.ts, code "empty_response").
   markAccountUnavailable treats any status >= 500 on a per-model-quota
   provider as an upstream per-model server error and locked claude-opus-5
   for 60s — on a perfectly healthy account. With the other two accounts
   genuinely rate-limited, all four targets were gone and the combo returned
   the pre-dispatch 429. A 500 was already exempt; this 502 is not even an
   upstream status, so exempt it too.

3. Every in-stream failure collapsed to "streaming upstream error", and the
   client saw only "Claude returned an empty response (no content block)".
   That masking hid both real rejections above. Carry the upstream error type
   and message into the failure reason and the call log.

Measured 2026-09-21: claude-opus-5 was failing ~58% of native Claude Code
passthrough requests (63 of 109 on one key), against 79% success overall.
The native claude passthrough stripped the top-level `safeguards` field
unconditionally to stop `400 safeguards: Extra inputs are not
permitted`. That 400 only happens when the field reaches Anthropic
without its paired `dangerous-tool-use-2026-09-03` beta; stripping it
always made every gateway session ineligible for server-side auto mode
(https://code.claude.com/docs/en/auto-mode-classifier-billing).

Decide with the same mergeClientAnthropicBeta the executor uses: keep
`safeguards` when the paired beta will be forwarded, strip it
otherwise. The strip and the existing temperature/top_p strip move
into stripClaudeRejectedTopLevelFields so chatCore.ts stays under its
frozen size, and isSyntheticEmptyStreamFailure moves out of auth.ts
for the same reason (also fixing its orphaned doc comment).
@fouadSalkini

Copy link
Copy Markdown
Contributor Author

Update (d8b9885): the safeguards strip is now conditional.

The original 400 (safeguards: Extra inputs are not permitted) only happens when the auto mode safeguards field reaches Anthropic without its paired dangerous-tool-use-2026-09-03 beta. Stripping it unconditionally fixed that 400, but it also made every gateway session ineligible for server-side auto mode. Claude Code then shows "this session isn't eligible because your requests go through " (https://code.claude.com/docs/en/auto-mode-classifier-billing).

What changed

  • unpairedClaudeClientFields(clientBeta) reuses mergeClientAnthropicBeta, so the strip decision is exactly what the executor will forward. safeguards is kept when the paired beta travels with it and stripped otherwise, so the 400 guard stays.
  • stripClaudeRejectedTopLevelFields(body, headers) now owns both the safeguards strip and the existing temperature + top_p strip. This keeps chatCore.ts under its frozen size. The call site is 1 line, and chatCore is back under the cap.
  • isSyntheticEmptyStreamFailure moved to src/sse/services/syntheticEmptyStream.ts because auth.ts was over its frozen cap (3612 > 3592). This also fixes the doc comment it had displaced.
  • origin/release/v3.8.51 is merged in (merge commit, no force-push), so the feat(sse): forward the auto mode classifier beta so gateway sessions stay eligible #14312 allowlist is present.

Tests

  • tests/unit/claude-passthrough-empty-response.test.ts: 12/12 pass. They are data-driven over no beta, an unrelated beta, the paired beta, and the paired beta with odd casing and spacing. They also cover both header shapes (fetch Headers and a plain record) and the top_p strip.
  • The tests fail on the previous head (missing export) and pass after.
  • check:file-size OK, check:cycles OK, eslint clean.
  • typecheck:core has one inherited error (cliproxyAccountHealth.ts:157).
  • chatcore-translation-paths.test.ts has 4 failures that also fail on the pure base tip (Codex Responses-native and DeepSeek replay).

Live evidence (operator production gateway, deployed ahead of merge together with #14694)

One Linux release build was deployed to both nodes. They are byte-identical: per-file digest 61c37a775cac7b7f96e499b7e065b6e93afb073cbe7b5ec01b6b7a5859eeea45, 14,765 files. Both returned healthy 4s after the swap. The hermetic probe (claude --bare -p … --permission-mode auto --model claude-sonnet-5, Claude Code 2.1.280, routed to provider claude) behaved as follows:

  • Before: the "isn't eligible" notice was present.
  • After: the notice is absent, the tool call ran, and the gateway logged no safeguards 400.

⚠️ base-red inherited: #14547

@diegosouzapw
diegosouzapw merged commit b3867e4 into diegosouzapw:release/v3.8.51 Sep 24, 2026
9 of 16 checks passed
SCys pushed a commit to SCys/OmniRoute that referenced this pull request Sep 25, 2026
… reasons

diegosouzapw#13630 gated the same-target retry after a pre-content streaming upstream
error on an exact match: handlePreContentStreamRetry()
(open-sse/services/combo/executeTargetClassify.ts:35) compared
quality.reason !== "streaming upstream error". diegosouzapw#14314 (b3867e4) then
intentionally appended the upstream detail to that reason in
validateResponseQuality() ("streaming upstream error: <detail>") so the
real Anthropic rejection reaches the call log. Every detailed reason
failed the exact match, so protected (fallbackOnlyOnQuotaExhaustion) and
native-pinned targets stopped retrying once and failed instead.

validateQuality.ts now owns STREAMING_UPSTREAM_ERROR_REASON, builds the
reason in one helper and exports isStreamingUpstreamErrorReason(), which
the retry gate uses (bare or "<base>: <detail>", nothing else). The two
reason assertions in the diegosouzapw#13630 test move to the exact new text diegosouzapw#14314
documents, and a classifier test pins both forms plus near-misses.

Refs diegosouzapw#14547
fouadSalkini added a commit to fouadSalkini/OmniRoute that referenced this pull request Sep 26, 2026
An empty Claude passthrough stream no longer empties the whole combo. The
chatCore.ts wiring for this change is carried by the agent sessions commit,
which owns the final version of that shared file.
fouadSalkini added a commit to fouadSalkini/OmniRoute that referenced this pull request Sep 26, 2026
…egosouzapw#14864 agent sessions

Agent session and project attribution (diegosouzapw#14833), the /v1/me/sessions
self-service endpoints (diegosouzapw#14840), and /v1/me/sessions/{id}/messages with
opt-in turn capture (diegosouzapw#14864), including the chatCore wiring, as deployed.

Migrations keep the deployed numbers 9191-9193, which production databases
have already applied. chatCore.ts also carries the diegosouzapw#14314 and diegosouzapw#14862 wiring.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants