Repository navigation
feat(sse): transparent stream recovery (free-claude-code port, Fase 4, opt-in) - #4131
Merged
Merged
Conversation
…v STREAM_RECOVERY_ENABLED)
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
diegosouzapw
added a commit
that referenced
this pull request
Jun 18, 2026
…ask 4.4) Extends the opt-in stream recovery (#4131) to the POST-commit case. Until now an upstream truncation after bytes already reached the client either errored or closed mid-response (an early-retry would replay text the client already saw). For a plain-text OpenAI-compatible stream we can instead re-request with the partial answer as an assistant prefill and stitch only the missing suffix. streamRecovery.ts — new pure, unit-tested primitives: - scanOpenAiSseText(): extract assistant text + detect tool_calls + terminal marker; reports parsedOpenAi:false for non-OpenAI bodies (Anthropic → safe fallback). - makeContinuationBody(): append the partial as an assistant turn (OpenAI shape). - trimContinuationOverlap(): drop a duplicated seam so the join is append-only. - createRecoverableStream() gains an optional continueStream() path: on a post-commit truncation it tracks the text already sent, re-requests, drains the continuation, emits the de-duplicated suffix + a clean terminal. Bounded by maxContinuations. Safety / scope (documented): - OFF by default behind its own setting (STREAM_RECOVERY_MIDSTREAM_ENABLED / ResilienceSettings.streamRecovery.continueMidStream), independent of the early-retry flag — different risk profile (recovered tail arrives as one burst). - NEVER continues once a tool call has started streaming (would corrupt tool JSON). - Only plain-text OpenAI-compatible upstreams; Anthropic/others fall back to the unchanged #4131 error/close behavior. chatCore wiring: the early-retry reopen thunk and the continuation thunk now share a runUpstreamStream(body) helper (DRY); continuation re-runs the upstream with makeContinuationBody(bodyToSend, …). Gated; OFF path byte-identical. Hard Rule #18: validated by integration tests that simulate a mid-stream cut with mocked streams (silent post-commit truncation AND a post-commit transport error), asserting the client sees the full answer exactly once with overlap trimmed — no VPS needed. 16 new tests (10 primitives + 4 wiring + 2 settings). 37 existing #4131 tests stay green. chatCore.ts +29 rebaselined (logic lives in streamRecovery.ts, <cap).
Merged
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
…, opt-in) (cheaper-inference#4131) Integrated into release/v3.8.29
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
…ask 4.4) (cheaper-inference#4147) Mid-stream continuation for truncated streams (FCC port, Task 4.4). Opt-in (STREAM_RECOVERY_MIDSTREAM_ENABLED, default OFF); plain-text OpenAI-compat only, nunca com tool-call; OFF byte-idêntico ao cheaper-inference#4131. Integrado em release/v3.8.29.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resumo
Porta o stream recovery transparente do free-claude-code (
core/anthropic/stream_recovery.py) para o OmniRoute: quando um stream SSE upstream trunca antes de qualquer byte chegar ao cliente, ele é retentado de forma invisível. É a Fase 4 do planodocs/superpowers/plans/2026-06-17-fcc-portable-features.md— o gap headline (#1 em ROI) do porte FCC→OmniRoute.Opt-in, default OFF. O holdback segura a janela de abertura por até
HOLDBACK_MS(750ms), o que adicionaria latência de time-to-first-token em toda request de streaming. Por isso é desligado por padrão e habilitado pelo operador viaResilienceSettings.streamRecovery.enabled(envSTREAM_RECOVERY_ENABLED=true|1|on). Com o setting OFF, o caminho de streaming é byte-idêntico ao atual (wrapReadableStreamWithFinalize).O que entra
open-sse/services/streamRecovery.ts(novo):HoldbackBuffer(segura a janela de abertura, commit por tempo OU bytes),TruncatedStreamError,isRetryableStreamError,hasTerminalMarker, ecreateRecoverableStream(wrapper deReadableStreamcom holdback + early-retry atéEARLY_RETRY_MAX=4re-aberturas).open-sse/config/constants.ts: constantesSTREAM_RECOVERY(750ms / 64KB / 4 retries — espelham o FCC).src/lib/resilience/settings.ts: novostreamRecovery.enabled(default lêSTREAM_RECOVERY_ENABLED, espelhando o padrão deproviderCooldown).open-sse/handlers/chatCore.ts: wiring condicional no ponto de retorno do stream — quando ligado, embrulha o body comcreateRecoverableStreame re-executa o mesmo upstream (mesma conta/body, sem re-adquirir o semáforo) na truncagem precoce..env.example+docs/reference/ENVIRONMENT.md.Semântica (verificada)
donesem marcador terminal ([DONE]/message_stop) com holdback não-commitado → descarta a janela + re-abre (até 4×).Validação (Hard Rule #18)
tests/unit/stream-recovery.test.ts, 17 testes):HoldbackBuffer(commit por tempo/bytes/flush/discard),isRetryableStreamError,hasTerminalMarker, ecreateRecoverableStream(retry transparente, give-up best-effort, commit-then-error, clean-stream, max-retries, cancel-no-reopen).tests/integration/stream-recovery-wiring.test.ts, 2 testes): fetch#1 trunca (sem[DONE]) → fetch#2 completa; ON recupera (2 fetches, cliente recebe só o stream recuperado,PARTIALdescartado não vaza); OFF chama o upstream 1× só (zero mudança de comportamento).tests/unit/resilience-settings-stream-recovery.test.ts, 4 testes): default OFF, override, normalização, merge.typecheck:coreexit 0, ESLint 0,check:file-size(chatCore rebaselinado 5898→5980 com justificativa),check:env-doc-sync,check:cycles,check:any-budget:t11— todos verdes.discard()agora vivo notryReopen; guarda decancelledcontra fetch desperdiçado no cancel).Deferido (follow-up)
Test Plan
node --import tsx/esm --test tests/unit/stream-recovery.test.ts→ 17/17node --import tsx/esm --test tests/unit/resilience-settings-stream-recovery.test.ts→ 4/4node --import tsx/esm --test tests/integration/stream-recovery-wiring.test.ts→ 2/2npm run typecheck:core→ exit 0npm run check:file-size && npm run check:env-doc-sync && npm run check:any-budget:t11→ verdes🤖 Generated with Claude Code