feat(sse): transparent stream recovery (free-claude-code port, Fase 4, opt-in)

.env.example

@@ -1250,6 +1250,15 @@ APP_LOG_TO_FILE=true
 # Accepted values: true|1|on (enable). Unset or anything else = disabled (default).
 # PROVIDER_COOLDOWN_ENABLED=true
 
+# Transparent stream recovery (free-claude-code port). When enabled, the opening SSE
+# window is briefly held (up to STREAM_RECOVERY.HOLDBACK_MS) so an upstream truncation
+# before any byte reaches the client can be retried invisibly. Opt-in: holding the
+# window adds up to that much time-to-first-token latency on every stream, so it is
+# OFF by default. Seeds ResilienceSettings.streamRecovery.enabled.
+# Used by: open-sse/services/streamRecovery.ts, open-sse/handlers/chatCore.ts
+# Accepted values: true|1|on (enable). Unset or anything else = disabled (default).
+# STREAM_RECOVERY_ENABLED=true
+
 # Stagger interval (ms) between provider token healthchecks at startup.
 # Used by: src/lib/tokenHealthCheck.ts.  Default: 3000.
 # HEALTHCHECK_STAGGER_MS=3000

config/quality/file-size-baseline.json

@@ -32,6 +32,7 @@
   "_rebaseline_2026_06_17_4034_usage_command": "PR #4034 own growth: apiKeys.ts 1633->1661 (+28 = the allow_usage_command additive column — fallback definition, parseAllowUsageCommand, prepared-statement SELECT column, and the metadata/create/update wiring, mirroring the existing disable_non_public_models accessor pattern) and chat.ts 1425->1432 (+7 = the handleInternalUsageCommand intercept hook at the existing post-auth chokepoint in handleChat). The reusable command logic lives in the new src/lib/usage/internalUsageCommand.ts (well under cap). Cohesive opt-in feature at the established column/dispatch boundaries; not extractable without splitting the api-keys domain module.",
   "_rebaseline_2026_06_17_ctx_editing_relays": "Context Editing relay coverage + 400-fallback (F4.2/F4.3) own growth: base.ts 1244->1292 (+48 at the existing dispatch chokepoint: extend the injection gate to anthropic-compatible-cc-* relays with an expanded comment documenting the claude-web / generic-anthropic-compatible exclusions; the contextEditingDisabled loop flag; and the 400-fallback block that, when an upstream rejects context_management with a 400, strips the param, re-serializes + re-signs (CCH), and retries the same URL once). Both edits sit on the per-URL request-build/fetch path and cannot be extracted without splitting the executor's fetch loop. No new file warranted — the logic is a few lines tightly coupled to the in-loop transformedBody/bodyString/response locals.",
   "_rebaseline_2026_06_17_ctx_editing_telemetry": "Context Editing telemetry (F4.1) own growth: chatCore.ts 5875->5898 (+23 = a best-effort, Claude-only, non-streaming block at the existing non-streaming response handler that extracts the provider's server-side context-editing receipt from responseBody and records it under engine \"context-editing\"). The pure extractor (extractContextEditingTelemetry) lives in the existing small open-sse/config/contextEditing.ts and the DB writer (recordContextEditingTelemetry) in src/lib/db/compressionAnalytics.ts — both under cap. Cohesive at the response-handling chokepoint next to the existing attachCompressionUsageReceipt call; not extractable without hiding the receipt-recording boundary.",
+  "_rebaseline_2026_06_17_stream_recovery": "Stream recovery wiring (free-claude-code port, Fase 4) own growth: chatCore.ts 5898->5980 (+82 at the existing streaming-return chokepoint: read the resolved streamRecovery.enabled setting once; when ON, a reopenStream() thunk re-executes the SAME upstream — it closes over the ~15 per-attempt executor locals (executor/provider/modelToCall/bodyToSend/upstreamStream/execCreds/extendedContext/headers builders/onCredentialsRefreshed/skipUpstreamRetry/contextEditingEnabled) — and the body is wrapped with createRecoverableStream for transparent early-retry; OFF keeps the byte-identical wrapReadableStreamWithFinalize path). All reusable logic (HoldbackBuffer, createRecoverableStream, isRetryableStreamError, hasTerminalMarker) lives in the new open-sse/services/streamRecovery.ts (well under cap). Not extractable without hiding the per-attempt executor-dispatch boundary; opt-in (default OFF). Structural shrink of chatCore.ts tracked in #3501.",
   "cap": 800,
   "frozen": {
     "open-sse/config/providerRegistry.ts": 4731,
@@ -47,7 +48,7 @@
     "open-sse/executors/muse-spark-web.ts": 1284,
     "open-sse/executors/perplexity-web.ts": 1013,
     "open-sse/handlers/audioSpeech.ts": 965,
-    "open-sse/handlers/chatCore.ts": 5898,
+    "open-sse/handlers/chatCore.ts": 5980,
     "open-sse/handlers/imageGeneration.ts": 3777,
     "open-sse/handlers/responseSanitizer.ts": 1103,
     "open-sse/handlers/search.ts": 1442,

docs/reference/ENVIRONMENT.md

@@ -758,6 +758,7 @@ Anthropic-compatible provider instead.
 | `PROVIDER_COOLDOWN_ENABLED`                     | _(unset → off)_  | `open-sse/services/providerCooldownTracker.ts`   | Opt-in global cross-request provider/connection cooldown tracking. OFF by default (overlaps Connection Cooldown / Provider Circuit Breaker). Accepts `true`/`1`/`on` to enable.                         |
 | `PROVIDER_COOLDOWN_MIN_MS`                      | `5000`           | `open-sse/services/providerCooldownTracker.ts`   | Minimum cooldown (ms) before a failed provider/connection is retried. Scaled exponentially with consecutive failures. Only used when `PROVIDER_COOLDOWN_ENABLED`.                                       |
 | `PROVIDER_COOLDOWN_MAX_MS`                      | `300000` (5 min) | `open-sse/services/providerCooldownTracker.ts`   | Maximum cooldown (ms) cap before a failed provider/connection is retried regardless. Only used when `PROVIDER_COOLDOWN_ENABLED`.                                                                        |
+| `STREAM_RECOVERY_ENABLED`                       | _(unset → off)_  | `open-sse/services/streamRecovery.ts`            | Opt-in transparent recovery of truncated upstream streams (free-claude-code port). Holds the opening SSE window so an early cutoff is retried invisibly; OFF by default (adds time-to-first-token latency). Accepts `true`/`1`/`on` to enable. |
 | `HEALTHCHECK_STAGGER_MS`                        | `3000`           | `src/lib/tokenHealthCheck.ts`                    | Stagger interval (ms) between provider token healthchecks at startup.                                                                                                                                   |
 | `REQUEST_RETRY`                                 | `2`              | `src/sse/services/cooldownAwareRetry.ts`         | Number of automatic retries on model-scoped cooldown responses before returning error to client.                                                                                                        |
 | `MAX_RETRY_INTERVAL_SEC`                        | `30`             | `src/sse/services/cooldownAwareRetry.ts`         | Max backoff interval (seconds) between cooldown retries. Capped by this value regardless of upstream `Retry-After`.                                                                                     |

open-sse/config/constants.ts

@@ -252,3 +252,24 @@ export const CREDENTIAL_HEALTH_CACHE_TTL = (() => {
   }
   return 300_000;
 })();
+
+/**
+ * Stream-recovery tuning (opt-in, see ResilienceSettings.streamRecovery).
+ *
+ * Ported from free-claude-code's always-on recovery (`core/anthropic/stream_recovery.py`).
+ * In OmniRoute the holdback is disabled by default because buffering the opening
+ * window adds up to HOLDBACK_MS of time-to-first-token latency on every stream;
+ * operators opt in via STREAM_RECOVERY_ENABLED / the resilience settings.
+ *
+ * - HOLDBACK_MS: how long the opening SSE window is held so an early truncation
+ *   can be retried transparently before any byte reaches the client.
+ * - BUFFER_MAX_BYTES: hard cap on the held window — commit (flush + passthrough)
+ *   as soon as this many bytes accumulate, regardless of the timer.
+ * - EARLY_RETRY_MAX: max transparent re-opens of the upstream stream while the
+ *   holdback is still uncommitted (free-claude-code uses 5 total attempts = 4 retries).
+ */
+export const STREAM_RECOVERY = {
+  HOLDBACK_MS: 750,
+  BUFFER_MAX_BYTES: 65536,
+  EARLY_RETRY_MAX: 4,
+} as const;

open-sse/handlers/chatCore.ts

@@ -67,7 +67,10 @@ import {
   STREAM_IDLE_TIMEOUT_MS,
   STREAM_READINESS_TIMEOUT_MS,
   ANTIGRAVITY_PRE_RESPONSE_TIMEOUT_CODE,
+  STREAM_RECOVERY,
 } from "../config/constants.ts";
+import { createRecoverableStream } from "../services/streamRecovery.ts";
+import { resolveResilienceSettings } from "@/lib/resilience/settings";
 import {
   classifyProviderError,
   PROVIDER_ERROR_TYPES,
@@ -4071,17 +4074,96 @@ export async function handleChatCore({
                   return res;
                 }
 
+                // Opt-in transparent stream recovery (free-claude-code port, default OFF).
+                // Only engages for a successful (2xx) stream — an error body must never be
+                // held or replayed. Setting is read once here from the cached resolved
+                // resilience settings; the default path is byte-for-byte unchanged.
+                const okStatus = res.response.status >= 200 && res.response.status < 300;
+                let streamRecoveryEnabled = false;
+                if (okStatus) {
+                  try {
+                    // Reuse the request-consolidated settings read (see line ~2076) — no
+                    // second DB/cache hit. Default OFF when the setting is absent.
+                    streamRecoveryEnabled = resolveResilienceSettings(settings).streamRecovery.enabled;
+                  } catch {
+                    streamRecoveryEnabled = false;
+                  }
+                }
+
+                let clientBody: ReadableStream<Uint8Array>;
+                if (streamRecoveryEnabled) {
+                  // Re-open the SAME upstream (same account + body) for a transparent retry.
+                  const reopenStream = async (): Promise<ReadableStream<Uint8Array> | null> => {
+                    try {
+                      const retryRaw = await executeWithUpstreamStartTimeout({
+                        executor,
+                        provider,
+                        model: modelToCall,
+                        signal: streamController.signal,
+                        log,
+                        execute: (signal) =>
+                          runWithCapture(providerRequestCapture, () =>
+                            executor.execute({
+                              model: modelToCall,
+                              body: bodyToSend,
+                              stream: upstreamStream,
+                              credentials: execCreds,
+                              signal,
+                              log,
+                              extendedContext,
+                              upstreamExtraHeaders: buildUpstreamHeadersForExecute(modelToCall),
+                              clientHeaders: buildExecutorClientHeaders(
+                                clientRawRequest?.headers,
+                                userAgent
+                              ),
+                              onCredentialsRefreshed,
+                              skipUpstreamRetry,
+                              contextEditing: { enabled: contextEditingEnabled },
+                            })
+                          ),
+                      });
+                      const retryRes = normalizeExecutorResult(retryRaw);
+                      const retryOk =
+                        retryRes.response.status >= 200 && retryRes.response.status < 300;
+                      if (retryOk && retryRes.response.body) {
+                        return retryRes.response.body as ReadableStream<Uint8Array>;
+                      }
+                      await retryRes.response.body?.cancel().catch(() => {});
+                      return null;
+                    } catch {
+                      return null;
+                    }
+                  };
+
+                  clientBody = createRecoverableStream(
+                    originalBody as ReadableStream<Uint8Array>,
+                    reopenStream,
+                    {
+                      finalize: acquireAccountSemaphoreRelease,
+                      onRetry: (attempt, err) =>
+                        log?.warn?.(
+                          "STREAM_RECOVERY",
+                          `transparent early-retry ${attempt}/${STREAM_RECOVERY.EARLY_RETRY_MAX} after ${
+                            (err as { name?: string })?.name || "truncation"
+                          }`
+                        ),
+                    }
+                  );
+                } else {
+                  clientBody = wrapReadableStreamWithFinalize(
+                    originalBody,
+                    acquireAccountSemaphoreRelease
+                  );
+                }
+
                 return {
                   ...res,
                   _executionCredentials: execCreds,
-                  response: new Response(
-                    wrapReadableStreamWithFinalize(originalBody, acquireAccountSemaphoreRelease),
-                    {
-                      status: res.response.status,
-                      statusText: res.response.statusText,
-                      headers: res.response.headers,
-                    }
-                  ),
+                  response: new Response(clientBody, {
+                    status: res.response.status,
+                    statusText: res.response.statusText,
+                    headers: res.response.headers,
+                  }),
                   headers: res.response.headers,
                 };
               }