Repository navigation
fix(sse): cap streaming headers-wait timeout to a client-realistic ceiling (#11526) - #11668
Merged
Merged
Conversation
…iling (#11526) The fetch-start (headers-wait) phase inherited the flat, non-adaptive FETCH_TIMEOUT_MS (default 600_000ms) with no ceiling comparable to a real client's patience, unlike the body-phase readiness watchdog which already adapts per payload shape. Codex's own hard client-abort window for a stalled turn is ~120s, 5x shorter than the old default — so when an upstream never returned any response at all (not even headers, e.g. a stalled NVIDIA target behind a tool-heavy Responses->Chat translation), OmniRoute kept the connection open on keepalives only, guaranteeing the client gave up first with an opaque 499 instead of OmniRoute detecting and failing the stall fast. Adds resolveFetchStartTimeout() (open-sse/utils/fetchStartTimeoutPolicy.ts) that caps the headers-wait timeout to 110s for STREAMING requests only, leaving non-streaming requests on the existing flat default. Wired into BaseExecutor.execute(). The existing TimeoutError classification path (chatCore.ts) already maps this to a 504, so no change was needed there. Regression test: tests/unit/issue-11526-repro.test.ts.
This was referenced Aug 26, 2026
Collaborator
Author
|
Validated in local merge-train (train tip a3a96af7308584900f0fab3ab4afd8499391230a, boarded #11663 #11664 #11665 #11667 #11668 #11669 #11670 #11678 on .113 /srv/omniroute-train) — typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity all GREEN. npm run test:unit showed 21 failures across 15 test files, none touching this PR's diff; reproduced identically (same test names, same asserts) on origin/release/v3.8.51 alone with zero PRs boarded (targeted re-run, 21/94 fail, same failure set) — confirmed base-red/environment-load drift per merge-gates.md §3, not introduced by any of the 8 boarded PRs. |
This was referenced Aug 26, 2026
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #11526
release/v3.8.51, not introduced by this diff)Root cause
Two watchdogs govern a stalled upstream on the Responses->Chat translated path, and they were asymmetric:
resolveStreamReadinessTimeout,open-sse/utils/streamReadinessPolicy.ts) — fires after headers arrive, tool-heavy/large-history adaptive. For the reported payload shape it computes well under Codex's own ~120s client-abort patience.fetchStartTimeoutMsinBaseExecutor.execute(), backed byFETCH_TIMEOUT_MS, default 600_000ms) — governs the wait before any response arrives at all, flat and non-adaptive.The issue's own log field
hasResponseBody: falseshows watchdog #1 never engaged — the upstream (an NVIDIA target) never returned a response at all. The only ceiling left was the 10-minute default, 5x longer than a real client's patience, so OmniRoute kept the connection alive on keepalives only, guaranteeing the client aborted first with an opaque 499 instead of OmniRoute detecting and failing the stall fast.Fix
Adds
resolveFetchStartTimeout()(open-sse/utils/fetchStartTimeoutPolicy.ts) that caps the headers-wait timeout to 110s (under Codex's ~120s abort window) for streaming requests only — non-streaming requests keep the existing flatFETCH_TIMEOUT_MSdefault unchanged, per the plan-file's risk note (some providers are legitimately slow to accept a connection on non-streaming paths). Wired intoBaseExecutor.execute()at the one call site that computesfetchStartTimeoutMs.The existing
TimeoutErrorclassification path (chatCore.ts) already maps a fetch-start timeout to HTTP 504 (GATEWAY_TIMEOUT), which is in the circuit-breaker-eligible{408,500,502,503,504}set perAGENTS.md— confirmed by reading the code, no change needed there.Scope note
This PR addresses the confirmed, in-our-code, fixable gap (the headers-phase timeout asymmetry). The plan-file's secondary hardening item — truncating the oversized (14KB) flattened
tool_searchtool description for non-Codex-native chat-completions upstreams — is explicitly deferred pending reporter/VPS confirmation that it's the actual NVIDIA-side trigger, soClosesis used because this PR is the full fix for the reported stall-then-499 symptom (the headers-phase ceiling), while that separate follow-up hardening remains open as a distinct, unconfirmed item.Testing (TDD, Hard Rule #18)
tests/unit/issue-11526-repro.test.ts— new regression test:resolveFetchStartTimeoutdidn't exist)FETCH_TIMEOUT_MSdefault; a base timeout already under the cap is left untouchedAlso re-ran the existing suites touching the changed symbols:
tests/unit/stream-readiness-policy.test.ts+tests/unit/runtime-timeouts.test.ts— 33/33 passtests/unit/executor-default-base.test.ts(BaseExecutor.execute / DefaultExecutor, incl. "BaseExecutor.execute clears the startup timeout after headers arrive") — 51/51 passGates run
node scripts/check/check-file-size.mjs— OKnode scripts/check/check-complexity.mjs— OK (2684 violations, baseline 2774)node scripts/check/check-cognitive-complexity.mjs— OK (1208 violations, baseline 1223)node scripts/check/check-changelog-integrity.mjs— OKnpm run typecheck:core— cleannpx eslint --suppressions-location config/quality/eslint-suppressions.json <changed files>— clean, 0 errors/warningsChangelog
changelog.d/fixes/11526-responses-headers-phase-timeout-cap.md