fix(sse): bound slow keepalive path with absolute last-resort deadline - #14808
Merged
diegosouzapw merged 5 commits intoSep 25, 2026
Merged
diegosouzapw merged 5 commits into
diegosouzapw merged 5 commits into
Conversation
maxmad64bis
force-pushed
the
fix/keepalive-deadline-last-resort
branch
from
September 24, 2026 22:00
e186199 to
9b5f5d6
Compare
A handler that never resolves before first byte no longer holds the early-keepalive slow path open forever: at 33 minutes the wrapper emits the route error frame, closes the stream, aborts the handler, and warns once with correlation.
maxmad64bis
marked this pull request as ready for review
September 25, 2026 08:13
maxmad64bis
force-pushed
the
fix/keepalive-deadline-last-resort
branch
from
September 25, 2026 08:14
9b5f5d6 to
cc87bd5
Compare
…row unbounded withDeadlineSignal registered every request under a string token in a module-level Map that nothing ever pruned, so each streamed request left an entry behind for the life of the process. Release the entry when the keepalive wrapper finishes (fast path, settle, client abort, cancel or deadline expiry), add a FinalizationRegistry backstop for requests that never reach the wrapper, and cap the map size as a last resort. Also restore the doc-comment line on SettledHandler that the original commit dropped by accident.
…4792 adds it diegosouzapw#14792 threads the same correlationId: reqId option into the chat completions keepalive call, one line above extraHeaders. Putting this branch's copy on the same line lets the two changes merge into a single property instead of a duplicate key (TS1117) when diegosouzapw#14792 lands first.
Open
4 tasks
3 of 5 tasks
diegosouzapw
merged commit Sep 25, 2026
8c05ec4
into
diegosouzapw:release/v3.8.51
12 of 16 checks passed
HouMinXi
added a commit
to HouMinXi/OmniRoute
that referenced
this pull request
Sep 26, 2026
…quest proxy The Next.js App Router hands route handlers a Proxy around the inbound request for dynamic "auto" routes. Passing that proxy as the Request constructor input makes undici read #state on the proxy receiver; ECMAScript gives a Proxy no [[PrivateFieldValues]], so every request on /v1/chat/completions, /v1/messages and /v1/responses answered HTTP 500 with "TypeError: Cannot read private member #state ..." after diegosouzapw#14808 wired withDeadlineSignal into those routes. Rebuild the wrapped request field-by-field (url/method/headers/body/ signal/duplex) — property reads go through the proxy get trap safely. Adds a regression test that wraps the input in a Next-style trap and asserts the wrap survives, with bug-injection verified (reverting to the constructor-input form fails the test). Signed-off-by: Minxi Hou <houminxi@gmail.com>
6 tasks done
hugoharman
pushed a commit
to hugoharman/OmniRoute
that referenced
this pull request
Sep 27, 2026
withDeadlineSignal() wrapped the inbound request with
`new Request(request, { signal, headers })`. Next.js hands App Router
handlers that have no `dynamic` export a Proxy around the NextRequest
(proxyNextRequest), and the Request constructor reads the input's private
#state field, which cannot be reached through a Proxy. Every call to
/v1/chat/completions, /v1/messages and /v1/responses therefore failed with
"TypeError: Cannot read private member #state from an object whose class
did not declare it" and HTTP 500 (regression from diegosouzapw#14808).
Rebuild the request from its public accessors (url, method, headers,
redirect, body stream with duplex "half"), matching rebuildRequest() in
chatBodyAdmission. Adds regression tests with a Proxy-wrapped request and
a body-less GET.
4 of 5 tasks
2 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A handler that never resolves before first byte holds the early-keepalive slow path open forever, pinning the client spinner and concurrency slots. The slow path only ticks keepalives until the handler settles, while legitimate waits (queue, cooldown, park-and-resume, readiness) all fit under a derived bound. At 33 minutes the wrapper now emits the route error frame, closes the stream, aborts the handler, and warns once with correlation.
Related Issues
Related to #12685
Upstream census (2026-09-24)
Validation
npm run lintTests Added Or Updated
tests/unit/early-stream-keepalive.test.ts— 4 new tests: expiry emits the error frame, aborts the deadline controller, and warns once with correlation; resolve under the deadline forwards with no warn;slowPathDeadlineMs <= 0disables the deadline; client abort before the deadline keeps the existing abort path with no deadline warn.Coverage Notes
open-sse/utils/earlyStreamKeepalive.ts(timer start, expiry path, opt-out, abort interplay); targeted suites are green (keepalive 22/22); the full suite runs in CI.Reviewer Notes
src/app/api/v1/chat/completions/route.tsandtests/unit/early-stream-keepalive.test.tswith open fix(api): thread correlationId into chat completions early keepalive #14792. Merge fix(api): thread correlationId into chat completions early keepalive #14792 first, then rebase this one; the overlapping hunks are one-line threading additions on each side.changelog.d/fixes/14808-keepalive-deadline-last-resort.md.Maintainer rework (merge-batch 2026-09-24)
c8b0144).deadlineControllersByTokenis keyed by strings, so each request left an entry behind for the life of the process. Now:releaseDeadlineController();FinalizationRegistrybackstop covers requests that never reach the wrapper (non-streaming paths);SettledHandlercomment had lost one line (thestrictNullChecks: falsesentence). It is back.correlationId: reqIdmoved in the chat route (73f896b). It now sits on the same line where fix(api): thread correlationId into chat completions early keepalive #14792 adds it, so the two merge into a single property instead of a duplicate key (TS1117).release/v3.8.51, which includes fix(api): thread correlationId into chat completions early keepalive #14792. The combined state has been checked.typecheck:coreandcheck:open-sse-typecheckare clean./v1/messagesretention canary passes on this branch. Timeouts had to be raised for that run because the devbox was overloaded; the tip fails the same way under that load.check-file-sizeflags onlyopen-sse/executors/opencode.ts. That file is identical to the tip, so the failure is inherited from the base.