Skip to content

fix(sse): publish the stream-failure handlers before the pipeline pumps (#17290) - #17329

Closed
diegosouzapw wants to merge 2 commits into
release/v3.8.53from
fix/release-v3.8.53-stream-failure-late-bound-handler
Closed

diegosouzapw wants to merge 2 commits into
release/v3.8.53from
fix/release-v3.8.53-stream-failure-late-bound-handler

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Summary

Root cause of the "flaky" tests/unit/perplexity-web-stream-error-boundary.test.ts (#17290), which failed 16 of 21 unit-shard attempts across the release/v3.8.53 PRs on 2026-10-20 and held three PRs after two re-runs each.

It is not a flaky test — it is a real race introduced by the chatCore decomposition in #14725 (2026-10-06):

  • runStreamingTail creates onPipelineStreamError / onClientDisconnectFinalize and hands them back to chatCore through carry, i.e. only after the tail has resolved.
  • chatCore's stream controller reads both handlers through getters at event time (() => onPipelineStreamError), so from "pipeline assembled and pumping" (assembleStreamingPipeline) until "tail returned" the getter answers null.
  • An upstream failure pumped through the pipeline during the tail's remaining awaits (emitRequestGamificationEvent, runPluginOnResponseHook, the Response construction) finds no handler: handleError falls through to clearPendingRequest, nothing calls handleStreamFailure, so the request is never finalized as a stream failure — no onStreamFailure for the caller (combo fallback), no stream_pipeline_error usage record.

Fast machines rarely hit the window; hosted CI runners hit it most of the time. Before #14725 the handler lived in the same closure as the getter, so it was visible the moment it was assigned.

Fix

  • open-sse/handlers/chatCore.ts: the two let bindings become a small holder object (lateBoundStreamHandlers) that the stream controller's getters read and that is passed to the tail as a dep.
  • open-sse/handlers/chatCore/streamingTail.ts: publishes both handlers into the holder right after creating them — before assembleStreamingPipeline starts pumping. The carry write-back in chatCore stays as a no-op safety net for early-return paths.

No behaviour change on the happy path; the failure path simply finalizes again, as it did before #14725.

Tests

  • New tests/unit/chatcore/late-bound-stream-handlers.test.ts: pins the holder wiring on both sides, asserts the publish happens before assembleStreamingPipeline( in the tail, and drives createChatStreamController to prove a pipeline error raised once the handler is installed reaches it.
  • Existing tests/unit/chatcore/{streaming-tail,leaf-deps-contract}.test.ts still pass (10/10 locally).
  • check:open-sse-typecheck 0 errors; eslint/prettier clean.
  • Failing-then-passing evidence (Hard Rule fix(ci): add environment for npm token access #18): the Perplexity boundary test fails on the base in CI (jobs 118797529115, 118795350932, 118800986841 — all the downstream pipeline must finalize the stream failure); this PR's own unit shard is the passing half.

Follow-ups

⚠️ base-red inherited: #17219 (Release-Green sweep reds on the base tip, unrelated to this change)

…ps (#17290)

Since #14725 split handleChatCore into leaves, the streaming tail returned
onPipelineStreamError / onClientDisconnectFinalize through `carry`, i.e. only after
runStreamingTail had resolved. chatCore's stream controller reads both through
getters at event time, so between "pipeline assembled and pumping" and "tail
returned" the getter answered null: an upstream failure pumped through the
pipeline during the tail's remaining awaits found no handler, and the request was
never finalized as a stream failure (no onStreamFailure for the caller, no
stream_pipeline_error record). Fast machines rarely hit the window; hosted CI
runners hit it on 16 of 21 unit-shard attempts on 2026-10-20 through
tests/unit/perplexity-web-stream-error-boundary.test.ts.

chatCore now owns a small holder shared by reference with the tail; the tail
publishes both handlers into it the moment it creates them, before
assembleStreamingPipeline starts pumping, restoring the pre-#14725 closure
timing. The carry write-back stays as a no-op safety net for early-return paths.

Regression test: tests/unit/chatcore/late-bound-stream-handlers.test.ts pins the
holder wiring on both sides, the publish-before-assemble order, and that a
pipeline error raised once the handler is installed reaches it.
@github-actions

Copy link
Copy Markdown
Contributor

CI Coverage Report

  • Coverage job: cancelled
  • PR test policy: success

Coverage artifact was not available for this run.

@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #17226 by @pacocartones, which was opened first (2026-10-20 01:36Z) and fixes the same race the same way: the streaming tail registers onPipelineStreamError / onClientDisconnectFinalize with chatCore before assembleStreamingPipeline starts pumping, instead of handing them back through carry after the tail resolved.

Two things learned since this PR was opened, both recorded on #17290:

  • The failure is deterministic on Node 24.22.0 (the fixture fails 3/3 there and passes on 24.21/24.20/24.18); hosted runners resolving node-version: 24 to whichever patch is cached is what made it look flaky. That is also why the .113 (24.18) never reproduced it in 15 runs.
  • With fix(streaming): register finalizers before eager pipeline starts #17226 applied on the release tip the fixture passes 8/8 on 24.22.0.

The evidence collected on this branch stays useful for #17226: the fixture passed on both hosted runs here (Unit Tests (2/8) of run 39487234913 and Unit Tests fast-path (2/4) of run 39487234900). The regression test from this PR (tests/unit/chatcore/late-bound-stream-handlers.test.ts, pinning that the handlers are published before the pipeline is assembled) can follow #17226 as a small separate PR if wanted.

@diegosouzapw
diegosouzapw deleted the fix/release-v3.8.53-stream-failure-late-bound-handler branch October 21, 2026 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant