fix(bifrost): finalize SSE relay usage after stream

[KooshaPari]

Summary

• keep Bifrost streaming requests tied to the returned SSE body lifecycle
• record relay usage after stream close/cancel/error instead of immediately after upstream headers
• keep the timeout active for streaming bodies and record timeout-aborted streams as 504 errors
• preserve the existing non-streaming behavior

Fixes #4595.

Verification

• node --import tsx --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit tests/unit/api/v1/bifrost-sidecar.test.ts (5/5)
• npm run typecheck:core -- --pretty false
• npm run check:file-size
• npm run check:env-doc-sync
• npm run check:public-creds

[gemini-code-assist]

Code Review

This pull request updates the Bifrost relay route to defer recording usage metrics for streaming responses until the stream has completed, errored, or been cancelled, while non-streaming responses continue to be recorded immediately. It introduces a finalizeReadableStream helper to track the stream lifecycle and adds a unit test to verify this behavior. The reviewer identified a potential issue in finalizeReadableStream where calling controller.close() or controller.error() on an already cancelled stream could throw a TypeError, and suggested checking the finalized state and wrapping controller calls in try-catch blocks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

When the stream is cancelled (e.g., due to client disconnect or timeout), cancel() is called on the ReadableStream, which invokes finalizeOnce(reason) and sets finalized = true.\n\nHowever, if a reader.read() operation was pending in pull(), it will reject with an error when the stream is cancelled. This triggers the catch (error) block in pull(), which then calls controller.error(error).\n\nCalling controller.error() on an already cancelled/closed stream can throw a TypeError (e.g., "Controller is already closed"). Since this happens inside an asynchronous pull handler, it can result in unhandled promise rejections or noisy error logs in production.\n\nTo prevent this, we should check if the stream has already been finalized before calling controller.close() or controller.error(), and wrap those calls in a try...catch block to safely ignore any errors if the stream state has already transitioned.

    async pull(controller) {\n      try {\n        const { done, value } = await reader.read();\n        if (finalized) return;\n        if (done) {\n          finalizeOnce();\n          try {\n            controller.close();\n          } catch {\n            // Ignore if already closed/errored/cancelled\n          }\n          return;\n        }\n        controller.enqueue(value);\n      } catch (error) {\n        if (finalized) return;\n        finalizeOnce(error);\n        try {\n          controller.error(error);\n        } catch {\n          // Ignore if already closed/errored/cancelled\n        }\n      }\n    },

[diegosouzapw]

Obrigado, @KooshaPari! 🙌 Reconstruí sobre o tip da release depois do #4593 (mesma área de bifrost/route.ts): reconciliei os dois — mantive o relaySecurity.ts extraído pelo #4593 e adicionei a finalização de stream deste PR (registro de usage no close/cancel/error + timeout→504). Os dois conjuntos de testes coexistem (8 verdes). Mergeando.