fix(sse-heartbeat): shape-aware keepalives keep streams alive through stricter proxies

[NomenAK]

What

Extends the existing sseHeartbeat.ts module with a shape option so synthetic keepalives are emitted in a format the downstream client (and any intermediate proxy) recognizes as activity:

• Anthropic-out clients: event: ping\ndata: {}\n\n (mirrors Anthropic native ping events)
• OpenAI Chat Completions clients: empty-delta chat.completion.chunk line
• OpenAI Responses API clients: data: {"type":"response.in_progress"}\n\n
• Comment (existing behavior): kept as the default so all current consumers are byte-identical to today.

Also bundles a separable change: lowers STREAM_IDLE_TIMEOUT_MS default from 600_000 → 300_000 (still env-overridable via STREAM_IDLE_TIMEOUT_MS). Happy to split this into a follow-up PR if preferred.

Why

Observed BYOK streaming sessions disconnecting mid-extended-thinking when the upstream provider (Anthropic Claude with thinking enabled on heavy contexts) goes silent for 60-120s. Some reverse proxies in front of the consumer count only data: SSE lines as keepalive activity — not SSE comments (: keepalive). The current comment-style heartbeat fires correctly but isn't visible to those proxies, so the connection drops before the upstream resumes streaming.

The 10-minute STREAM_IDLE_TIMEOUT_MS default also turned out to be dead weight in practice: downstream proxies abandon well before we hit it, so we never get a useful failure signal. 5 minutes is still a safety ceiling but actionable.

How

• open-sse/utils/sseHeartbeat.ts: adds HEARTBEAT_SHAPES enum, shapeForClientFormat() helper mapping client format → shape, and an internal buildHeartbeatPayload() builder. Existing intervalMs / signal parameters preserved; new intervalMs <= 0 returns a passthrough TransformStream (disables heartbeat cleanly).
• open-sse/handlers/chatCore.ts:4276: passes intervalMs: SSE_HEARTBEAT_INTERVAL_MS and shape: shapeForClientFormat(clientResponseFormat) to the existing heartbeat call site.
• open-sse/handlers/responsesHandler.ts:74: passes shape: HEARTBEAT_SHAPES.OPENAI_RESPONSES_IN_PROGRESS.
• open-sse/config/constants.ts: exports new SSE_HEARTBEAT_INTERVAL_MS (default 15s, env-overridable; set to 0 to disable).
• src/shared/utils/runtimeTimeouts.ts: surfaces sseHeartbeatIntervalMs in UpstreamTimeoutConfig and tightens DEFAULT_STREAM_IDLE_TIMEOUT_MS to 300_000.
• Strip-logic non-collision verified: open-sse/utils/stream.ts:903-905,1515 matches /^event:\s*keepalive\b/i. None of the new heartbeat shapes produce that literal — a regression test in tests/unit/sse-heartbeat.test.ts covers every shape against this regex.

Tests

• tests/unit/sse-heartbeat.test.ts: extended from 2 to 9 tests. Existing 2 still byte-identical (back-compat).
• tests/unit/sse-heartbeat-integration.test.ts: new file, 3 time-based integration tests piping a fake upstream through the heartbeat and asserting downstream sees correct-shape output within 200ms, plus regression vs the strip regex.
• tests/unit/runtime-timeouts.test.ts: extended for the new config field + the lowered idle default.
• npm run typecheck:core: clean.

Notes

Happy to split the STREAM_IDLE_TIMEOUT_MS default change into a separate PR if you'd prefer keeping this PR strictly about the heartbeat shape — the env var remains the authoritative override either way. Same for the shape default: kept as "comment" (current behavior) so no surprise for existing consumers; new shapes are opt-in via the wiring sites.

Happy to revise the API shape, the format strings, or the wiring approach if you'd prefer a different layout.

[gemini-code-assist]

Code Review

This pull request introduces configurable SSE heartbeat shapes (Anthropic ping, OpenAI chunk, and OpenAI responses) to prevent connection timeouts during long upstream processing. It also lowers the default stream idle timeout to 5 minutes and adds environment variable support for heartbeat intervals. Feedback identifies a redundant heartbeat transform in responsesHandler.ts and suggests optimizations in sseHeartbeat.ts, such as using the built-in identity transform and hoisting the TextEncoder to the module level to reduce allocations.

[gemini-code-assist]

This adds a second heartbeat transform to the stream. The handleChatCore function already applies an SSE heartbeat transform at the end of its pipeline (see chatCore.ts:4276). Since handleChatCore is called with convertedBody.stream = true, it will always attach a heartbeat if SSE_HEARTBEAT_INTERVAL_MS is enabled. This redundancy results in multiple keepalive events being sent to the client and may cause issues if the first heartbeat is processed by transformStream in an unexpected way.

[NomenAK]

Thanks for catching this! You're right there's a redundancy. The first heartbeat wrap (in chatCore.ts:4276) emits shape comment when called from this path — because responsesHandler calls handleChatCore with clientRawRequest: null, so clientResponseFormat falls back to the default. The second wrap here (responsesHandler.ts) overrides with OPENAI_RESPONSES_IN_PROGRESS to match the post-transform shape that downstream clients actually parse.

Two ways to clean this up:

1. Thread clientResponseFormat: "openai-responses" through convertedBody so chatCore's heartbeat picks the right shape directly, then remove this second wrap.
2. Add an omitHeartbeat: true option to handleChatCore and keep this wrap as the single authoritative emitter.

I have a slight lean toward option 1 since it keeps a single point of control, but it touches the convertedBody contract slightly. Happy to do either — which would you prefer?

[gemini-code-assist]

The TransformStream constructor creates an identity (passthrough) transform by default if no transformer is provided. You can simplify this block to use the built-in identity transform, which is more idiomatic and potentially more efficient.

    return new TransformStream<Uint8Array, Uint8Array>();

[gemini-code-assist]

TextEncoder is a stateless global object. Creating a new instance for every heartbeat transform is unnecessary. It is better to move this to the top level of the module to be reused across all streams, reducing object allocation overhead.

[diegosouzapw]

Hey @NomenAK! Thank you for the shape-aware heartbeat work — exactly what was needed for strict proxies during long thinking phases.

Integrated as b9db934. One small reviewer adjustment: I reverted the DEFAULT_STREAM_IDLE_TIMEOUT_MS default change (300_000 → kept at 600_000) since some slow-thinking Claude/extended-thinking models on heavy contexts still rely on the 10-minute legacy ceiling. The heartbeat itself + the env-overridable STREAM_IDLE_TIMEOUT_MS give operators the knob without breaking that path. The rest of the diff (shape enum, helpers, integration tests) is preserved as-is. Thanks again!