Skip to content

fix(stream): skip [DONE] terminator for Claude SSE clients - #2190

Merged
diegosouzapw merged 3 commits into
diegosouzapw:release/v3.8.0from
NomenAK:upstream/stream-claude-sse-done-gate
May 12, 2026
Merged

diegosouzapw merged 3 commits into
diegosouzapw:release/v3.8.0from
NomenAK:upstream/stream-claude-sse-done-gate

Conversation

@NomenAK

@NomenAK NomenAK commented May 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Anthropic SSE streams terminate naturally on the event: message_stop frame — they do not emit an OpenAI-style data: [DONE]\n\n terminator. Emitting one to Claude clients (Capy, claude-cli, Anthropic SDK) causes parser errors and observable UI artifacts (assistant text rendered in the wrong panel, followups looping with retries).

Gate the [DONE] emission so it only fires for OpenAI clients (Chat Completions / Responses). Symmetric to the existing clientExpectsResponsesStream gate that already silences [DONE] for the Responses API.

Related Issues

  • Closes #
  • Related to #

Motivation

Captured artifact for a /v1/messages BYOK call to Claude Opus through OmniRoute :

event: message_stop
data: {"type":"message_stop"}

: x-omniroute-cache-hit=false
: x-omniroute-latency-ms=498
: x-omniroute-tokens-in=28328
: x-omniroute-model=claude-opus-4-7

data: [DONE]   ← does not belong on Anthropic SSE

data: [DONE] is not part of the Anthropic Messages SSE spec (docs). When the client encounters it after message_stop, the strict parsers in the Anthropic SDKs raise Unexpected event "[DONE]" and the client either drops the response or misclassifies the trailing text as a stream error, surfacing as : "the assistant's reply is rendered in the wrong UI section / retries the request".

The OpenAI Responses API has the same constraint — it terminates on response.completed, not [DONE] — and the codebase already has a clientExpectsResponsesStream gate for that case. This PR is the symmetric gate for Claude clients.

Changes

open-sse/utils/stream.ts :

  • Introduce clientExpectsClaudeStream alongside the existing clientExpectsResponsesStream flag.
  • Both [DONE] emission sites (post-stream end and post-error) now check !clientExpectsResponsesStream && !clientExpectsClaudeStream before writing the terminator.

Validation

  • Manual : capture artifact pre/post — confirmed [DONE] no longer appears after message_stop on /v1/messages calls.
  • OpenAI Chat Completions endpoint still receives [DONE] (regression check).
  • OpenAI Responses endpoint still skips [DONE] (existing behavior preserved).

🤖 Generated with Claude Code

@NomenAK
NomenAK requested a review from diegosouzapw as a code owner May 12, 2026 09:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the SSE stream utility to prevent emitting the [DONE] terminator for Claude-formatted streams, which avoids parser errors in specific SDKs. The reviewer suggests consolidating the logic for emitting the terminator into a single variable, shouldEmitDoneTerminator, to improve code readability and simplify the conditional checks in multiple locations.

Comment thread open-sse/utils/stream.ts Outdated
await emitFinalSseMetadata(controller, usage);
doneSent = true;
if (!clientExpectsResponsesStream) {
if (!clientExpectsResponsesStream && !clientExpectsClaudeStream) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use the consolidated shouldEmitDoneTerminator variable here to simplify the condition.

Suggested change
if (!clientExpectsResponsesStream && !clientExpectsClaudeStream) {
if (shouldEmitDoneTerminator) {

Comment thread open-sse/utils/stream.ts Outdated
await emitFinalSseMetadata(controller, state?.usage as Record<string, unknown> | null);
doneSent = true;
if (!clientExpectsResponsesStream) {
if (!clientExpectsResponsesStream && !clientExpectsClaudeStream) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use the consolidated shouldEmitDoneTerminator variable here to simplify the condition.

Suggested change
if (!clientExpectsResponsesStream && !clientExpectsClaudeStream) {
if (shouldEmitDoneTerminator) {

Anthropic SSE streams terminate naturally on message_stop — there is
no `data: [DONE]` line. OmniRoute was unconditionally appending one
at end of every stream (gated only on OPENAI_RESPONSES), which:

- Capy (Anthropic SDK) sees an extra unparseable line after
  message_stop. Result: text content gets rendered in the "Thought"
  area of the UI, follow-up turns retry from a corrupt state.
- Native claude-cli, claude-code, and other Anthropic SDK consumers
  hit the same parse hiccup but tolerate it differently.

Add `clientExpectsClaudeStream` gate alongside the existing
`clientExpectsResponsesStream`. Both the passthrough and translate
finalization branches now check both flags before emitting `[DONE]`.

For Claude clients: stream ends after message_stop, with the
trailing `: x-omniroute-*` metadata comments. Standards-compliant
SSE — no terminator line needed.

Tested with Capy BYOK → Opus 4.7: first-turn thinking renders in the
correct UI section; followup turns no longer trigger a retry loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@NomenAK
NomenAK force-pushed the upstream/stream-claude-sse-done-gate branch from 7b06630 to 330bb5e Compare May 12, 2026 10:08
@NomenAK

NomenAK commented May 12, 2026

Copy link
Copy Markdown
Contributor Author

Addressed @gemini-code-assist's review — refactored to consolidate the gate into a single shouldEmitDoneTerminator variable used at both [DONE] emission sites. Commit history kept single via amend. CI rerunning.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Thanks for the update, NomenAK. Consolidating the gate into shouldEmitDoneTerminator is a much cleaner approach and significantly improves maintainability by ensuring consistent behavior across both emission sites. This looks good.

@diegosouzapw

Copy link
Copy Markdown
Owner

Thanks @NomenAK. I synced this branch with release/v3.8.0, added stream coverage proving Claude SSE clients do not receive a [DONE] terminator, and validated it locally with:\n\nnode --import tsx/esm --test tests/unit/stream-utils.test.ts\n\nI'll merge this through GitHub so the contribution is credited correctly.

@diegosouzapw
diegosouzapw merged commit 13f7ebc into diegosouzapw:release/v3.8.0 May 12, 2026
3 checks passed
HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…apw#2190)

Integrated into release/v3.8.0 after syncing the contributor branch and validating tests/unit/stream-utils.test.ts locally.
Poid-ZA pushed a commit to Poid-ZA/OmniRoute that referenced this pull request Aug 5, 2026
…apw#2190)

Integrated into release/v3.8.0 after syncing the contributor branch and validating tests/unit/stream-utils.test.ts locally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants