Skip to content

fix: make passthrough stream format-aware for Responses SSE (#186) #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
diegosouzapw merged 1 commit into from
Mar 2, 2026
Merged

fix: make passthrough stream format-aware for Responses SSE (#186) #188

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 57 additions & 37 deletions open-sse/utils/stream.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,51 +136,71 @@ export function createSSEStream(options: any = {}) {
try {
let parsed = JSON.parse(trimmed.slice(5).trim());

// Sanitize: strip non-standard fields for OpenAI SDK compatibility
parsed = sanitizeStreamingChunk(parsed);
// Detect Responses SSE payloads (have a `type` field like "response.created",
// "response.output_item.added", etc.) and skip Chat Completions-specific
// sanitization to avoid corrupting the stream for Responses-native clients.
const isResponsesSSE =
parsed.type &&
typeof parsed.type === "string" &&
parsed.type.startsWith("response.");

if (isResponsesSSE) {
// Responses SSE: only extract usage, forward payload as-is
const extracted = extractUsage(parsed);
if (extracted) {
usage = extracted;
}
// Track content length from Responses format
if (parsed.delta && typeof parsed.delta === "string") {

@kilo-code-bot kilo-code-bot Bot Mar 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Incomplete content length tracking for Responses SSE format.

The code only tracks content length when parsed.delta exists (line 154-156), but Responses SSE events can contain content in other ways:

  • response.output_item.added with item.content[].text
  • response.content_part.added with part.text

Compare to Chat Completions handling at lines 178-180 which checks both delta.content and delta.reasoning_content. This could lead to underestimation of usage for Responses SSE format when content arrives in non-delta events.

totalContentLength += parsed.delta.length;
}
} else {
// Chat Completions: full sanitization pipeline
parsed = sanitizeStreamingChunk(parsed);

const idFixed = fixInvalidId(parsed);
const idFixed = fixInvalidId(parsed);

if (!hasValuableContent(parsed, FORMATS.OPENAI)) {
continue;
}
if (!hasValuableContent(parsed, FORMATS.OPENAI)) {
continue;
}
Comment on lines +157 to +165

Copilot AI Mar 2, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In PASSTHROUGH mode, any JSON data: payload that is not detected as Responses SSE is still treated as an OpenAI Chat Completions chunk (sanitized + hasValuableContent(..., FORMATS.OPENAI)), which will corrupt/drop same-format passthrough streams for other formats (e.g. Gemini/Claude) when sourceFormat === targetFormat. Consider gating the “Chat Completions” pipeline on an explicit Chat shape check (e.g., parsed.object === "chat.completion.chunk" / choices[0].delta), and otherwise forwarding the payload as-is (similar to the Responses branch).

Copilot uses AI. Check for mistakes.

const delta = parsed.choices?.[0]?.delta;
const delta = parsed.choices?.[0]?.delta;

// Extract <think> tags from streaming content
if (delta?.content && typeof delta.content === "string") {
const { content, thinking } = extractThinkingFromContent(delta.content);
delta.content = content;
if (thinking && !delta.reasoning_content) {
delta.reasoning_content = thinking;
// Extract <think> tags from streaming content
if (delta?.content && typeof delta.content === "string") {
const { content, thinking } = extractThinkingFromContent(delta.content);
delta.content = content;
if (thinking && !delta.reasoning_content) {
delta.reasoning_content = thinking;
}
}
}

const content = delta?.content || delta?.reasoning_content;
if (content && typeof content === "string") {
totalContentLength += content.length;
}
const content = delta?.content || delta?.reasoning_content;
if (content && typeof content === "string") {
totalContentLength += content.length;
}

const extracted = extractUsage(parsed);
if (extracted) {
usage = extracted;
}
const extracted = extractUsage(parsed);
if (extracted) {
usage = extracted;
}

const isFinishChunk = parsed.choices?.[0]?.finish_reason;
if (isFinishChunk && !hasValidUsage(parsed.usage)) {
const estimated = estimateUsage(body, totalContentLength, FORMATS.OPENAI);
parsed.usage = filterUsageForFormat(estimated, FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n`;
usage = estimated;
injectedUsage = true;
} else if (isFinishChunk && usage) {
const buffered = addBufferToUsage(usage);
parsed.usage = filterUsageForFormat(buffered, FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n`;
injectedUsage = true;
} else if (idFixed) {
output = `data: ${JSON.stringify(parsed)}\n`;
injectedUsage = true;
const isFinishChunk = parsed.choices?.[0]?.finish_reason;
if (isFinishChunk && !hasValidUsage(parsed.usage)) {
const estimated = estimateUsage(body, totalContentLength, FORMATS.OPENAI);
parsed.usage = filterUsageForFormat(estimated, FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n`;
usage = estimated;
injectedUsage = true;
} else if (isFinishChunk && usage) {
const buffered = addBufferToUsage(usage);
parsed.usage = filterUsageForFormat(buffered, FORMATS.OPENAI);
output = `data: ${JSON.stringify(parsed)}\n`;
injectedUsage = true;
} else if (idFixed) {
output = `data: ${JSON.stringify(parsed)}\n`;
injectedUsage = true;
}
}
Comment on lines +142 to 204

@gemini-code-assist gemini-code-assist Bot Mar 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic for extracting usage is duplicated in both the if and else branches. This can be refactored by moving the extractUsage call to before the conditional logic, as it's capable of handling different formats. This will reduce code duplication and improve maintainability.

                const isResponsesSSE =
                  parsed.type &&
                  typeof parsed.type === "string" &&
                  parsed.type.startsWith("response.");

                const extracted = extractUsage(parsed);
                if (extracted) {
                  usage = extracted;
                }

                if (isResponsesSSE) {
                  // Responses SSE: only track content length, forward payload as-is
                  if (parsed.delta && typeof parsed.delta === "string") {
                    totalContentLength += parsed.delta.length;
                  }
                } else {
                  // Chat Completions: full sanitization pipeline
                  parsed = sanitizeStreamingChunk(parsed);

                  const idFixed = fixInvalidId(parsed);

                  if (!hasValuableContent(parsed, FORMATS.OPENAI)) {
                    continue;
                  }

                  const delta = parsed.choices?.[0]?.delta;

                  // Extract <think> tags from streaming content
                  if (delta?.content && typeof delta.content === "string") {
                    const { content, thinking } = extractThinkingFromContent(delta.content);
                    delta.content = content;
                    if (thinking && !delta.reasoning_content) {
                      delta.reasoning_content = thinking;
                    }
                  }

                  const content = delta?.content || delta?.reasoning_content;
                  if (content && typeof content === "string") {
                    totalContentLength += content.length;
                  }

                  const isFinishChunk = parsed.choices?.[0]?.finish_reason;
                  if (isFinishChunk && !hasValidUsage(parsed.usage)) {
                    const estimated = estimateUsage(body, totalContentLength, FORMATS.OPENAI);
                    parsed.usage = filterUsageForFormat(estimated, FORMATS.OPENAI);
                    output = `data: ${JSON.stringify(parsed)}\n`;
                    usage = estimated;
                    injectedUsage = true;
                  } else if (isFinishChunk && usage) {
                    const buffered = addBufferToUsage(usage);
                    parsed.usage = filterUsageForFormat(buffered, FORMATS.OPENAI);
                    output = `data: ${JSON.stringify(parsed)}\n`;
                    injectedUsage = true;
                  } else if (idFixed) {
                    output = `data: ${JSON.stringify(parsed)}\n`;
                    injectedUsage = true;
                  }
                }

} catch {}

@kilo-code-bot kilo-code-bot Bot Mar 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Silent exception swallowing.

The catch {} block silently swallows all parsing exceptions without any logging. This makes debugging difficult in production when SSE parsing fails. Consider adding at least a console.warn or reqLogger error.

}
Expand Down
Loading