Skip to content

fix(response): unify JSON response projection across non-streaming and SSE→JSON paths - #2348

Open
warelik wants to merge 2 commits into
decolua:masterfrom
warelik:fix/unified-response-projection
Open

warelik wants to merge 2 commits into
decolua:masterfrom
warelik:fix/unified-response-projection

Conversation

@warelik

@warelik warelik commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #2347
Related: #2046, #1396

Summary

Unifies non-streaming and forced-SSE→JSON response projection on upstream v0.5.30. Provider responses are normalized to the existing OpenAI Chat Completion pivot, then projected into the client's OpenAI, Claude, Gemini-family, Ollama, or Responses envelope. Tool calls, reasoning, finish reasons, usage, and client-native response structure are preserved instead of leaking raw OpenAI choices[] or dropping Gemini function calls.

The streaming diagnostic path emits only counts and HMAC digests. It now performs no body scans unless debug logging is explicitly enabled. Passthrough terminal handling also forwards exactly one real upstream [DONE] and synthesizes one only when upstream omitted it.

Changes

  • completionProjector.js: shared OpenAI→client projection and Responses→OpenAI normalization.
  • nonStreamingHandler.js / sseToJsonHandler.js: use the shared projection contract; remove duplicate/dead projection helpers.
  • toolSemanticsTrace.js: safe request/provider/client tool semantics tracing.
  • src/sse/utils/logger.js: export explicit debug state so tracing is not activated merely because a logger has a debug() method.
  • sseDoneTracker.js: pure terminal tracker; first upstream [DONE] is forwarded, duplicates are suppressed, and flush only synthesizes a missing terminal.
  • stream.js, chatCore.js, streamingHandler.js: tracing and projection wiring. Source/target trace context is carried on the existing request logger, avoiding a positional-signature conflict with passthrough-specific features.

Files

10 files: four handlers, completionProjector.js, completionProjectorSelfCheck.mjs, toolSemanticsTrace.js, sseDoneTracker.js, stream.js, and src/sse/utils/logger.js.

Verification

  • completionProjectorSelfCheck.mjs: 22/22
  • production build: 126/126
  • built-server smoke: OpenAI and Claude streaming/non-streaming passed; exactly one OpenAI [DONE] and one Claude message_stop
  • standalone git apply --check / git diff --check: clean on 9845a17

@warelik

warelik commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Code-review follow-up: logToolSemantics now early-returns when debug logging is inactive, so the hot chat/stream path skips the full request/response body scan and HMAC digest work unless the trace will actually be emitted. 20/20 self-check still passes.

@warelik
warelik force-pushed the fix/unified-response-projection branch from 59e157e to fc96ef8 Compare July 13, 2026 08:12
diegosouzapw added a commit to diegosouzapw/OmniRoute that referenced this pull request Jul 17, 2026
…nvelope (#7255)

* fix(sse): project non-streaming JSON back to the Gemini/Antigravity envelope

The streaming and non-streaming response paths disagreed on how a response is
projected back into a non-OpenAI client's wire format.

Streaming goes through the translator registry, where the
FORMATS.OPENAI -> FORMATS.ANTIGRAVITY translator
(open-sse/translator/response/openai-to-antigravity.ts) projects each OpenAI
chunk into the `{ response: { candidates: [...] } }` envelope, mapping
tool_calls to `functionCall` parts and reasoning to `thought` parts.

The non-streaming path uses translateNonStreamingResponse() instead. Its
"Phase 3: translate back to client source format" step only special-cased
FORMATS.CLAUDE — every other non-OpenAI client format fell through and returned
the raw OpenAI chat.completion intermediate. A Gemini/Antigravity client issuing
a non-streaming request therefore received `choices[]`/`tool_calls` instead of
`candidates[]`/`functionCall`: the client's parser sees no candidates and the
function calls are effectively dropped, so tool-calling silently breaks on the
JSON path while working over SSE.

Adds convertOpenAINonStreamingToGeminiFamily() and wires it into Phase 3 for
FORMATS.GEMINI / FORMATS.ANTIGRAVITY, mirroring the shape the streaming
translator already emits so both paths agree. Tool-call `arguments` are parsed
through a non-throwing helper: a provider emitting truncated JSON degrades that
call's args to `{}` rather than raising an uncaught SyntaxError in the shared
response hot path (matching the streaming translator's behaviour).

Scoped deliberately narrow: only the Gemini-family projection gap proven by the
failing test is closed. The Ollama/Responses projections and the SSE terminal
tracker from the upstream change are not ported — OmniRoute has no OLLAMA format
in FORMATS, and its Responses/[DONE] handling already lives in
nonStreamingSse.ts + the registry.

Co-authored-by: W ARELIK <warelik@users.noreply.github.com>
Inspired-by: decolua/9router#2348

* chore(changelog): fragment for #7255

---------

Co-authored-by: W ARELIK <warelik@users.noreply.github.com>
@zierocode

Copy link
Copy Markdown
Contributor

Fresh gap probe on 2026-07-23 at fc96ef808bb42617601f9d77113ed69decb2bcd6:

  • the PR's dependency-free self-check still passes 22/22;
  • Chat Completion to Responses projection omits response.model;
  • it preserves token totals but drops input_tokens_details.cached_tokens and output_tokens_details.reasoning_tokens.

Reproduction:

node --input-type=module -e 'import { projectCompletionToClientFormat } from "./open-sse/translator/response/completionProjector.js"; const out=projectCompletionToClientFormat({id:"chatcmpl-gap",model:"resolved-model",choices:[{message:{role:"assistant",content:"ok"},finish_reason:"stop"}],usage:{prompt_tokens:12,prompt_tokens_details:{cached_tokens:3},completion_tokens:5,completion_tokens_details:{reasoning_tokens:2},total_tokens:17}},"openai-responses"); console.log(JSON.stringify({model:out.model,usage:out.usage,terminalShape:out.status}))'

Observed:

{"usage":{"input_tokens":12,"output_tokens":5,"total_tokens":17},"terminalShape":"completed"}

Expected additions:

{"model":"resolved-model","usage":{"input_tokens":12,"input_tokens_details":{"cached_tokens":3},"output_tokens":5,"output_tokens_details":{"reasoning_tokens":2},"total_tokens":17}}

PR #2453 already owns native Responses SSE-to-JSON preservation for these fields. This gap is specifically the unified Chat-to-client projection implemented here, so no competing projection PR is needed if it can be covered in this branch.

HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…nvelope (diegosouzapw#7255)

* fix(sse): project non-streaming JSON back to the Gemini/Antigravity envelope

The streaming and non-streaming response paths disagreed on how a response is
projected back into a non-OpenAI client's wire format.

Streaming goes through the translator registry, where the
FORMATS.OPENAI -> FORMATS.ANTIGRAVITY translator
(open-sse/translator/response/openai-to-antigravity.ts) projects each OpenAI
chunk into the `{ response: { candidates: [...] } }` envelope, mapping
tool_calls to `functionCall` parts and reasoning to `thought` parts.

The non-streaming path uses translateNonStreamingResponse() instead. Its
"Phase 3: translate back to client source format" step only special-cased
FORMATS.CLAUDE — every other non-OpenAI client format fell through and returned
the raw OpenAI chat.completion intermediate. A Gemini/Antigravity client issuing
a non-streaming request therefore received `choices[]`/`tool_calls` instead of
`candidates[]`/`functionCall`: the client's parser sees no candidates and the
function calls are effectively dropped, so tool-calling silently breaks on the
JSON path while working over SSE.

Adds convertOpenAINonStreamingToGeminiFamily() and wires it into Phase 3 for
FORMATS.GEMINI / FORMATS.ANTIGRAVITY, mirroring the shape the streaming
translator already emits so both paths agree. Tool-call `arguments` are parsed
through a non-throwing helper: a provider emitting truncated JSON degrades that
call's args to `{}` rather than raising an uncaught SyntaxError in the shared
response hot path (matching the streaming translator's behaviour).

Scoped deliberately narrow: only the Gemini-family projection gap proven by the
failing test is closed. The Ollama/Responses projections and the SSE terminal
tracker from the upstream change are not ported — OmniRoute has no OLLAMA format
in FORMATS, and its Responses/[DONE] handling already lives in
nonStreamingSse.ts + the registry.

Co-authored-by: W ARELIK <warelik@users.noreply.github.com>
Inspired-by: decolua/9router#2348

* chore(changelog): fragment for diegosouzapw#7255

---------

Co-authored-by: W ARELIK <warelik@users.noreply.github.com>
golamrabbi696 added a commit to golamrabbi696/EzRouter that referenced this pull request Aug 12, 2026
…d SSE-to-JSON paths

Rebase unified client-format projection onto upstream v0.5.30. Gate semantic tracing on explicit debug state, carry trace context without competing with passthrough decloaking, and forward exactly one stream terminal marker.\n\nVerification: completionProjectorSelfCheck 22/22; production build 126/126; OpenAI and Claude streaming/non-streaming smoke passed.
@warelik
warelik force-pushed the fix/unified-response-projection branch from fc96ef8 to a69d489 Compare August 22, 2026 05:12
Client-keyed Codex branch and pre-projection reasoning strip
broke neighboring Claude/Responses paths. Branch on targetFormat,
project thoughts first, restore SSE errors and cache usage, drop
HMAC tracing from this leak fix.
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…nvelope (diegosouzapw#7255)

* fix(sse): project non-streaming JSON back to the Gemini/Antigravity envelope

The streaming and non-streaming response paths disagreed on how a response is
projected back into a non-OpenAI client's wire format.

Streaming goes through the translator registry, where the
FORMATS.OPENAI -> FORMATS.ANTIGRAVITY translator
(open-sse/translator/response/openai-to-antigravity.ts) projects each OpenAI
chunk into the `{ response: { candidates: [...] } }` envelope, mapping
tool_calls to `functionCall` parts and reasoning to `thought` parts.

The non-streaming path uses translateNonStreamingResponse() instead. Its
"Phase 3: translate back to client source format" step only special-cased
FORMATS.CLAUDE — every other non-OpenAI client format fell through and returned
the raw OpenAI chat.completion intermediate. A Gemini/Antigravity client issuing
a non-streaming request therefore received `choices[]`/`tool_calls` instead of
`candidates[]`/`functionCall`: the client's parser sees no candidates and the
function calls are effectively dropped, so tool-calling silently breaks on the
JSON path while working over SSE.

Adds convertOpenAINonStreamingToGeminiFamily() and wires it into Phase 3 for
FORMATS.GEMINI / FORMATS.ANTIGRAVITY, mirroring the shape the streaming
translator already emits so both paths agree. Tool-call `arguments` are parsed
through a non-throwing helper: a provider emitting truncated JSON degrades that
call's args to `{}` rather than raising an uncaught SyntaxError in the shared
response hot path (matching the streaming translator's behaviour).

Scoped deliberately narrow: only the Gemini-family projection gap proven by the
failing test is closed. The Ollama/Responses projections and the SSE terminal
tracker from the upstream change are not ported — OmniRoute has no OLLAMA format
in FORMATS, and its Responses/[DONE] handling already lives in
nonStreamingSse.ts + the registry.

Co-authored-by: W ARELIK <warelik@users.noreply.github.com>
Inspired-by: decolua/9router#2348

* chore(changelog): fragment for diegosouzapw#7255

---------

Co-authored-by: W ARELIK <warelik@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Non-streaming and SSE→JSON responses leak OpenAI format to non-OpenAI clients (function calls dropped for Gemini, Claude, Ollama)

2 participants