Skip to content

feat(translator): OpenAI SSE → Gemini SSE for /v1beta/models route - #4453

Merged
diegosouzapw merged 1 commit into
release/v3.8.32from
feat/port-pr-225-gemini-sse-v1beta
Jun 20, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.32from
feat/port-pr-225-gemini-sse-v1beta

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Summary

Ports decolua/9router#225 by @SteelMorgan.

The @google/genai SDK (Gemini CLI) always calls
/v1beta/models/{model}:streamGenerateContent?alt=sse for chat and expects
Gemini SSE chunks. The previous v1beta route forwarded raw OpenAI SSE from
handleChat, so the SDK crashed on the [DONE] sentinel that OpenAI SSE
terminates with (Gemini SSE has no sentinel — the stream just closes):

SyntaxError: Unexpected token 'D', "[DONE]" is not valid JSON

Changes

  • New translator open-sse/translator/response/openai-to-gemini-sse.ts

    • transformOpenAISSEToGeminiSSE(upstream, model) — TransformStream that
      re-emits each OpenAI delta as a Gemini-shape candidate, maps
      finish_reason → finishReason (STOP / MAX_TOKENS / SAFETY), attaches
      usageMetadata + modelVersion on the final chunk, and surfaces
      reasoning_content as { thought: true } parts for thinking models.
    • convertOpenAIResponseToGemini(response, model) — sibling JSON
      converter for the non-streaming :generateContent path.
    • OPENAI_TO_GEMINI_FINISH_REASON exported as a shared constant.
    • Chunk-boundary safe: the transformer buffers partial SSE lines across
      TextDecoder boundaries so it never JSON.parses a half-event.
  • Route rewrite src/app/api/v1beta/models/[...path]/route.ts

    • Streaming intent now derived from the URL action suffix
      (:streamGenerateContent vs :generateContent), the canonical Gemini
      API convention. generationConfig.stream is not a real Gemini field
      and the SDK never sets it.
    • Streaming branch pipes the upstream OpenAI SSE through
      transformOpenAISSEToGeminiSSE; non-streaming branch hands the JSON
      to convertOpenAIResponseToGemini.
    • Existing Zod validation, buildClientRawRequest plumbing, and
      sanitizeErrorMessage error path are preserved untouched.
  • Skipped from upstream: the first hunk of chore(deps): bump actions/cache from 4 to 5 #225 (filter nameless
    Responses-API tools in openai-to-responses.js) is not ported —
    OmniRoute's open-sse/translator/request/openai-responses.ts already has
    a much richer tool-conversion pipeline (web-search passthrough, MCP
    namespace flattening, local_shell mapping, etc.) and the
    empty-name-filter case is unreachable in practice in this codebase.

Test plan

  • node --import tsx/esm --test tests/unit/translator-openai-to-gemini-sse.test.ts — 11/11 pass
  • node --import tsx/esm --test tests/unit/v1beta-models-route.test.ts — 2/2 pass (route GET path unaffected)
  • npm run typecheck:core — clean
  • npx eslint on changed files — clean
  • node scripts/check/check-file-size.mjs — clean

TDD coverage

11 new tests in tests/unit/translator-openai-to-gemini-sse.test.ts:

  • finish-reason map covers all four canonical OpenAI values
  • per-chunk mapper: drops role-only deltas, maps content + reasoning,
    attaches usage/modelVersion on the final chunk, maps each
    finish_reason correctly
  • full SSE conversion drops the [DONE] sentinel and produces the exact
    Gemini chunk shape the SDK expects
  • chunk-boundary buffering test feeds an event split mid-string-literal
    and asserts the transformer re-assembles it
  • non-OK upstreams pass through unchanged
  • JSON converter: maps choice → candidate, passes through already-Gemini
    bodies, surfaces upstream errors with the right status code

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

…ls route

The @google/genai SDK (Gemini CLI) always calls
`:streamGenerateContent?alt=sse` for chat and expects Gemini SSE chunks
— no `[DONE]` sentinel, the stream just closes. The v1beta route was
forwarding OpenAI SSE from `handleChat` verbatim, so the SDK crashed
on the `[DONE]` line with:

    SyntaxError: Unexpected token 'D', "[DONE]" is not valid JSON

Add a new `transformOpenAISSEToGeminiSSE()` (and a sibling
`convertOpenAIResponseToGemini()` for the non-streaming JSON path) in
`open-sse/translator/response/openai-to-gemini-sse.ts`. The TransformStream
re-emits each OpenAI delta as a Gemini-shape candidate, maps
`finish_reason` → `finishReason` (STOP / MAX_TOKENS / SAFETY), attaches
`usageMetadata` + `modelVersion` on the final chunk, and surfaces
`reasoning_content` as `{ thought: true }` parts for thinking models.

Streaming intent is now derived from the URL action suffix
(`:streamGenerateContent` vs `:generateContent`), which is the canonical
Gemini API convention — `generationConfig.stream` is not a real Gemini
field and the SDK never sets it.

TDD: 11 new tests in `tests/unit/translator-openai-to-gemini-sse.test.ts`
covering the finish-reason map, per-chunk transform, full SSE conversion
(including chunk-boundary buffering and pass-through of non-OK upstreams),
and the non-streaming JSON conversion.

Ported from decolua/9router#225 by SteelMorgan.

Co-authored-by: SteelMorgan <steelmorgan33@gmail.com>
@diegosouzapw
diegosouzapw force-pushed the feat/port-pr-225-gemini-sse-v1beta branch from 0253bd3 to 8c7ed3d Compare June 20, 2026 23:26
@diegosouzapw
diegosouzapw merged commit 7abace5 into release/v3.8.32 Jun 20, 2026
3 checks passed
@diegosouzapw
diegosouzapw deleted the feat/port-pr-225-gemini-sse-v1beta branch June 21, 2026 12:33
tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant