Skip to content

fix: preserve Gemini thought and content in same chunk - #5094

Open
CtriXin wants to merge 1 commit into
QuantumNous:mainfrom
CtriXin:fix/gemini-thought-output-split
Open

fix: preserve Gemini thought and content in same chunk#5094
CtriXin wants to merge 1 commit into
QuantumNous:mainfrom
CtriXin:fix/gemini-thought-output-split

Conversation

@CtriXin

@CtriXin CtriXin commented May 25, 2026

Copy link
Copy Markdown

Summary

  • split Gemini thought parts from normal text parts when building reasoning_content
  • allow thinking_delta and text_delta to both emit from the same OpenAI delta during Claude SSE conversion
  • add regression tests for both the Gemini relay and the OpenAI -> Claude stream conversion

Root cause

When a Gemini chunk contained both part.Thought=true and normal answer text, new-api currently mixed the two behaviors:

  1. relay/channel/gemini/relay-gemini.go treated the whole chunk as reasoning once any thought part appeared, so final answer text could be folded into reasoning_content.
  2. service/convert.go used mutually exclusive reasoning/content emission for Claude SSE, so a chunk with both fields could drop the final text_delta.

This change keeps reasoning and answer text separate all the way through the relay/conversion pipeline.

Tests

  • go test ./relay/channel/gemini ./service

Summary by CodeRabbit

  • Refactor

    • Enhanced Gemini integration response formatting to properly separate reasoning content from main answers
    • Improved Claude streaming response handling to correctly manage both reasoning and text content
  • Tests

    • Added test coverage validating proper separation of reasoning and answer content in Gemini responses
    • Added test coverage validating reasoning and text handling in Claude streaming responses

Review Change Stack

Agent-Model: gpt-5
Agent-Family: openai
Agent-Session: 019e556d-e697-7580-9ae0-e03202597043
Agent-Step: 0.0.1
@coderabbitai

coderabbitai Bot commented May 25, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

This PR improves reasoning content handling across LLM provider conversions. Gemini responses now separate "thought" parts into dedicated ReasoningContent fields in both streaming and non-streaming paths. OpenAI-to-Claude streaming is refactored to use reusable helper closures for thinking vs text delta emission, eliminating code duplication while maintaining state consistency.

Changes

Reasoning Content Separation

Layer / File(s) Summary
Gemini reasoning separation in non-streaming and streaming
relay/channel/gemini/relay-gemini.go, relay/channel/gemini/relay_gemini_reasoning_test.go
Non-streaming responseGeminiChat2OpenAI and streaming streamResponseGeminiChat2OpenAI now route Thought parts into a separate reasoning builder while regular content (text, code, images) routes to content. Reasoning is accumulated into choice.Message.ReasoningContent (non-stream) or choice.Delta.ReasoningContent (stream) with newline separation via shared writeSeparated helper. Tests validate reasoning/answer separation and multiple-part accumulation.
Claude streaming delta emission refactoring
service/convert.go, service/convert_stream_claude_test.go
StreamResponseOpenAI2Claude introduces emitThinkingDelta and emitTextDelta helper closures that centralize content block start/delta emission, message-type transition detection, and block advancement. These helpers replace duplicated inline branching in both initial and general delta paths, reducing redundancy while preserving tool-call and stop-reason behavior. Tests verify first-chunk and mid-stream reasoning/text separation.

Sequence Diagram(s)

sequenceDiagram
  participant GeminiResp as Gemini Response.Parts
  participant ReasoningBld as Reasoning Builder
  participant ContentBld as Content Builder
  participant OpenAIChoice as OpenAI Choice
  GeminiResp->>ReasoningBld: Thought part
  GeminiResp->>ContentBld: Text/Code/Image part
  ReasoningBld->>OpenAIChoice: Set ReasoningContent
  ContentBld->>OpenAIChoice: Set Message.Content
Loading
sequenceDiagram
  participant OpenAIDelta as OpenAI Delta (reasoning/text)
  participant EmitHelper as emitThinkingDelta/emitTextDelta
  participant SSEEmit as Emit SSE Event
  participant StateTrack as Update LastMessagesType
  OpenAIDelta->>EmitHelper: Delta received
  alt Message type changed
    EmitHelper->>SSEEmit: Emit content_block_start
  end
  EmitHelper->>SSEEmit: Emit content_block_delta
  EmitHelper->>StateTrack: Cache message type
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • QuantumNous/new-api#2854: Adjusts Claude SSE stop/advance/indexing behavior for thinking vs text blocks in the same StreamResponseOpenAI2Claude streaming path.
  • QuantumNous/new-api#2035: Fixes Claude thinking block representation (string*string and nil-safety) in OpenAI→Claude streaming conversion.
  • QuantumNous/new-api#4520: Changes Message.ReasoningContent representation to *string across Gemini/OpenAI/Claude handlers.

Suggested reviewers

  • seefs001
  • Calcium-Ion

Poem

🐰 Thoughts now flow in separate streams,
Reasoning apart from answers gleams,
Gemini's wisdom, Claude's reply,
Each provider gets its reasoning sigh,
Refactored helpers, duplicates gone,
The conversion logic marches on! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main fix: preserving both Gemini thought (reasoning) and content (answer text) together in the same chunk, which directly addresses the root cause described in the PR objectives.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@relay/channel/gemini/relay-gemini.go`:
- Around line 1095-1100: The helper writeSeparated currently adds a newline and
increments the appended counter even for empty strings; update the function(s)
(the writeSeparated closure at/around writeSeparated and the second identical
closure later) to first return early when s == "" (or len(s) == 0) so no newline
is written and appended is not incremented; ensure the logic still writes a
preceding '\n' only when *appended > 0 and only increments *appended after
actually writing a non-empty string.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: aedd4a2b-92db-4cb7-88f1-fac582d72767

📥 Commits

Reviewing files that changed from the base of the PR and between 583da45 and 94f2d42.

📒 Files selected for processing (4)
  • relay/channel/gemini/relay-gemini.go
  • relay/channel/gemini/relay_gemini_reasoning_test.go
  • service/convert.go
  • service/convert_stream_claude_test.go

Comment on lines +1095 to +1100
writeSeparated := func(builder *strings.Builder, appended *int, s string) {
if *appended > 0 {
builder.WriteByte('\n')
}
appended++
builder.WriteString(s)
*appended += 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Guard writeSeparated against empty strings.

Line 1095 and Line 1209 currently increment the append counter even when s == "". With empty text parts (e.g., Line 1129/1272 path), this can create phantom separators/newlines and formatting drift.

💡 Proposed fix
-			writeSeparated := func(builder *strings.Builder, appended *int, s string) {
+			writeSeparated := func(builder *strings.Builder, appended *int, s string) {
+				if s == "" {
+					return
+				}
 				if *appended > 0 {
 					builder.WriteByte('\n')
 				}
 				builder.WriteString(s)
 				*appended += 1
 			}
@@
-		writeSeparated := func(builder *strings.Builder, appended *int, s string) {
+		writeSeparated := func(builder *strings.Builder, appended *int, s string) {
+			if s == "" {
+				return
+			}
 			if *appended > 0 {
 				builder.WriteByte('\n')
 			}
 			builder.WriteString(s)
 			*appended += 1
 		}

Also applies to: 1209-1214

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/gemini/relay-gemini.go` around lines 1095 - 1100, The helper
writeSeparated currently adds a newline and increments the appended counter even
for empty strings; update the function(s) (the writeSeparated closure at/around
writeSeparated and the second identical closure later) to first return early
when s == "" (or len(s) == 0) so no newline is written and appended is not
incremented; ensure the logic still writes a preceding '\n' only when *appended
> 0 and only increments *appended after actually writing a non-empty string.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant