fix(gemini): preserve thinking across relay formats - #1
Closed
karlorz wants to merge 27 commits into
Closed
Conversation
(cherry picked from commit c7d9971)
(cherry picked from commit 9077df9)
(cherry picked from commit 8b2884b)
(cherry picked from commit 5230827)
(cherry picked from commit 7371ea5)
(cherry picked from commit 3026132)
(cherry picked from commit 6eab6bb)
(cherry picked from commit a1f5842)
(cherry picked from commit e2a497a)
(cherry picked from commit 7086bf8)
(cherry picked from commit 7f8c29e)
(cherry picked from commit 95e7303)
Audit follow-up on the WebSocket relay port: - Billing: split call teardown into aborted vs settled. A client disconnect, upstream failure or terminal error event now settles on the observed usage (terminal-event usage, or output text counted via CountTextToken with the pre-consume prompt estimate) instead of refunding everything; refunds remain only when upstream never produced anything. Settlement bills a locked snapshot of usage so a late terminal event cannot race the reader. - Validation: response.create frames now run the shared helper.ValidateResponsesRequest (model + max_output_tokens bound), closing the WebSocket bypass of the billing-multiplier limits. The input-required check stays HTTP-only since incremental WS turns may omit input. - Token estimate: dto Input gains the function_call_output 'output' field so tool-result turns are counted and sensitive-checked instead of estimated at zero. - Robustness: 30s write deadlines on both legs; target state lock split from the write lock so a stuck write cannot block close paths; SetReadLimit (WEBSOCKET_MAX_MESSAGE_MB, default 16MB) on client and upstream; per-user concurrent session cap (RESPONSES_WEBSOCKET_MAX_PER_USER, default 8). - Cross-goroutine client read-deadline refresh kept, with the gorilla v1.5.3 passthrough justification documented in place. - Dial errors sanitize the upstream URL via SanitizeURLForLog. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit eb76195) (cherry picked from commit 5decd88)
Resolve the read limit lazily so it follows constant.MaxRequestBodyMB (populated by InitEnv after package init) unless WEBSOCKET_MAX_MESSAGE_MB overrides it, and document why permessage-deflate stays disabled: gorilla enforces SetReadLimit against the compressed wire length, so enabling it would reopen the decompression-bomb hole on the WebSocket side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit d6942d6) (cherry picked from commit 76bf0f0)
The stream field is stripped from response.create frames before parsing, so GenRelayInfoResponses recorded WebSocket calls as non-stream, which also hid their first-response time in the usage-log UI (the timing cell only shows it for stream calls). Set IsStream explicitly like the realtime relay does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit 954cb82) (cherry picked from commit b07e801)
(cherry picked from commit 521a3db)
(cherry picked from commit 1a3e2c4)
问题： 在非 passThrough 模式下，客户端发送的 reasoning_content: "" 经过 Go struct 反序列化再序列化后，因 string + omitempty 无法区分空串和 字段缺失，导致空的思考内容被静默丢弃。 根因： dto.Message.ReasoningContent 和 Message.Reasoning 使用 string（非指针） 加 omitempty，违反 AGENTS.md Rule 6（可选标量字段必须用指针类型）。 修复： 1. Message.ReasoningContent/Reasoning 类型从 string 改为 *string - nil = 字段缺失 → JSON 省略 - &"" = 显式空串 → JSON 保留 reasoning_content: "" 2. 新增 Message.GetReasoningContent() 辅助方法 3. 更新所有读写处：relay-openai, relay-claude, relay-gemini, ollama 4. 新增测试覆盖空串保留、字段省略、getter 回退逻辑 (cherry picked from commit 8ca1033)
(cherry picked from commit 465c5ed)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
8ca103342d, ReasoningContent/Reasoning -> *string) and fix:gemini to claude tool_use err QuantumNous/new-api#5041 (465c5eda, gemini->claude tool_use fix)fix(gemini): preserve thinking across relay formats(97e79ae): pointer GeminiThinkingConfig, Claude->Gemini thinking translation, thought/text/tool-call separation, Claude thinking blocks (stream + non-stream), reasoning_effort -> ThinkingLevelTest plan