fix: map Responses reasoning stream to chat completion deltas - #2837
Conversation
WalkthroughAdds a DTO for reasoning-summary parts and three optional fields to response stream messages; extends the responses streaming handler to accumulate per-item reasoning-summary text and emit incremental deltas; relaxes request transformation to set Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Handler as ResponsesHandler
participant Store as ReasoningState
participant Emitter as StreamEmitter
Client->>Handler: send reasoning_summary_part / reasoning_summary_text.delta
Handler->>Store: key = responsesStreamIndexKey(itemID, indices)
Handler->>Store: prevText = reasoningSummaryTextByKey[key]
Handler->>Handler: delta = stringDeltaFromPrefix(prevText, newText)
Handler->>Emitter: sendReasoningSummaryDelta(delta)
Emitter-->>Client: stream chunk (reasoning_summary_text.delta)
Handler->>Store: reasoningSummaryTextByKey[key] = newText
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@relay/channel/openai/chat_via_responses.go`:
- Around line 272-302: The code currently emits summary deltas from both the
"response.reasoning_summary_text.delta" case and the
"response.reasoning_summary_part.added/done" case, causing duplicates; add a
guard to skip handling "response.reasoning_summary_text.delta" if the
corresponding summary part has already been processed: introduce a map (e.g.,
processedReasoningSummaryKeys map[string]bool) keyed the same way as
responsesStreamIndexKey(strings.TrimSpace(streamResp.ItemID),
streamResp.SummaryIndex); in the "response.reasoning_summary_part.added" /
".done" branch, after validating the key and updating reasoningSummaryTextByKey,
set processedReasoningSummaryKeys[key] = true; in the
"response.reasoning_summary_text.delta" branch, compute the same key and return
true/skip (do not call sendReasoningSummaryDelta) when
processedReasoningSummaryKeys[key] is true, otherwise proceed as before and
optionally update reasoningSummaryTextByKey.
In `@service/openaicompat/chat_to_responses.go`:
- Around line 349-353: The current block in chat_to_responses.go sets
out.Reasoning = &dto.Reasoning{Effort: req.ReasoningEffort, Summary: "detailed"}
whenever req.ReasoningEffort != "", which incorrectly requests a summary even
when effort == "none"; change the guard so you only set Summary (or create
out.Reasoning with Summary) when req.ReasoningEffort is non-empty and not equal
to "none" (e.g., check req.ReasoningEffort != "" && req.ReasoningEffort !=
"none"), ensuring dto.Reasoning.Summary is omitted when effort disables
reasoning.
🧹 Nitpick comments (1)
relay/channel/openai/chat_via_responses.go (1)
123-181: Deduplicate the identical reasoning senders.
BothsendReasoningDeltaandsendReasoningSummaryDeltaare identical. Consider extracting a single helper to reduce maintenance drift.♻️ Possible refactor
- sendReasoningDelta := func(delta string) bool { - if delta == "" { - return true - } - if !sendStartIfNeeded() { - return false - } - - usageText.WriteString(delta) - chunk := &dto.ChatCompletionsStreamResponse{ - Id: responseId, - Object: "chat.completion.chunk", - Created: createAt, - Model: model, - Choices: []dto.ChatCompletionsStreamResponseChoice{ - { - Index: 0, - Delta: dto.ChatCompletionsStreamResponseChoiceDelta{ - ReasoningContent: &delta, - }, - }, - }, - } - if err := helper.ObjectData(c, chunk); err != nil { - streamErr = types.NewOpenAIError(err, types.ErrorCodeBadResponse, http.StatusInternalServerError) - return false - } - return true - } - - sendReasoningSummaryDelta := func(delta string) bool { + sendReasoningLikeDelta := func(delta string) bool { if delta == "" { return true } if !sendStartIfNeeded() { return false } @@ if err := helper.ObjectData(c, chunk); err != nil { streamErr = types.NewOpenAIError(err, types.ErrorCodeBadResponse, http.StatusInternalServerError) return false } return true } + + sendReasoningDelta := func(delta string) bool { + return sendReasoningLikeDelta(delta) + } + + sendReasoningSummaryDelta := func(delta string) bool { + return sendReasoningLikeDelta(delta) + }
| case "response.reasoning_text.delta": | ||
| if !sendReasoningDelta(streamResp.Delta) { | ||
| return false | ||
| } | ||
|
|
||
| case "response.reasoning_text.done": | ||
|
|
||
| case "response.reasoning_summary_text.delta": | ||
| if !sendReasoningSummaryDelta(streamResp.Delta) { | ||
| return false | ||
| } | ||
|
|
||
| case "response.reasoning_summary_text.done": | ||
|
|
||
| case "response.reasoning_summary_part.added", "response.reasoning_summary_part.done": | ||
| key := responsesStreamIndexKey(strings.TrimSpace(streamResp.ItemID), streamResp.SummaryIndex) | ||
| if key == "" || streamResp.Part == nil { | ||
| break | ||
| } | ||
| // Only handle summary text parts, ignore other part types. | ||
| if streamResp.Part.Type != "" && streamResp.Part.Type != "summary_text" { | ||
| break | ||
| } | ||
| prev := reasoningSummaryTextByKey[key] | ||
| next := streamResp.Part.Text | ||
| delta := stringDeltaFromPrefix(prev, next) | ||
| reasoningSummaryTextByKey[key] = next | ||
| if !sendReasoningSummaryDelta(delta) { | ||
| return false | ||
| } | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
OpenAI Responses API streaming event types reasoning_summary_text reasoning_summary_part mutually exclusive
💡 Result:
They’re not mutually exclusive.
In Responses API streaming, response.reasoning_summary_part.* events tell you when a summary part object is created/finalized (e.g., a summary_text part being added/done). [1]
Separately, response.reasoning_summary_text.* events stream the text payload for that summary_text part as it’s generated (via .delta) and then provide the completed text (via .done). [2], [3]
So in a typical stream you can see reasoning_summary_part.added → one or more reasoning_summary_text.delta → reasoning_summary_text.done (and/or reasoning_summary_part.done). This mirrors the general pattern where a “part added/done” event exists alongside a “text delta/done” event for the text inside that part. [4]
Sources:
[1] https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning (reasoning_summary_part.added/done) (platform.openai.com)
[2] https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning (reasoning_summary_text.delta) (platform.openai.com)
[3] https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning (reasoning_summary_text.done) (platform.openai.com)
[4] https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning (content_part + output_text delta pattern) (platform.openai.com)
Citations:
- 1: https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning
- 2: https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning
- 3: https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning
- 4: https://platform.openai.com/docs/api-reference/responses-streaming/response/reasoning
🏁 Script executed:
# First, let's examine the file structure and context around the lines in question
find . -name "chat_via_responses.go" -type f | head -5Repository: QuantumNous/new-api
Length of output: 107
🏁 Script executed:
# Let's also check if there are any tests or documentation in the repo about this
find . -type f \( -name "*.md" -o -name "*test*.go" -o -name "*spec*" \) | grep -i "respond\|reason" | head -10Repository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# Read the relevant code section and surrounding context
cat -n ./relay/channel/openai/chat_via_responses.go | sed -n '250,320p'Repository: QuantumNous/new-api
Length of output: 2659
🏁 Script executed:
# Check if there's any deduplication or existing guards
rg -A 10 -B 10 "reasoningSummaryTextByKey" ./relay/channel/openai/chat_via_responses.go | head -80Repository: QuantumNous/new-api
Length of output: 1501
🏁 Script executed:
# Look at the sendReasoningSummaryDelta function to understand what it does
rg -A 15 "func.*sendReasoningSummaryDelta" ./relay/channel/openai/chat_via_responses.goRepository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# Search for sendReasoningSummaryDelta function definition
rg "sendReasoningSummaryDelta" ./relay/channel/openai/chat_via_responses.go -B 2 -A 10Repository: QuantumNous/new-api
Length of output: 1051
🏁 Script executed:
# Check the overall file to see if there's any deduplication map or logic
wc -l ./relay/channel/openai/chat_via_responses.goRepository: QuantumNous/new-api
Length of output: 111
🏁 Script executed:
# Look for the function definition anywhere in the file
grep -n "sendReasoningSummaryDelta" ./relay/channel/openai/chat_via_responses.goRepository: QuantumNous/new-api
Length of output: 225
Add a guard to prevent double-emitting summary deltas.
Both response.reasoning_summary_text.delta and response.reasoning_summary_part.added/done are sent together by the OpenAI Responses API, not mutually exclusive. The current code emits summary deltas from both event types, causing duplication. Once a reasoning_summary_part event is processed, skip subsequent reasoning_summary_text.delta events to avoid redundant emissions.
Suggested fix
+ case "response.reasoning_summary_part.added", "response.reasoning_summary_part.done":
+ reasoningSummaryTextByKey["__summary_part_mode__"] = "1"
key := responsesStreamIndexKey(strings.TrimSpace(streamResp.ItemID), streamResp.SummaryIndex) case "response.reasoning_summary_text.delta":
+ if _, ok := reasoningSummaryTextByKey["__summary_part_mode__"]; ok {
+ break
+ }
if !sendReasoningSummaryDelta(streamResp.Delta) {🤖 Prompt for AI Agents
In `@relay/channel/openai/chat_via_responses.go` around lines 272 - 302, The code
currently emits summary deltas from both the
"response.reasoning_summary_text.delta" case and the
"response.reasoning_summary_part.added/done" case, causing duplicates; add a
guard to skip handling "response.reasoning_summary_text.delta" if the
corresponding summary part has already been processed: introduce a map (e.g.,
processedReasoningSummaryKeys map[string]bool) keyed the same way as
responsesStreamIndexKey(strings.TrimSpace(streamResp.ItemID),
streamResp.SummaryIndex); in the "response.reasoning_summary_part.added" /
".done" branch, after validating the key and updating reasoningSummaryTextByKey,
set processedReasoningSummaryKeys[key] = true; in the
"response.reasoning_summary_text.delta" branch, compute the same key and return
true/skip (do not call sendReasoningSummaryDelta) when
processedReasoningSummaryKeys[key] is true, otherwise proceed as before and
optionally update reasoningSummaryTextByKey.
| if req.ReasoningEffort != "" { | ||
| out.Reasoning = &dto.Reasoning{ | ||
| Effort: req.ReasoningEffort, | ||
| Effort: req.ReasoningEffort, | ||
| Summary: "detailed", | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
OpenAI Responses API reasoning.effort reasoning.summary valid values constraints
💡 Result:
reasoning.effort (Responses API)
Type: string (in reasoning: { effort: ... })
Supported values (overall): none, minimal, low, medium, high, xhigh [1]
Key constraints / model limits:
reasoningis for gpt-5 and o-series reasoning models (not all models). [1]- gpt-5.1: default
none; supports onlynone,low,medium,high. [1] - Models before gpt-5.1: default
medium; do not supportnone. [1] - gpt-5-pro: defaults to and only supports
high. [1] - gpt-5.2-pro: supports only
medium,high,xhigh. [4][5] xhighis supported for models aftergpt-5.1-codex-max. [1]
reasoning.summary (Responses API)
Type: string (in reasoning: { summary: ... })
Valid values: auto, concise, detailed [1]
Constraints:
conciseis supported forcomputer-use-previewmodels and reasoning models aftergpt-5. [1]reasoning.generate_summaryis deprecated; usereasoning.summaryinstead. [1]
🏁 Script executed:
cat -n service/openaicompat/chat_to_responses.go | sed -n '340,360p'Repository: QuantumNous/new-api
Length of output: 577
🏁 Script executed:
# Search for ReasoningEffort validation and usage
rg "ReasoningEffort" service/openaicompat/ -B 2 -A 2Repository: QuantumNous/new-api
Length of output: 487
🏁 Script executed:
# Check if there's any validation of effort values in this file or related files
rg "none|minimal|low|medium|high|xhigh" service/openaicompat/chat_to_responses.go -iRepository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# Look at the function signature and docstring
head -n 20 service/openaicompat/chat_to_responses.goRepository: QuantumNous/new-api
Length of output: 420
🏁 Script executed:
# Check the dto.Reasoning struct to understand its design
rg "type Reasoning struct" . -A 10Repository: QuantumNous/new-api
Length of output: 611
🏁 Script executed:
# Look for test cases related to reasoning effort
fd "test" service/openaicompat/ --type f | xargs rg -l "ReasoningEffort|Reasoning" 2>/dev/null | head -5Repository: QuantumNous/new-api
Length of output: 220
🏁 Script executed:
# Check test file for reasoning-related tests
cat service/openaicompat/chat_to_responses_test.go 2>/dev/null | head -100Repository: QuantumNous/new-api
Length of output: 45
🏁 Script executed:
# Look at responses_to_chat.go to understand the reverse flow
rg "Reasoning|ReasoningEffort" service/openaicompat/responses_to_chat.go -B 3 -A 3Repository: QuantumNous/new-api
Length of output: 406
Guard against setting Reasoning.Summary when effort disables reasoning.
When req.ReasoningEffort == "none", the code still sets Summary: "detailed", which contradicts the OpenAI Responses API semantics—effort: "none" disables reasoning output, so requesting a summary is meaningless and may fail validation.
Update the condition to exclude "none":
Suggested fix
- if req.ReasoningEffort != "" {
+ if req.ReasoningEffort != "" && strings.ToLower(req.ReasoningEffort) != "none" {
out.Reasoning = &dto.Reasoning{
Effort: req.ReasoningEffort,
Summary: "detailed",
}
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if req.ReasoningEffort != "" { | |
| out.Reasoning = &dto.Reasoning{ | |
| Effort: req.ReasoningEffort, | |
| Effort: req.ReasoningEffort, | |
| Summary: "detailed", | |
| } | |
| if req.ReasoningEffort != "" && strings.ToLower(req.ReasoningEffort) != "none" { | |
| out.Reasoning = &dto.Reasoning{ | |
| Effort: req.ReasoningEffort, | |
| Summary: "detailed", | |
| } | |
| } |
🤖 Prompt for AI Agents
In `@service/openaicompat/chat_to_responses.go` around lines 349 - 353, The
current block in chat_to_responses.go sets out.Reasoning =
&dto.Reasoning{Effort: req.ReasoningEffort, Summary: "detailed"} whenever
req.ReasoningEffort != "", which incorrectly requests a summary even when effort
== "none"; change the guard so you only set Summary (or create out.Reasoning
with Summary) when req.ReasoningEffort is non-empty and not equal to "none"
(e.g., check req.ReasoningEffort != "" && req.ReasoningEffort != "none"),
ensuring dto.Reasoning.Summary is omitted when effort disables reasoning.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@relay/channel/openai/chat_via_responses.go`:
- Around line 163-192: The sendReasoningSummaryDelta function is appending a
newline to every streaming delta via appendTrailingNewline(delta), which
corrupts reconstructed reasoning content; remove the per-delta
appendTrailingNewline call and emit the raw delta instead (keep
usageText.WriteString(delta) as-is), and add logic to append a trailing newline
only when the reasoning summary is complete (e.g., introduce a
sendFinalReasoningSummary or a final boolean on sendReasoningSummaryDelta to
append appendTrailingNewline only for the final chunk) so streaming chunks are
not mutated.
- Around line 297-312: Add handling for the
"response.reasoning_summary_text.delta" event so incremental deltas are applied
to the buffer rather than ignored: in the switch that currently processes
"response.reasoning_summary_part.added" and ".done", add a case for
"response.reasoning_summary_text.delta" that uses
responsesStreamIndexKey(strings.TrimSpace(streamResp.ItemID),
streamResp.SummaryIndex) to get the key, skips if key=="" or
streamResp.Part==nil and only processes when streamResp.Part.Type is "" or
"summary_text", then treat streamResp.Part.Text as an incremental fragment to
append to the existing buffer stored in reasoningSummaryTextByKey[key] (instead
of overwriting), compute the visible delta via stringDeltaFromPrefix(prev,
newCombinedText) and call sendReasoningSummaryDelta(delta); ensure the ".added"
case still initializes the entry and ".done" finalizes by replacing/confirming
full part.text as now implemented.
🧹 Nitpick comments (1)
relay/channel/openai/chat_via_responses.go (1)
133-161: Consider removing or documenting commented-out code.This large block of commented code (~30 lines here, plus lines 283-295) adds noise. If
sendReasoningDeltaand the related event handlers are planned for future use, consider adding a brief TODO comment explaining the intent. Otherwise, remove the dead code to improve readability.
fix: default summary = detailed fix ReasoningContent fix ReasoningContent fix ReasoningContent fix ReasoningContent Revert "fix ReasoningContent" This reverts commit 45a88f7. fix ReasoningContent fix ReasoningContent
fafea9d to
7e13a01
Compare
…reasoning fix: map Responses reasoning stream to chat completion deltas
2 participants
Summary by CodeRabbit