feat(responses): bridge /v1/responses to chat-completions adaptors (Claude first) - #3290
feat(responses): bridge /v1/responses to chat-completions adaptors (Claude first)#32900-don wants to merge 1 commit into
Conversation
71eca10 to
17790e2
Compare
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds routing to forward selected /v1/responses requests through /v1/chat/completions (streaming and non-streaming), updates the Responses streaming DTO shape, and implements conversion helpers plus a stateful chat→responses streaming adapter to translate chat SSE chunks into Responses SSE events. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client
participant RespHandler as ResponsesHandler
participant ConvSvc as ConversionService
participant Adaptor as ChannelAdaptor
participant Upstream as ChatUpstream
participant StreamAd as StreamAdapter
Client->>RespHandler: POST /v1/responses
RespHandler->>RespHandler: decide routing (model / flags)
alt routed via chat/completions
RespHandler->>ConvSvc: ResponsesRequestToChatCompletionsRequest
ConvSvc-->>RespHandler: GeneralOpenAIRequest / error
RespHandler->>Adaptor: POST /v1/chat/completions (converted)
Adaptor->>Upstream: upstream request
alt upstream SSE
Upstream-->>RespHandler: SSE chunks
RespHandler->>StreamAd: HandleChatChunk per chunk
StreamAd-->>RespHandler: Responses SSE events
RespHandler-->>Client: SSE events (Responses format)
else upstream JSON
Upstream-->>RespHandler: JSON response
RespHandler->>ConvSvc: ChatCompletionsResponseToResponsesResponse
ConvSvc-->>RespHandler: Responses JSON
RespHandler-->>Client: Responses JSON
end
else native responses flow
RespHandler->>Adaptor: POST /v1/responses (normal)
Adaptor-->>Client: Responses reply
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 858-868: The block that marshals responsesResp uses json.Marshal
directly; replace that call with common.Marshal (i.e., responseData, err =
common.Marshal(responsesResp)) and keep the existing error handling (return
types.NewError(err, types.ErrorCodeBadResponseBody)); also remove any direct
dependency on encoding/json from this file's imports if it's only used for this
marshal so the code follows the common/json.go wrapper requirement. Ensure the
change is applied in the same case branch that converts with
ResponseClaude2OpenAI and service.ChatCompletionsResponseToResponsesResponse so
behavior and error mapping remain identical.
In `@relay/responses_via_chat_completions.go`:
- Around line 160-207: oaiChatStreamToResponsesHandler currently only
accumulates content via contentBuilder and ignores streaming tool calls; add a
slice variable (e.g., toolCalls := []dto.ToolCall{}) near where usage/model are
declared, append any entries from choice.Delta.ToolCalls inside the chunk
processing loop (using the existing for _, choice := range chunk.Choices block),
and then include these accumulated toolCalls when constructing the final
synthetic response (the same place where contentBuilder.String(), model, usage,
and finishReason are used) so streamed tool calls are preserved and returned.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f4403ad5-9767-4cf8-a691-40ed598a3734
📒 Files selected for processing (9)
dto/openai_response.gorelay/channel/claude/adaptor.gorelay/channel/claude/relay-claude.gorelay/responses_handler.gorelay/responses_via_chat_completions.goservice/openai_chat_responses_compat.goservice/openaicompat/chat_stream_to_responses_stream.goservice/openaicompat/responses_to_chat.goweb/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
💤 Files with no reviewable changes (1)
- web/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
d02513a to
5397d8d
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
relay/channel/claude/relay-claude.go (1)
851-857: Pre-existing:json.Marshalused instead ofcommon.Marshal.Line 854 uses
json.Marshal(openaiResponse)directly, which violates the coding guideline. While this is pre-existing code not introduced by this PR, consider fixing it for consistency since you're modifying this function.♻️ Proposed fix
case types.RelayFormatOpenAI: openaiResponse := ResponseClaude2OpenAI(&claudeResponse) openaiResponse.Usage = *claudeInfo.Usage - responseData, err = json.Marshal(openaiResponse) + responseData, err = common.Marshal(openaiResponse) if err != nil { return types.NewError(err, types.ErrorCodeBadResponseBody) }As per coding guidelines: "All JSON marshal/unmarshal operations MUST use wrapper functions in
common/json.go."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@relay/channel/claude/relay-claude.go` around lines 851 - 857, Replace the direct call to json.Marshal when encoding openaiResponse with the project's wrapper common.Marshal; locate the block that creates openaiResponse via ResponseClaude2OpenAI(&claudeResponse) and assigns openaiResponse.Usage = *claudeInfo.Usage, then swap json.Marshal(openaiResponse) for common.Marshal(openaiResponse) and preserve the existing error handling (return types.NewError(err, types.ErrorCodeBadResponseBody)) so behavior is unchanged aside from using the mandated wrapper.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/responses_via_chat_completions.go`:
- Around line 174-225: After the bufio.NewScanner read loop, check scanner.Err()
and handle any non-nil error instead of silently ignoring it: immediately after
the for scanner.Scan() { ... } block, call if err := scanner.Err(); err != nil {
/* propagate / return the error, or set a failure on the surrounding handler and
log it with context (include resp.Body or request id) */ }. Update the
surrounding function that reads from resp.Body to propagate that error (or
return a non-nil error) so truncated or I/O-failed streams are reported instead
of returning partial content; place this check directly after the scanner loop
that populates contentBuilder, toolCalls, model, usage, and finishReason.
In `@service/openaicompat/chat_stream_to_responses_stream.go`:
- Around line 110-119: The fallback generation for missing call.ID (the block
that sets callID when call.Index is nil and s.ToolCallOrder is empty) can
produce collisions because fmt.Sprintf("call_%d", idx) resets per chunk; change
it to use a monotonically-increasing, concurrency-safe counter on the stream
struct (e.g., add s.callFallbackCounter uint64 and increment with
atomic.AddUint64) and format callID from that counter (or combine the counter
with idx/chunk identifier) instead of just idx; update references in the call-ID
assignment logic (where callID, call.Index, s.ToolCallOrder and idx are used) to
use the new counter for unique IDs.
---
Nitpick comments:
In `@relay/channel/claude/relay-claude.go`:
- Around line 851-857: Replace the direct call to json.Marshal when encoding
openaiResponse with the project's wrapper common.Marshal; locate the block that
creates openaiResponse via ResponseClaude2OpenAI(&claudeResponse) and assigns
openaiResponse.Usage = *claudeInfo.Usage, then swap json.Marshal(openaiResponse)
for common.Marshal(openaiResponse) and preserve the existing error handling
(return types.NewError(err, types.ErrorCodeBadResponseBody)) so behavior is
unchanged aside from using the mandated wrapper.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2fa6af5b-707d-45ed-b9e8-ac24f991157b
📒 Files selected for processing (9)
dto/openai_response.gorelay/channel/claude/adaptor.gorelay/channel/claude/relay-claude.gorelay/responses_handler.gorelay/responses_via_chat_completions.goservice/openai_chat_responses_compat.goservice/openaicompat/chat_stream_to_responses_stream.goservice/openaicompat/responses_to_chat.goweb/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
💤 Files with no reviewable changes (1)
- web/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
✅ Files skipped from review due to trivial changes (1)
- service/openai_chat_responses_compat.go
🚧 Files skipped from review as they are similar to previous changes (1)
- relay/channel/claude/adaptor.go
fab4b53 to
4ba4a03
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@service/openaicompat/chat_stream_to_responses_stream.go`:
- Around line 88-102: The code is reserving an output slot for streaming
reasoning (setting ReasoningOutputIndex and incrementing NextOutputIndex when
emitting response.reasoning_summary_text.delta) but FinalEvents() and
buildFinalOutput() never produce the corresponding
response.output_item.added/done or a concrete output item, causing output_index
mismatches; fix by either (A) stop consuming an output slot for reasoning deltas
(do not set ReasoningOutputIndex/advance NextOutputIndex when emitting
response.reasoning_summary_text.delta) so subsequent items keep consistent
indices, or (B) ensure the reserved slot is materialized by emitting a matching
response.output_item.added (and response.output_item.done) and including the
reasoning item in buildFinalOutput() so FinalEvents() produces the same
output_index; update all places using ReasoningOutputIndex/NextOutputIndex
(including the emission site for response.reasoning_summary_text.delta and the
logic in FinalEvents() and buildFinalOutput()) to keep indices consistent.
In `@service/openaicompat/responses_to_chat.go`:
- Around line 261-289: Non-function tools appended into
dto.ToolCallRequest.Custom are never consumed by the Claude relay (which only
reads tool.Function.* and textRequest.WebSearchOptions), so built-in tools like
web_search_preview and file_search are dropped; update the Claude relay to
inspect dto.ToolCallRequest.Custom when Type != "function", unmarshal the Custom
JSON and map known tool types (e.g., "web_search_preview", "file_search") into
the fields the relay sends upstream (for example populate
textRequest.WebSearchOptions or the equivalent request fields it expects), or
add a clear unmarshalling/mapping layer in relay-claude to handle these custom
tool payloads so non-function tools are forwarded instead of ignored.
- Around line 148-157: flushToolCalls currently constructs dto.Message with
Content: nil which downstream relay rewrites to "..." and injects synthetic
text; instead create the assistant message with a non-nil empty content before
attaching tool calls so no synthetic text is added. In other words, inside
flushToolCalls (and where dto.Message is constructed for assistant tool_call
turns) ensure msg.Content is initialized to an explicit empty text content
(e.g., an empty string content object) prior to calling
msg.SetToolCalls(pendingToolCalls) and appending to messages so the assistant
tool-call turn has empty, not nil, content.
- Around line 455-472: The current code only appends an assistant "message"
output when choice.Message.IsStringContent() is true, which drops text when the
message is multipart; change the logic in the block handling text content
(around choice.Message.IsStringContent()) to call and flatten
choice.Message.ParseContent() instead of gating on IsStringContent(), iterate
parsed parts to collect any text parts, and append a single dto.ResponsesOutput
with Type "message" and Content entries for each text part (preserving existing
fields like ID, Status, Role and using output_text for text parts) so multipart
text is emitted the same as a plain string.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 810f0ca6-b901-414c-8281-019a9b5b16f8
📒 Files selected for processing (9)
dto/openai_response.gorelay/channel/claude/adaptor.gorelay/channel/claude/relay-claude.gorelay/responses_handler.gorelay/responses_via_chat_completions.goservice/openai_chat_responses_compat.goservice/openaicompat/chat_stream_to_responses_stream.goservice/openaicompat/responses_to_chat.goweb/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
💤 Files with no reviewable changes (1)
- web/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
✅ Files skipped from review due to trivial changes (1)
- relay/responses_via_chat_completions.go
🚧 Files skipped from review as they are similar to previous changes (4)
- relay/channel/claude/adaptor.go
- relay/responses_handler.go
- service/openai_chat_responses_compat.go
- dto/openai_response.go
4ba4a03 to
b165f6e
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
service/openaicompat/chat_stream_to_responses_stream.go (1)
87-102:⚠️ Potential issue | 🟠 MajorReasoning deltas consume an output slot that never exists in
response.completed.output.
HandleChatChunk()reservesNextOutputIndexfor reasoning, butFinalEvents()/buildFinalOutput()never materialize a matching output item. If reasoning arrives before text or tool calls, later streamedoutput_indexvalues no longer line up with the finalresponse.completed.outputarray.Also applies to: 181-191, 446-484
service/openaicompat/responses_to_chat.go (3)
148-157:⚠️ Potential issue | 🟠 MajorDon't emit
nilcontent for assistant tool-call turns.This builds assistant tool-call messages with
Content: nil. Downstream,relay/channel/claude/relay-claude.gorewrites nil assistant content to"...", so replayedfunction_callturns gain synthetic text and the prompt changes across tool iterations.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@service/openaicompat/responses_to_chat.go` around lines 148 - 157, The flushToolCalls helper is creating assistant messages with Content: nil which downstream code rewrites to "..." and alters prompts; update the flushToolCalls implementation so the dto.Message created for tool-call turns uses a non-nil empty content (e.g., Content: "" or an explicit empty content object) instead of nil before calling msg.SetToolCalls(pendingToolCalls) and appending to messages, ensuring the assistant tool-call turn has a stable, non-nil Content field.
455-473:⚠️ Potential issue | 🟠 MajorFlatten multipart assistant content before emitting
output_text.This only emits a message output when
choice.Message.IsStringContent()is true. If an adaptor returns text as multipart content,/v1/responsescomes back without the assistantmessageoutput even though the text is available inParseContent().🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@service/openaicompat/responses_to_chat.go` around lines 455 - 473, The current block only emits an assistant message when choice.Message.IsStringContent() is true, so multipart text parts are dropped; update the logic in the responses construction (the block building dto.ResponsesOutput for Type:"message") to flatten/parse multipart content via choice.Message.ParseContent() (or equivalent API on choice.Message) and extract any text parts, concatenating or emitting them as a single output_text entry when present; keep using dto.ResponsesOutput and dto.ResponsesOutputContent, preserve existing fields (ID, Status, Role), and ensure you still prefer StringContent() when available but fall back to parsed/flattened parts to avoid losing multipart text.
261-289:⚠️ Potential issue | 🟠 MajorClaude still drops non-function tools from this conversion.
These tools are packed into
dto.ToolCallRequest.Custom, but the Claude path passes the converted chat request straight intoConvertOpenAIRequestand only readstool.Function.*/WebSearchOptions.web_search_preview,file_search, and similar built-ins never make it upstream.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@relay/responses_via_chat_completions.go`:
- Around line 79-95: The current branch uses the upstream Content-Type
(isStream) to choose oaiChatStreamToResponsesHandler which fully buffers the
chat stream; change this so buffering is only used when the original caller did
NOT request streaming: detect the caller's stream flag (e.g. the request/Info
struct field that represents stream=true or the request query/header) and only
call oaiChatStreamToResponsesHandler when that flag is false; otherwise route
the upstream stream into the incremental Responses stream translator/state
machine (i.e., use the incremental chunk-to-Responses handler instead of
buffering). Update the isStream branch logic around isStream,
oaiChatStreamToResponsesHandler and oaiChatToResponsesHandler (and apply the
same gating for the other affected block at lines 155-268).
---
Duplicate comments:
In `@service/openaicompat/responses_to_chat.go`:
- Around line 148-157: The flushToolCalls helper is creating assistant messages
with Content: nil which downstream code rewrites to "..." and alters prompts;
update the flushToolCalls implementation so the dto.Message created for
tool-call turns uses a non-nil empty content (e.g., Content: "" or an explicit
empty content object) instead of nil before calling
msg.SetToolCalls(pendingToolCalls) and appending to messages, ensuring the
assistant tool-call turn has a stable, non-nil Content field.
- Around line 455-473: The current block only emits an assistant message when
choice.Message.IsStringContent() is true, so multipart text parts are dropped;
update the logic in the responses construction (the block building
dto.ResponsesOutput for Type:"message") to flatten/parse multipart content via
choice.Message.ParseContent() (or equivalent API on choice.Message) and extract
any text parts, concatenating or emitting them as a single output_text entry
when present; keep using dto.ResponsesOutput and dto.ResponsesOutputContent,
preserve existing fields (ID, Status, Role), and ensure you still prefer
StringContent() when available but fall back to parsed/flattened parts to avoid
losing multipart text.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 702db28f-e355-4b5f-904d-fc93fcb716c1
📒 Files selected for processing (9)
dto/openai_response.gorelay/channel/claude/adaptor.gorelay/channel/claude/relay-claude.gorelay/responses_handler.gorelay/responses_via_chat_completions.goservice/openai_chat_responses_compat.goservice/openaicompat/chat_stream_to_responses_stream.goservice/openaicompat/responses_to_chat.goweb/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
💤 Files with no reviewable changes (1)
- web/src/components/table/usage-logs/UsageLogsColumnDefs.jsx
🚧 Files skipped from review as they are similar to previous changes (4)
- relay/channel/claude/adaptor.go
- relay/responses_handler.go
- service/openai_chat_responses_compat.go
- dto/openai_response.go
c1138f1 to
d20aa03
Compare
This comment was marked as spam.
This comment was marked as spam.
7783a5d to
311978b
Compare
766d821 to
1ad7774
Compare
6c3c8a3 to
38c1634
Compare
c8b6bb2 to
0baffcd
Compare
4435141 to
653f3ab
Compare
6982634 to
60e6f53
Compare
18724e1 to
fa671a8
Compare
230d558 to
b897aeb
Compare
b897aeb to
b056f53
Compare
db08aff to
328938b
Compare
9888e46 to
91790b7
Compare
…chat completions conversion Adds a compatibility shim that lets clients using the OpenAI Responses API hit channels backed by adaptors that only speak Chat Completions. The shim converts the request, streams the chat-style response, and emits Responses API events back to the client. Updated for upstream's ResponsesOutput.Arguments switch to json.RawMessage (was string when this PR was authored).
91790b7 to
ba86160
Compare
|
Closing: no longer maintaining these against upstream. |
📝 变更描述 / Description
Channels backed by adaptors that only speak Chat Completions can now serve
/v1/responsesrequests. The shim converts the Responses API request to Chat Completions, routes through the existing adaptor pipeline (Claude first), and converts the response back to Responses API events. Any future adaptor opts in by adding a 3-lineConvertOpenAIResponsesRequestmethod.Two reusable converters live in
service/openaicompat/:ResponsesRequestToChatCompletionsRequest(responses_to_chat.go): developer-role mapping, function_call merging, content type detection, PromptCacheKey passthrough.ChatToResponsesStreamState(chat_stream_to_responses_stream.go): streaming state machine that emits proper Responses API SSE events from a Chat Completions stream.The Claude adaptor chains these with its existing
ConvertOpenAIRequest, reusing all upstream conversion logic.bufio.Scannererrors are now checked after stream completion so truncated streams surface as errors instead of silently returning partial data, and tool call IDs use a UUID-based fallback instead of the loop index to prevent collisions.Updated for upstream's
ResponsesOutput.Argumentsswitch tojson.RawMessage(wasstringwhen the PR was first authored).Why not existing PRs?
vs #2892 (494 lines, Claude-only): reimplements logic already in
RequestOpenAI2ClaudeMessageand does naive SSE passthrough without converting to Responses API event format. vs #2817 (1039 lines): similar approach but ships its own stream state and converter inside the Claude adaptor. This PR adopts the best ideas from both (adaptor-level pattern from #2817, tool-call bugfix from #2892) and extracts a reusable converter so non-Claude adaptors can opt in cheaply.🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
📸 运行证明 / Proof of Work
Backend:
go build -o /dev/null .passes.Frontend:
cd web/default && bun run buildpasses (no frontend changes in this PR).Functional verification:
/v1/responsesthrough a Claude channel: SSE events match Responses API format/v1/responsesthrough a Claude channel: correct response structurefunction_call+function_call_outputitems converts cleanly