Skip to content

Feat: responses to anthropic - #4819

Open
clansty wants to merge 6 commits into
QuantumNous:mainfrom
clansty:feat/responses-to-anthropic
Open

Feat: responses to anthropic#4819
clansty wants to merge 6 commits into
QuantumNous:mainfrom
clansty:feat/responses-to-anthropic

Conversation

@clansty

@clansty clansty commented May 12, 2026

Copy link
Copy Markdown
Contributor

Codex CLI 等客户端使用 OpenAI Responses API。当入口是 Responses API 出口是 Anthropic 的时候,目前 new-api 要么是返回 not implemented,要么必须走 Responses → Chat Completions → Claude 两跳,沿途会丢失:

  • thinking / reasoning(被压成 reasoning_content 字符串,签名丢失)
  • custom tool(freeform grammar 工具,例如 Codex 的 apply_patch
  • Responses 协议的 output_item / summary_text / encrypted_content 等结构性事件

本 PR 在 relay/channel/claude/ 内实现 Responses ↔ Anthropic Messages 的直接双向转换器,目标是让 Codex 这类客户端在配 Claude 渠道时完全无感

📝 变更描述 / Description

  • 完整的 Responses ↔ Anthropic Messages 字段映射,尽可能不丢失信息
  • 由于 Anthropic 没有 freeform 的 custom tool(codex 的 apply patch 是一个),会转换为 {"input":"内容"} 这样的 json 格式,并且移除描述中的 This is a FREEFORM tool 等字样
  • 自动移除那些只有 OpenAI 上游支持的服务端工具
  • 完整思考签名支持,签名和类型 JSON+base64 放在正确的区块里

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

  • Closes # (如有)

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
    有一个现成的 PR 在,但是那个是经过 Chat Completions 中转的,我觉得不算重复
  • [ ] Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

(请在此粘贴截图、关键日志或测试报告,以证明变更生效)
image
image

Summary by CodeRabbit

  • New Features

    • Added OpenAI Responses API format support with streaming and non-streaming response modes.
    • Implemented encrypted reasoning and redacted thinking capabilities in responses.
    • Enhanced tool handling with improved custom tool conversions between Claude and OpenAI formats.
  • Improvements

    • Updated response data structures to align with OpenAI Responses API specification.
    • Improved request conversion logic for better protocol compatibility.

Review Change Stack

clansty added 6 commits May 13, 2026 03:20
Implement a direct bidirectional converter between OpenAI Responses API
and Anthropic Messages API so a /v1/responses request can target a Claude
channel without round-tripping through Chat Completions, which would lose
reasoning encrypted_content / thinking signature semantics.

Request path (Responses -> Claude):
- Reasoning effort maps to Anthropic adaptive thinking (effort=minimal
  disables it; summary=none sets display=omitted).
- Reasoning items decode encrypted_content back into Claude thinking or
  redacted_thinking blocks with the original signature, so multi-turn
  signature verification at Anthropic succeeds.
- Tools/tool_choice/system/stop/metadata are mapped directly.
- previous_response_id, text.format=json_schema, allowed_tools, and most
  server-side tools are rejected with 400 rather than silently degraded;
  allowed_tools would otherwise broaden the permitted tool set.

Response path (Claude -> Responses):
- ClaudeResponsesStreamState turns Anthropic message_start /
  content_block_* / message_delta / message_stop events into the
  corresponding Responses SSE sequence (response.created, in_progress,
  output_item.added, content_part.added, output_text.delta/done,
  reasoning_summary_text.delta/done, function_call_arguments.delta/done,
  output_item.done, completed/incomplete) with a monotonic sequence_number.
- Server-side Claude blocks (server_tool_use, web_search_tool_result,
  code_execution_tool_result) are skipped so they cannot leak into the
  output as type="unknown".
- stop_reason=max_tokens maps to response.incomplete with
  incomplete_details.reason=max_output_tokens (matches OpenAI schema).

encrypted_content envelope:
- Anthropic signatures and redacted_thinking data are wrapped in an
  na1.<base64url(JSON{v,t,s|d})> envelope that the gateway round-trips on
  behalf of the client. Strict version/kind/non-empty/size validation is
  applied on decode; bare strings without the prefix are still accepted
  as legacy raw Anthropic signatures.
- This is NOT cryptographic encryption: signature/data is recoverable by
  anything that can read encrypted_content. AEAD with a server key is a
  follow-up if Anthropic signature leakage to third parties is a concern.

Tests cover stream sequencing, signature/redacted round-trip, max_tokens
incomplete, assistant turn ordering enforcement, allowed_tools rejection,
malformed envelope rejection, interleaved thinking/text and server tool
skip.
…for Codex

Extend the direct Responses↔Anthropic Messages converter so OpenAI Codex
(and any client using OpenAI Responses API custom tools / built-in server
tools) can target a Claude channel without losing semantics or hitting
upstream 400s.

Built-in server tool stripping:
- web_search / web_search_preview / file_search / code_interpreter /
  computer_use / image_generation / mcp are silently dropped before the
  request reaches Anthropic. Anthropic either has no equivalent, has a
  paid equivalent with reversed semantics (Codex's web_search with
  external_web_access=false MUST NOT trigger Anthropic's paid web_search),
  or requires separate enablement. Models simply do not see these tools,
  matching the copilot-gateway behaviour.
- tool_choice is sanitised against the surviving tool set: pointing at a
  stripped tool unsets tool_choice; "required" with zero surviving tools
  unsets it; "none" is preserved. Without this, Anthropic 400s on every
  Codex request that includes a built-in tool and a specific tool_choice.

Custom tool transparent round-trip:
- type=custom is downgraded to a function tool with a {"input": string}
  schema; lark/regex grammars are injected into the tool description with
  an 8 KiB cap so very large grammars cannot blow past Anthropic limits.
- ConvertResponsesRequestToClaude now also returns the set of custom tool
  names so the response path can restore tool_use blocks back to
  custom_tool_call items (with raw string input) instead of leaking
  function_call shape to clients that key off `type`. The set is
  unconditionally written to gin.Context, including nil, so retries and
  pass-through paths cannot inherit a stale set from a prior turn.
- Duplicate tool names (function/function, custom/custom or function/custom)
  are rejected with 400 because Anthropic tool_use only echoes name, not
  type, so the response-side restore would be ambiguous.

Streaming custom_tool_call_input.delta:
- New token-aware partial JSON state machine for `{"input": "<raw>"}` so
  each Anthropic input_json_delta chunk emits the matching raw string
  characters live, instead of waiting for content_block_stop. Tracks
  top-level object structure, skips string/object/array values, so a
  value containing the literal `"input"` no longer triggers early done.
- \uXXXX surrogate pairs (e.g. emoji) are recombined via utf16.DecodeRune;
  isolated surrogates fall back to U+FFFD instead of two replacement chars.
- maxScanBytes (4 MiB) and maxInputBytes (1 MiB) caps protect against
  unbounded accumulation from a runaway model.
- If the streamer fails or the model deviates from the {input:string}
  schema, content_block_stop falls back to the raw JSON via
  extractCustomToolInput rather than emitting an empty input, and any
  un-emitted remainder is sent as a final delta event before done.
- extractCustomToolInput now distinguishes three cases: input present and
  string → string; input missing or non-string → full raw JSON. Empty
  string is only returned when the model actually output `"input":""`.

Tests:
- 14 new cases covering JSON token-awareness ("input" in values / inside
  user_input keys / nested objects / arrays), surrogate pair recombination
  including split across chunks, oversize / scan-limit rejection, schema
  deviation fallback, duplicate name rejection (3 variants), tool_choice
  sanitisation against the surviving tool set (4 variants) and grammar
  description truncation. 53 tests pass, 3 pre-existing failures unchanged.
OpenAI Codex's apply_patch custom tool description starts with "This is a
FREEFORM tool, so do not wrap the patch in JSON." Because we downgrade
custom tools to function tools with an {input: string} schema, that
sentence directly contradicts the actual wire protocol the model sees and
makes Claude refuse to produce the JSON wrapper.

Strip any sentence containing the word "freeform" from the description
before forwarding to Anthropic so the grammar / usage guidance the rest
of the description carries still reaches the model.
OpenAI /v1/responses SSE wire format prefixes every chunk with both
`event: <type>` and `data: <json>`. The Claude→Responses adapter was
only emitting the data line via helper.ObjectData, which made Codex (and
any client dispatching on the SSE event name) fall back to untyped
handling and drop fields.

Switch the streaming and final-event emission to helper.ResponseChunkData
so the event header is written alongside the JSON payload, matching the
canonical OpenAI Responses stream format already produced by the native
openai relay path.
OpenAI Responses wire format requires function_call.arguments to be a JSON
string (clients call JSON.parse on it) even though it always carries a
JSON object inside that string. Our Claude→Responses converter was storing
the parsed JSON literal directly into json.RawMessage, which made
response.output_item.done emit:

  "arguments": {"cmd": "pwd"}

instead of the canonical:

  "arguments": "{\"cmd\":\"pwd\"}"

Codex and any other client that does JSON.parse(item.arguments) blows up
on the object form. Wrap the raw JSON bytes via argumentsAsJSONString so
the rendered field is a quoted string, matching the upstream OpenAI
Responses stream.
Multi-turn Codex sessions echo previously emitted custom_tool_call /
custom_tool_call_output items back into the next request's input. The
converter was rejecting them with "unknown input item type" because the
request-side parser only knew about function_call / function_call_output.

Map them to the same Anthropic tool_use / tool_result blocks the request
path already produces for the corresponding tool definitions:
- custom_tool_call: wrap the raw string input as {"input": <string>} so
  it matches the {input: string} schema we synthesised when downgrading
  the custom tool to a function tool.
- custom_tool_call_output: identical to function_call_output (Anthropic
  has no distinct concept).
@clansty clansty changed the title Feat/responses to anthropic Feat: responses to anthropic May 12, 2026
@coderabbitai

coderabbitai Bot commented May 12, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

This PR adds complete support for the OpenAI Responses format in the Claude relay by implementing bidirectional request/response conversion, streaming state management for custom tool input parsing, and reasoning signature encoding/decoding.

Changes

OpenAI Responses Format Integration

Layer / File(s) Summary
OpenAI Responses & Claude DTOs
dto/claude.go, dto/openai_response.go
ClaudeMediaMessage gains Data field for redacted thinking; IncompleteDetails changes Reasoning to optional Reason; ResponsesOutput makes multiple fields optional and adds Summary/EncryptedContent; ResponsesStreamResponse adds annotation/text/arguments/input fields and generalizes Part to any.
Custom Tool Input Streaming
relay/channel/claude/custom_input_streamer.go, relay/channel/claude/custom_input_streamer_test.go
Byte-level incremental JSON parser (customInputStreamer) extracts input field from {"input":"..."} with full escape/surrogate-pair handling, size limits, and fallback support; comprehensive tests cover chunking, escaping, non-string inputs, and limit enforcement.
Reasoning Signature Encoding
relay/channel/claude/reasoning_encoding.go, relay/channel/claude/reasoning_encoding_test.go
Envelope-based encoding/decoding for thinking signatures and redacted thinking using na1. prefix with JSON payload, base64url encoding, version validation, and legacy bare signature support; round-trip and error tests validate envelope format.
Responses→Claude Request Conversion
relay/channel/claude/responses_to_claude.go
Converts OpenAI Responses requests to Claude format: validates fields, builds system/messages from instructions/input, converts function and custom tools (downgrading custom to {input:string} schema), enforces reasoning ordering, maps reasoning effort, handles tool choice and metadata.
Claude→Responses Streaming Conversion
relay/channel/claude/claude_to_responses.go
Streaming state machine (ClaudeResponsesStreamState) accumulates Claude chunks into Responses events: allocates output items per content block, emits delta events for text/reasoning/tool inputs, handles custom tool input with fallback JSON parsing, maps stop reasons to status, produces final/failed events.
Relay Integration
relay/channel/claude/adaptor.go, relay/channel/claude/relay-claude.go
Adaptor calls ConvertResponsesRequestToClaude and stores custom tool names in context; ClaudeResponseInfo gains ResponsesState field; streaming handler initializes state, converts chunks via HandleClaudeChunk, and emits SSE events; non-streaming handler converts full response.
Comprehensive Test Suite
relay/channel/claude/responses_conversion_test.go
Tests cover streaming event sequencing for interleaved thinking/text/tools, request/response round-trips, reasoning effort mapping, incomplete output handling, tool choice sanitization, custom tool transformations (stripping hints, converting to functions, tracking names), malformed envelope rejection, and custom tool transparency.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • QuantumNous/new-api#2889: Both add support for the OpenAI /v1/responses format and implement bidirectional conversion between Responses and Claude formats.
  • QuantumNous/new-api#3080: Both modify relay-claude.go streaming state and tool-call handling in the Claude relay channel.
  • QuantumNous/new-api#2837: Both modify OpenAI Responses DTOs (ResponsesStreamResponse and summary part handling).

Suggested reviewers

  • Calcium-Ion
  • seefs001

Poem

🐰 A rabbit hops through Responses and Claude,
Converting requests with careful applaud,
Custom tools streaming, reasoning encoded tight,
Bidirectional magic to make protocols right! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 29.17% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Feat: responses to anthropic' is concise and directly describes the main change: implementing a converter between OpenAI Responses API and Anthropic Messages within the Claude relay channel.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@relay/channel/claude/claude_to_responses.go`:
- Around line 3-5: The file currently imports encoding/json and uses
json.RawMessage and direct marshal/unmarshal calls; remove the direct
encoding/json import and replace all uses of json.RawMessage and any
json.Marshal/json.Unmarshal calls with the repo JSON wrappers from
common/json.go (use the provided RawMessage type/alias and the wrapper functions
such as common.Marshal/ common.Unmarshal or the equivalents exported by
common/json.go). Update all occurrences referenced (e.g., the json.RawMessage
usages and marshal/unmarshal logic around the handlers in
claude_to_responses.go) to call the common/json.go helpers so no business code
imports encoding/json directly.

In `@relay/channel/claude/reasoning_encoding.go`:
- Around line 58-64: The code currently calls base64.RawURLEncoding.DecodeString
on body and only afterwards checks len(raw) against reasoningMaxRawBytes, which
allows large allocations; instead, before calling DecodeString, compute the
expected decoded length using base64.RawURLEncoding.DecodedLen(len(body)) and
reject with the same "invalid reasoning envelope: payload too large" error if
that decoded length exceeds reasoningMaxRawBytes (use the existing symbols
encrypted, reasoningEnvelopePrefix, body, reasoningMaxRawBytes and replace the
post-decode size check), then proceed to DecodeString only when the size is
acceptable.

In `@relay/channel/claude/relay-claude.go`:
- Around line 973-979: The new RelayFormatOpenAIResponses case uses json.Marshal
directly for responsesResp (in the RelayFormatOpenAIResponses branch), bypassing
the shared wrapper; replace json.Marshal(responsesResp) with the common.Marshal
wrapper to match the module's JSON handling (use the common.Marshal function
from common/json.go) and preserve the existing error handling around
responseData, err for the ConvertClaudeResponseToResponses result.

In `@relay/channel/claude/responses_to_claude.go`:
- Around line 117-137: The code currently parses an instructions array into
asArray and silently skips unsupported entries; instead validate every element
and fail fast: iterate asArray (the loop using item, m and
stringFromInputTextPart) and if an element is not a map[string]any or
stringFromInputTextPart returns false/empty, return an error (400/bad request)
that the instructions array contains unsupported parts; only build and return
blocks (dto.ClaudeMediaMessage entries) when every item is a valid text part.
Ensure the error message clearly identifies that non-text or malformed
instruction parts were provided so callers see the 400 rather than getting a
partial/empty prompt.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 48ba9c88-35fb-4e07-950f-f2c89bad76ca

📥 Commits

Reviewing files that changed from the base of the PR and between aa56667 and 54f11b8.

📒 Files selected for processing (11)
  • dto/claude.go
  • dto/openai_response.go
  • relay/channel/claude/adaptor.go
  • relay/channel/claude/claude_to_responses.go
  • relay/channel/claude/custom_input_streamer.go
  • relay/channel/claude/custom_input_streamer_test.go
  • relay/channel/claude/reasoning_encoding.go
  • relay/channel/claude/reasoning_encoding_test.go
  • relay/channel/claude/relay-claude.go
  • relay/channel/claude/responses_conversion_test.go
  • relay/channel/claude/responses_to_claude.go

Comment on lines +3 to +5
import (
"encoding/json"
"strconv"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Remove direct encoding/json usage from relay business code.

This file still directly imports/uses encoding/json (json.RawMessage). Please move this through the common/json.go abstraction so it follows repo JSON policy consistently.

As per coding guidelines **/*.go: "All JSON marshal/unmarshal operations MUST use wrapper functions from common/json.go ... Do NOT directly import or call encoding/json in business code."

Also applies to: 745-746, 763-772

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/claude/claude_to_responses.go` around lines 3 - 5, The file
currently imports encoding/json and uses json.RawMessage and direct
marshal/unmarshal calls; remove the direct encoding/json import and replace all
uses of json.RawMessage and any json.Marshal/json.Unmarshal calls with the repo
JSON wrappers from common/json.go (use the provided RawMessage type/alias and
the wrapper functions such as common.Marshal/ common.Unmarshal or the
equivalents exported by common/json.go). Update all occurrences referenced
(e.g., the json.RawMessage usages and marshal/unmarshal logic around the
handlers in claude_to_responses.go) to call the common/json.go helpers so no
business code imports encoding/json directly.

Comment on lines +156 to +208
case customStateInTargetValueEscape:
switch b {
case 'u':
p.state = customStateInTargetValueUnicode
p.unicodeHex = p.unicodeHex[:0]
case '"', '\\', '/':
out.WriteByte(b)
p.pending.WriteByte(b)
p.state = customStateInTargetValue
case 'b':
p.writeRune('\b', out)
p.state = customStateInTargetValue
case 'f':
p.writeRune('\f', out)
p.state = customStateInTargetValue
case 'n':
p.writeRune('\n', out)
p.state = customStateInTargetValue
case 'r':
p.writeRune('\r', out)
p.state = customStateInTargetValue
case 't':
p.writeRune('\t', out)
p.state = customStateInTargetValue
default:
out.WriteByte(b)
p.pending.WriteByte(b)
p.state = customStateInTargetValue
}
case customStateInTargetValueUnicode:
p.unicodeHex = append(p.unicodeHex, b)
if len(p.unicodeHex) == 4 {
r := decodeHexQuad(p.unicodeHex)
p.unicodeHex = p.unicodeHex[:0]
if utf16.IsSurrogate(r) {
if p.pendingHighSurrogate == 0 && r >= 0xD800 && r <= 0xDBFF {
p.pendingHighSurrogate = r
} else if p.pendingHighSurrogate != 0 && r >= 0xDC00 && r <= 0xDFFF {
combined := utf16.DecodeRune(p.pendingHighSurrogate, r)
p.pendingHighSurrogate = 0
p.writeRune(combined, out)
} else {
p.pendingHighSurrogate = 0
p.writeRune(utf8.RuneError, out)
}
} else {
if p.pendingHighSurrogate != 0 {
p.writeRune(utf8.RuneError, out)
p.pendingHighSurrogate = 0
}
p.writeRune(r, out)
}
p.state = customStateInTargetValue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fail on malformed escapes instead of rewriting them.

Unknown escape sequences and invalid \u/surrogate pairs are currently turned into a different string while Parsed() can still end up true. That bypasses the caller’s fallback path and silently corrupts streamed custom tool input.

🛠 Preserve fallback on invalid JSON
 	case customStateInTargetValue:
 		switch b {
 		case '"':
+			if p.pendingHighSurrogate != 0 {
+				p.state = customStateFailed
+				return
+			}
 			p.finishedRaw = p.pending.String()
 			p.parsed = true
 			p.state = customStateDone
@@
 	case customStateInTargetValueEscape:
 		switch b {
@@
 		default:
-			out.WriteByte(b)
-			p.pending.WriteByte(b)
-			p.state = customStateInTargetValue
+			p.state = customStateFailed
+			return
 		}
 	case customStateInTargetValueUnicode:
 		p.unicodeHex = append(p.unicodeHex, b)
 		if len(p.unicodeHex) == 4 {
 			r := decodeHexQuad(p.unicodeHex)
 			p.unicodeHex = p.unicodeHex[:0]
+			if r == utf8.RuneError {
+				p.state = customStateFailed
+				return
+			}
 			if utf16.IsSurrogate(r) {
 				if p.pendingHighSurrogate == 0 && r >= 0xD800 && r <= 0xDBFF {
 					p.pendingHighSurrogate = r
 				} else if p.pendingHighSurrogate != 0 && r >= 0xDC00 && r <= 0xDFFF {
 					combined := utf16.DecodeRune(p.pendingHighSurrogate, r)
 					p.pendingHighSurrogate = 0
 					p.writeRune(combined, out)
 				} else {
-					p.pendingHighSurrogate = 0
-					p.writeRune(utf8.RuneError, out)
+					p.state = customStateFailed
+					return
 				}

Comment on lines +58 to +64
body := strings.TrimPrefix(encrypted, reasoningEnvelopePrefix)
raw, decErr := base64.RawURLEncoding.DecodeString(body)
if decErr != nil {
return "", "", "", errors.New("invalid reasoning envelope: base64 decode: " + decErr.Error())
}
if len(raw) > reasoningMaxRawBytes {
return "", "", "", errors.New("invalid reasoning envelope: payload too large")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Reject oversized envelopes before decoding.

DecodeString allocates from the full base64 payload before the reasoningMaxRawBytes guard runs. A large encrypted_content can therefore burn memory/CPU on the request path even though the decoded payload is guaranteed to be rejected.

🛡️ Fail before allocating the decoded buffer
 	body := strings.TrimPrefix(encrypted, reasoningEnvelopePrefix)
+	if base64.RawURLEncoding.DecodedLen(len(body)) > reasoningMaxRawBytes {
+		return "", "", "", errors.New("invalid reasoning envelope: payload too large")
+	}
 	raw, decErr := base64.RawURLEncoding.DecodeString(body)
 	if decErr != nil {
 		return "", "", "", errors.New("invalid reasoning envelope: base64 decode: " + decErr.Error())
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
body := strings.TrimPrefix(encrypted, reasoningEnvelopePrefix)
raw, decErr := base64.RawURLEncoding.DecodeString(body)
if decErr != nil {
return "", "", "", errors.New("invalid reasoning envelope: base64 decode: " + decErr.Error())
}
if len(raw) > reasoningMaxRawBytes {
return "", "", "", errors.New("invalid reasoning envelope: payload too large")
body := strings.TrimPrefix(encrypted, reasoningEnvelopePrefix)
if base64.RawURLEncoding.DecodedLen(len(body)) > reasoningMaxRawBytes {
return "", "", "", errors.New("invalid reasoning envelope: payload too large")
}
raw, decErr := base64.RawURLEncoding.DecodeString(body)
if decErr != nil {
return "", "", "", errors.New("invalid reasoning envelope: base64 decode: " + decErr.Error())
}
if len(raw) > reasoningMaxRawBytes {
return "", "", "", errors.New("invalid reasoning envelope: payload too large")
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/claude/reasoning_encoding.go` around lines 58 - 64, The code
currently calls base64.RawURLEncoding.DecodeString on body and only afterwards
checks len(raw) against reasoningMaxRawBytes, which allows large allocations;
instead, before calling DecodeString, compute the expected decoded length using
base64.RawURLEncoding.DecodedLen(len(body)) and reject with the same "invalid
reasoning envelope: payload too large" error if that decoded length exceeds
reasoningMaxRawBytes (use the existing symbols encrypted,
reasoningEnvelopePrefix, body, reasoningMaxRawBytes and replace the post-decode
size check), then proceed to DecodeString only when the size is acceptable.

Comment on lines +973 to +979
case types.RelayFormatOpenAIResponses:
responsesResp := ConvertClaudeResponseToResponses(&claudeResponse, getResponsesCustomToolNames(c))
if claudeInfo.Created > 0 {
responsesResp.CreatedAt = int(claudeInfo.Created)
}
responseData, err = json.Marshal(responsesResp)
if err != nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n '\bjson\.(Marshal|Unmarshal)\(' relay/channel/claude/relay-claude.go

Repository: QuantumNous/new-api

Length of output: 315


🏁 Script executed:

sed -n '973,979p' relay/channel/claude/relay-claude.go

Repository: QuantumNous/new-api

Length of output: 359


🏁 Script executed:

sed -n '960,990p' relay/channel/claude/relay-claude.go

Repository: QuantumNous/new-api

Length of output: 1246


🏁 Script executed:

grep -n 'import' relay/channel/claude/relay-claude.go | head -20

Repository: QuantumNous/new-api

Length of output: 209


🏁 Script executed:

rg -n 'common\.Marshal|common\.Unmarshal' relay/channel/claude/relay-claude.go

Repository: QuantumNous/new-api

Length of output: 464


🏁 Script executed:

git -C . diff HEAD -- relay/channel/claude/relay-claude.go | grep -A 30 -B 5 'RelayFormatOpenAI'

Repository: QuantumNous/new-api

Length of output: 45


🏁 Script executed:

git -C . log --oneline -1

Repository: QuantumNous/new-api

Length of output: 134


🏁 Script executed:

rg -B 2 -A 5 'case types.RelayFormatOpenAI:' relay/channel/claude/relay-claude.go | head -30

Repository: QuantumNous/new-api

Length of output: 416


🏁 Script executed:

# Check if common package is properly imported
head -50 relay/channel/claude/relay-claude.go | grep -E '^import|common'

Repository: QuantumNous/new-api

Length of output: 266


Use common.Marshal in the new Responses branch.

At line 978, the new RelayFormatOpenAIResponses case uses direct json.Marshal(), which bypasses the shared JSON wrapper used throughout the Claude relay module.

♻️ Localized fix
-		responseData, err = json.Marshal(responsesResp)
+		responseData, err = common.Marshal(responsesResp)

Per coding guidelines, all JSON marshal/unmarshal operations in business code must use wrapper functions from common/json.go.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/claude/relay-claude.go` around lines 973 - 979, The new
RelayFormatOpenAIResponses case uses json.Marshal directly for responsesResp (in
the RelayFormatOpenAIResponses branch), bypassing the shared wrapper; replace
json.Marshal(responsesResp) with the common.Marshal wrapper to match the
module's JSON handling (use the common.Marshal function from common/json.go) and
preserve the existing error handling around responseData, err for the
ConvertClaudeResponseToResponses result.

Comment on lines +117 to +137
var asArray []any
if err := common.Unmarshal(raw, &asArray); err == nil {
blocks := make([]dto.ClaudeMediaMessage, 0, len(asArray))
for _, item := range asArray {
m, ok := item.(map[string]any)
if !ok {
continue
}
if text, ok := stringFromInputTextPart(m); ok && text != "" {
blocks = append(blocks, dto.ClaudeMediaMessage{
Type: "text",
Text: common.GetPointer(text),
})
}
}
if len(blocks) == 0 {
return nil, nil
}
return blocks, nil
}
return nil, errors.New("instructions must be a string or an array of input_text parts")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Don't silently drop unsupported instructions parts.

This loop skips any non-text entry in the instructions array and still returns a partial or empty system prompt. That loses caller-provided instructions without surfacing a 400, which is exactly the degradation this converter is otherwise avoiding.

🧭 Tighten validation here
 	for _, item := range asArray {
 		m, ok := item.(map[string]any)
 		if !ok {
-			continue
+			return nil, errors.New("instructions must be a string or an array of text/input_text parts")
 		}
-		if text, ok := stringFromInputTextPart(m); ok && text != "" {
+		text, ok := stringFromInputTextPart(m)
+		if !ok {
+			return nil, errors.New("instructions must be a string or an array of text/input_text parts")
+		}
+		if text != "" {
 			blocks = append(blocks, dto.ClaudeMediaMessage{
 				Type: "text",
 				Text: common.GetPointer(text),
 			})
 		}
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/claude/responses_to_claude.go` around lines 117 - 137, The code
currently parses an instructions array into asArray and silently skips
unsupported entries; instead validate every element and fail fast: iterate
asArray (the loop using item, m and stringFromInputTextPart) and if an element
is not a map[string]any or stringFromInputTextPart returns false/empty, return
an error (400/bad request) that the instructions array contains unsupported
parts; only build and return blocks (dto.ClaudeMediaMessage entries) when every
item is a valid text part. Ensure the error message clearly identifies that
non-text or malformed instruction parts were provided so callers see the 400
rather than getting a partial/empty prompt.

@siakinnik

Copy link
Copy Markdown

Tested OpenAi-api support using node.js with npm package "openai". Works with and without stream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants