feat: add native Claude Messages compatibility - #6652
Open
Citrus086 wants to merge 2 commits into
Open
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
WalkthroughThis PR adds Claude count-tokens endpoint support with dedicated request and response handling, converts all middleware error responses to a relay-aware helper that formats Claude-native error envelopes, and adds stream-terminal tracking to detect incomplete streams and classify success correctly. ChangesClaude Count-Tokens Feature
Relay Error Response Unification
Stream Terminal Tracking
Estimated code review effort: 4 (Complex) | ~75 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant Client
participant Router as relay-router.go
participant Controller as controller.Relay
participant Helper as ClaudeCountTokensHelper
participant Adaptor as Claude Adaptor
participant Upstream as Claude API
Client->>Router: POST /v1/messages/count_tokens
Router->>Controller: Relay(RelayModeClaudeCountTokens)
Controller->>Controller: skip token estimation, billing prep
Controller->>Helper: ClaudeCountTokensHelper(c, info)
Helper->>Adaptor: ConvertClaudeCountTokensRequest
Adaptor-->>Helper: ClaudeCountTokensRequest
Helper->>Upstream: send request
Upstream-->>Helper: response
Helper->>Adaptor: DoClaudeCountTokensResponse
Adaptor->>Adaptor: ClaudeCountTokensHandler validates InputTokens
Adaptor-->>Client: copy native response or Claude error
sequenceDiagram
participant Middleware
participant AbortHelper as abortWithRelayMessage
participant ErrorTypes as types.NewAPIError
participant Client
Middleware->>AbortHelper: abort(ctx, statusCode, message, errorCode)
AbortHelper->>AbortHelper: check request path
alt path is /v1/messages
AbortHelper->>ErrorTypes: ToClaudeErrorResponse(requestID)
ErrorTypes-->>AbortHelper: ClaudeErrorResponse
else other path
AbortHelper->>AbortHelper: build generic error response
end
AbortHelper-->>Client: JSON error response
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@service/error.go`:
- Around line 151-154: When io.ReadAll fails in the error path after
CloseResponseBodyGracefully, the code returns NewClaudeError without preserving
the upstream request ID from the response headers. In the error handling branch
where io.ReadAll fails (inside the if err != nil block), extract the request-id
from resp.Header.Get("request-id") and include it in the NewClaudeError call
alongside the existing err, types.ErrorCodeReadResponseBodyFailed, and
statusCode parameters so that the upstream request ID is not lost when the
body-read fails.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 926b698e-7184-422d-a4a0-07aaa758dcf0
📒 Files selected for processing (31)
controller/relay.gomiddleware/auth.gomiddleware/distributor.gomiddleware/jimeng_adapter.gomiddleware/model-rate-limit.gomiddleware/performance.gomiddleware/utils.gomiddleware/utils_test.gorelay/channel/adapter.gorelay/channel/claude/adaptor.gorelay/channel/claude/count_tokens_test.gorelay/channel/claude/relay-claude.gorelay/channel/claude/relay_claude_test.gorelay/claude_handler.gorelay/common/relay_info.gorelay/common/stream_status.gorelay/common/stream_status_test.gorelay/constant/relay_mode.gorelay/constant/relay_mode_test.gorelay/helper/stream_scanner.gorelay/helper/stream_scanner_test.gorelaykit/dto/claude.gorelaykit/relayconvert/convmeta/format.gorelaykit/relayconvert/internal/claude_messages/to_oai_chat_resp.gorelaykit/types/error.gorelaykit/types/error_test.gorouter/relay-router.goservice/error.goservice/error_test.goservice/log_info_generate.goservice/text_quota.go
Rain-kl
added a commit
to Rain-kl/new-api
that referenced
this pull request
Aug 5, 2026
Add native /v1/messages/count_tokens, preserve Anthropic error format and request_id, and treat Claude streams as complete only on message_stop. Incomplete streams are marked so success is not misreported. Unsupported count_tokens channels return an explicit error instead of local estimation. Upstream: QuantumNous#6652
This was referenced Aug 6, 2026
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
变更描述 / Description
补齐 Claude Messages 接口的几个原生行为：目前
/v1/messages已经可以使用，但还有几个地方会影响 Claude Code 一类的客户端：/v1/messages/count_tokensmessage_stop和意外断开本 PR 做了以下调整：
/v1/messages/count_tokensrequest_idmessage_stop才记为正常完成这些不影响现有 OpenAI 接口
变更类型 / Type of change
提交前检查项 / Checklist
运行证明 / Proof of Work
改动在下述情况生效：
Summary by CodeRabbit
POST /v1/messages/count_tokens.