feat: smart channel routing based on request API type - #5017
Open
xemxx wants to merge 9 commits into
Open
Conversation
added 3 commits
May 21, 2026 15:29
When multiple channels share the same priority for a model, prefer channels whose native API type matches the client's request format (OpenAI/Claude/Gemini). This avoids unnecessary request/response conversion and improves relay performance. Changes: - Add RelayFormatToAPIType() mapping in types/relay_format.go - Pass RelayFormat through RetryParam to channel selection - Smart routing in model/ability.go (DB query path, memory cache disabled) - Smart routing in model/channel_cache.go (memory cache enabled) - Default to OpenAI format in middleware/distributor.go (pre-detection phase) Fixes QuantumNous#4982
When the smart channel routing query joins abilities with channels, both tables expose 'group' and 'weight' columns, causing SQLite to fail with 'ambiguous column name' errors. - Qualify the group filter with abilities. prefix in JOINed queries - Wrap the priority subquery in parentheses so the generated SQL is valid (priority = (SELECT MAX(priority) ...) instead of priority = SELECT ...) - Order by abilities.weight DESC explicitly, and select abilities.* on the filtered query to avoid column collision when scanning rows
The Distribute middleware runs before the per-route handler that sets the actual RelayFormat, so smart channel routing was always seeing RelayFormatOpenAI and could never prefer a Claude- or Gemini-native channel for /v1/messages or Gemini paths. Add InferRelayFormatFromPath, which mirrors the routing table in router/relay-router.go, and use it in the distributor so requests hitting /v1/messages get RelayFormatClaude (and so on) before channel selection runs.
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds relay-format inference and API-type mapping helpers, then uses them to route abilities and same-priority channels toward matches for the request path’s inferred API type. ChangesSmart Channel Routing by Relay Format
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@types/relay_format.go`:
- Around line 70-71: The case in types/relay_format.go that matches routes with
strings.HasPrefix(path, "/v1/engines/") && strings.HasSuffix(path,
"/embeddings") currently returns RelayFormatGemini which misclassifies legacy
OpenAI-style embeddings; update that branch to return the OpenAI embeddings
relay format constant (e.g., RelayFormatOpenAIEmbeddings or the project’s
embeddings-specific RelayFormat) instead of RelayFormatGemini so
/v1/engines/*/embeddings is routed to the embeddings backend.
- Around line 50-76: The InferRelayFormatFromPath function currently returns
RelayFormatOpenAI in the default case which biases routing for unknown paths;
change the default return to RelayFormatUnknown to preserve neutral behavior,
and add a new case branch so paths starting with "/v1beta/models/" return
RelayFormatGemini (similar to the existing "/v1/models/" handling); update the
switch in InferRelayFormatFromPath accordingly and ensure RelayFormatUnknown is
used downstream where absence of an API-type hint is intended.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a466504f-b7f0-43c6-98eb-6743b74a7024
📒 Files selected for processing (6)
controller/relay.gomiddleware/distributor.gomodel/ability.gomodel/channel_cache.goservice/channel_select.gotypes/relay_format.go
…d unknown paths Address review feedback on smart channel routing: - /v1/engines/*/embeddings is OpenAI's legacy embeddings endpoint, not Gemini. Return RelayFormatEmbedding so it routes to the correct backend. - /v1beta/models/ is the Gemini API path used by upstream clients (see middleware/distributor.go handling); add it alongside /v1/models/ so Gemini routes are inferred consistently. - Default branch now returns an empty RelayFormat instead of RelayFormatOpenAI. Downstream GetChannel treats empty as 'no API-type hint' and falls back to the original priority/weight-based selection, which is the truly neutral behaviour for unknown paths.
Author
关于 #4759@seefs001 在 #4759 提了同主题的修复(早本 PR 10 天),思路一致:同模型多渠道时优先选原生格式渠道。两者实现路径不同,简单对比:
另:#4759 引用了 我都可以配合 — 关闭本 PR、或在 #4759 基础上补齐路径覆盖、或在本 PR 引入 |
added 3 commits
May 26, 2026 18:24
…am-advanced-custom # Conflicts: # controller/relay.go # middleware/distributor.go # model/ability.go # model/channel_cache.go # service/channel_select.go
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@model/ability.go`:
- Around line 182-191: The DB filtering in ability matching should preserve the
cache path’s explicit map lookup check, because a missing apiTypeById entry
currently defaults to 0 and can incorrectly match APITypeOpenAI. Update the
logic in the DB path within the ability filtering routine (the loop over
abilities in the method handling channels and abilities) to use an ok-style
presence check before comparing the mapped API type to expectedAPIType, matching
the behavior used in the cache path and preventing unmapped channels from
slipping through.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 9cb91808-a2c5-495f-913c-886862f9f876
📒 Files selected for processing (2)
model/ability.gomodel/channel_cache.go
added 2 commits
June 30, 2026 16:42
When a channel id is absent from the lookup map (channel deleted or ChannelType2APIType returned !ok), apiTypeById[id] yields the int zero value, which equals APITypeOpenAI. With expectedAPIType=0 that produced a false match. Use the two-value map lookup so only channels with a known api type can be preferred. Reported by CodeRabbit on PR QuantumNous#5017.
# Conflicts: # model/ability.go
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
📝 变更描述 / Description
当前 new-api 支持同一模型配置多个渠道（如 Claude、OpenAI、Gemini 等），但渠道选择仅按优先级 + 权重随机，未考虑客户端请求的 API 格式。这导致：
当请求模型例如 mimo-v2.5-pro 的时候，小米官方提供两种接口，但是 new-api 这块默认渠道只能配置 openai 或者 claude，如果我同时需要两种接口访问，当前只能配置一种然后做转换，但是转换有时候在 claude code 或者 hermes agent 会失败
这时需要一个策略来自动选择，比如配置两个小米官方的渠道，分别是 openai 和 claude，当用户访问的时候，自动选择对应的官方接口去直接访问，这样避免转换带来的误差以及问题出现，但是也要考虑现有逻辑，所以需要当两个渠道同优先级的时候再进入这种策略，否则依旧保证高优先级的先访问，走默认的转换逻辑
兼容性：
RelayFormat时跳过智能路由，保持原有行为🚀 变更类型 / Type of change
🔗 关联任务 / Related Issue
✅ 提交前检查项 / Checklist
Bug fix，我已提交或关联对应 Issue，且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。📸 运行证明 / Proof of Work
Summary by CodeRabbit