✨ feat: Enhance model listing and retrieval with support for Anthropic and Gemini models; refactor routes for better API key handling - #1547
Conversation
WalkthroughThe changes introduce support for multiple model providers (Anthropic, Gemini, OpenAI) in model listing and retrieval endpoints. Function signatures and routing logic are updated to route requests based on headers and query parameters. New data structures for Anthropic and Gemini models are added, and middleware is adjusted to handle provider-specific authentication and authorization scenarios. Changes
Sequence Diagram(s)Dynamic Model Listing Based on ProvidersequenceDiagram
participant Client
participant Router
participant Middleware
participant Controller
Client->>Router: GET /v1/models (with headers/query)
Router->>Middleware: Apply TokenAuth
Middleware->>Router: Set Authorization header (provider-specific)
Router->>Controller: ListModels(ctx, modelType)
Controller->>Controller: Determine provider (Anthropic/Gemini/OpenAI)
Controller->>Controller: Format response (provider-specific DTO)
Controller->>Client: Return JSON response
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15 minutes Possibly related PRs
Poem
Note 🔌 MCP (Model Context Protocol) integration is now available in Early Access!Pro users can now connect to remote MCP servers under the Integrations page to get reviews and chat conversations that understand additional development context. 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
dto/pricing.go (1)
21-35: Usinginterface{}forfeits type safetyAll GeminiModel fields are declared as
interface{}even for well-defined primitive types (e.g.,name,displayName,inputTokenLimit…).
This invites runtime panics and burdens every caller with manual type assertions.Define concrete types or pointers where the schema is known; fall back to
anyonly for genuinely polymorphic fields.controller/model.go (1)
193-203: NilnextPageTokenmarshals asnullWhen replying for Gemini,
nextPageTokenis set tonil(typed asinterface{}).
Clients expecting a string may choke. Prefer the empty string or omit the field.- "nextPageToken": nil, + "nextPageToken": "",
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
controller/model.go(4 hunks)dto/pricing.go(1 hunks)middleware/auth.go(1 hunks)router/relay-router.go(2 hunks)
🧰 Additional context used
🧠 Learnings (4)
📓 Common learnings
Learnt from: antecanis8
PR: QuantumNous/new-api#1500
File: relay/channel/gemini/adaptor.go:0-0
Timestamp: 2025-08-04T14:36:34.528Z
Learning: All currently serviced Gemini embedding models support the outputDimensionality parameter, not just specific models like text-embedding-004.
Learnt from: antecanis8
PR: QuantumNous/new-api#1500
File: relay/channel/gemini/adaptor.go:0-0
Timestamp: 2025-08-04T14:36:34.528Z
Learning: In the Gemini API batchEmbedContents endpoint, each request item in the "requests" array should include its own "model" field, as shown in the official documentation at https://ai.google.dev/api/embeddings?hl=zh-cn#method:-models.batchembedcontents. The model field is not a top-level parameter.
📚 Learning: 2025-06-21T03:37:41.726Z
Learnt from: 9Ninety
PR: QuantumNous/new-api#1273
File: relay/channel/gemini/relay-gemini.go:97-116
Timestamp: 2025-06-21T03:37:41.726Z
Learning: In relay/channel/gemini/relay-gemini.go, the thinking budget calculation logic (including the MaxOutputTokens multiplication) was introduced in PR #1247. PR #1273 focused specifically on decoupling the thoughts summary feature from thinking budget settings and did not modify the existing thinking budget behavior.
Applied to files:
dto/pricing.gorouter/relay-router.go
📚 Learning: 2025-08-04T14:36:34.528Z
Learnt from: antecanis8
PR: QuantumNous/new-api#1500
File: relay/channel/gemini/adaptor.go:0-0
Timestamp: 2025-08-04T14:36:34.528Z
Learning: In the Gemini API batchEmbedContents endpoint, each request item in the "requests" array should include its own "model" field, as shown in the official documentation at https://ai.google.dev/api/embeddings?hl=zh-cn#method:-models.batchembedcontents. The model field is not a top-level parameter.
Applied to files:
dto/pricing.go
📚 Learning: 2025-08-08T17:12:43.146Z
Learnt from: RedwindA
PR: QuantumNous/new-api#1537
File: relay/gemini_handler.go:330-342
Timestamp: 2025-08-08T17:12:43.146Z
Learning: In the new-api repository, the `GeminiEmbeddingHandler` function in `relay/gemini_handler.go` is designed specifically for native Gemini embedding requests and therefore does not require the `ConvertGeminiRequest` step that is used in the chat handler. The embedding requests are already in the native Gemini format and don't need conversion.
Applied to files:
router/relay-router.go
There was a problem hiding this comment.
Potential panic when user has zero visible models
useranthropicModels[0] and useranthropicModels[len-1] assume the slice is non-empty.
With strict token model limits (or mis-configuration) the slice can be empty → runtime panic.
Guard before indexing:
if len(useranthropicModels) == 0 {
c.JSON(200, gin.H{"data": []dto.AnthropicModel{}})
return
}🤖 Prompt for AI Agents
In controller/model.go around lines 175 to 191, the code indexes
useranthropicModels[0] and useranthropicModels[len-1] without checking if the
slice is empty, which can cause a runtime panic if there are no models. Add a
guard condition to check if useranthropicModels is non-empty before accessing
these indices, and handle the empty case appropriately by either omitting those
fields or providing safe default values in the JSON response.
There was a problem hiding this comment.
🛠️ Refactor suggestion
Avoid overwriting Authorization with an empty key
anthropicKey might be empty when only the path matches /v1/messages; the current branch still executes and sets
Authorization: Bearer (no token). Down-stream parsing will then treat the token as empty, leading to failed look-ups or accidental fall-through to other credential sources.
-if strings.Contains(c.Request.URL.Path, "/v1/messages") || (anthropicKey != "" && c.Request.Header.Get("anthropic-version") != "") {
- c.Request.Header.Set("Authorization", "Bearer "+anthropicKey)
+if anthropicKey != "" && (strings.Contains(c.Request.URL.Path, "/v1/messages") ||
+ c.Request.Header.Get("anthropic-version") != "") {
+ c.Request.Header.Set("Authorization", "Bearer "+anthropicKey)
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| anthropicKey := c.Request.Header.Get("x-api-key") | |
| // 检查path包含/v1/messages | |
| if strings.Contains(c.Request.URL.Path, "/v1/messages") { | |
| // 从x-api-key中获取key | |
| key := c.Request.Header.Get("x-api-key") | |
| if key != "" { | |
| c.Request.Header.Set("Authorization", "Bearer "+key) | |
| } | |
| // 或者是否 x-api-key 不为空且存在anthropic-version | |
| // 谁知道有多少不符合规范没写anthropic-version的 | |
| // 所以就这样随它去吧（ | |
| if strings.Contains(c.Request.URL.Path, "/v1/messages") || (anthropicKey != "" && c.Request.Header.Get("anthropic-version") != "") { | |
| c.Request.Header.Set("Authorization", "Bearer "+anthropicKey) | |
| } | |
| anthropicKey := c.Request.Header.Get("x-api-key") | |
| // 检查path包含/v1/messages | |
| // 或者是否 x-api-key 不为空且存在anthropic-version | |
| // 谁知道有多少不符合规范没写anthropic-version的 | |
| // 所以就这样随它去吧（ | |
| if anthropicKey != "" && (strings.Contains(c.Request.URL.Path, "/v1/messages") || | |
| c.Request.Header.Get("anthropic-version") != "") { | |
| c.Request.Header.Set("Authorization", "Bearer "+anthropicKey) | |
| } |
🤖 Prompt for AI Agents
In middleware/auth.go around lines 195 to 202, the code sets the Authorization
header to "Bearer " even when anthropicKey is empty if the path contains
"/v1/messages". To fix this, modify the condition to only set the Authorization
header when anthropicKey is not empty, ensuring you do not overwrite
Authorization with an empty token and cause downstream parsing issues.
There was a problem hiding this comment.
Path check misses “/v1/models” (no trailing slash)
The new prefixes cover /v1beta/* and /v1/models/…, but not the plain listing route /v1/models.
That endpoint will no longer have the query/header key copied into Authorization, breaking requests that rely on it.
Add an explicit check:
- strings.HasPrefix(c.Request.URL.Path, "/v1/models/") {
+ c.Request.URL.Path == "/v1/models" ||
+ strings.HasPrefix(c.Request.URL.Path, "/v1/models/") {🤖 Prompt for AI Agents
In middleware/auth.go around lines 204 to 206, the path checks for "/v1/models"
endpoints miss the exact "/v1/models" route without a trailing slash, causing
authorization headers not to be copied for that route. Add an explicit condition
to check if the request URL path equals "/v1/models" to ensure this route is
properly handled alongside the existing prefix checks.
There was a problem hiding this comment.
Wrong controller invoked – list route calls RetrieveModel
The handler for GET /v1/models selects Gemini and then calls controller.RetrieveModel, which expects a :model path param and returns a single model.
Clients requesting a list will receive a 404-style error.
-case c.GetHeader("x-goog-api-key") != "" || c.Query("key") != "": // 单独的适配
- controller.RetrieveModel(c, constant.ChannelTypeGemini)
+case c.GetHeader("x-goog-api-key") != "" || c.Query("key") != "":
+ controller.ListModels(c, constant.ChannelTypeGemini)🤖 Prompt for AI Agents
In router/relay-router.go around lines 19 to 27, the GET /v1/models route
incorrectly calls controller.RetrieveModel for Gemini channel instead of listing
models. Replace the call to controller.RetrieveModel with controller.ListModels
for the Gemini case to correctly return the list of models without requiring a
:model path parameter.
There was a problem hiding this comment.
🛠️ Refactor suggestion
No Gemini branch for /:model retrieval
GET /v1/models/{id} may be called with Gemini credentials, but the switch omits that case.
Either explicitly reject Gemini here or add a Gemini branch for parity.
🤖 Prompt for AI Agents
In router/relay-router.go around lines 30 to 37, the GET handler for "/:model"
does not handle the case when Gemini credentials are provided, which may cause
unexpected behavior. Update the switch statement to explicitly check for Gemini
credentials and either add a branch to call controller.RetrieveModel with the
Gemini channel type or explicitly reject the request with an appropriate
response. This ensures consistent handling of all credential types.
…c and Gemini models; refactor routes for better API key handling
d18200f to
fdb6a3c
Compare
✨ feat: Enhance model listing and retrieval with support for Anthropic and Gemini models; refactor routes for better API key handling
2 participants
Summary by CodeRabbit
/v1beta/modelsand/v1beta/openai/models.