feat: added list models request#645
Conversation
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds a /v1/models listing API: new Bifrost ListModels APIs, model-list schemas with pagination, provider ListModels implementations across providers, HTTP routing/integrations, UI/config toggles, OpenAPI updates, tests and provider response converters; also standardizes HTTP method usage and updates default API versions for some providers. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant HTTPHandler
participant Bifrost
participant Provider
rect rgba(0,128,96,0.06)
Client->>HTTPHandler: GET /v1/models?provider=openai&page_size=50
end
HTTPHandler->>Bifrost: ListModelsRequest(ctx, req)
Bifrost->>Bifrost: getProviderByKey(provider) / selectKeyFromProviderForModel(ListModelsRequest)
alt provider specified
Bifrost->>Provider: ListModels(ctx, key, request)
else provider omitted
Bifrost->>Bifrost: iterate configured providers -> Provider.ListModels (per-provider calls)
end
rect rgba(0,96,192,0.06)
Provider->>Provider: Build request -> execute -> parse -> convert to BifrostListModelsResponse
Provider-->>Bifrost: BifrostListModelsResponse (data, extra_fields, latency)
end
Bifrost->>Bifrost: Aggregate / ApplyPagination / Track latency/errors
Bifrost->>HTTPHandler: paginated ListModelsResponse
HTTPHandler-->>Client: 200 { data:[...], next_page_token: "..." }
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
3 times, most recently
from
80805a2 to
2c910e9
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
🧹 Nitpick comments (11)
transports/bifrost-http/handlers/inference.go (1)
289-317: Normalize provider input for consistencyConsider lowercasing the provider to match internal keys.
- provider := string(ctx.QueryArgs().Peek("provider")) + provider := strings.ToLower(string(ctx.QueryArgs().Peek("provider")))core/schemas/providers/openrouter/models.go (1)
10-12: Micro-opt: preallocate response slice capacityAvoid reallocations by reserving len(response.Data).
- bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, - } + bifrostResponse := &schemas.BifrostListModelsResponse{ + Data: make([]schemas.Model, 0, len(response.Data)), + }core/schemas/providers/anthropic/models.go (1)
10-12: Micro-opt: preallocate response slice capacityReserve len(response.Data).
- bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, - } + bifrostResponse := &schemas.BifrostListModelsResponse{ + Data: make([]schemas.Model, 0, len(response.Data)), + }core/schemas/providers/mistral/models.go (1)
10-12: Micro-opt: preallocate response slice capacityReserve len(response.Data).
- bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, - } + bifrostResponse := &schemas.BifrostListModelsResponse{ + Data: make([]schemas.Model, 0, len(response.Data)), + }core/providers/mistral.go (1)
235-238: Avoid double unmarshal when RawResponse isn’t requestedYou parse into interface{} unconditionally. Only unmarshal raw when sendBackRawResponse is true to save CPU/allocations.
- var rawResponse interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { - return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.Mistral) - } - - return &mistralResponse, rawResponse, latency, nil + var rawResponse interface{} + if provider.sendBackRawResponse { + if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { + return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.Mistral) + } + } + return &mistralResponse, rawResponse, latency, nilAlso applies to: 240-241
core/schemas/providers/cohere/models.go (1)
12-14: Micro-opt: preallocate response slice capacityReserve len(response.Models).
- bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, - } + bifrostResponse := &schemas.BifrostListModelsResponse{ + Data: make([]schemas.Model, 0, len(response.Models)), + }core/providers/anthropic.go (2)
859-862: Avoid double unmarshal when RawResponse isn’t requestedOnly unmarshal raw when sendBackRawResponse is true.
- var rawResponse interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { - return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) - } - - return &anthropicResponse, rawResponse, latency, nil + var rawResponse interface{} + if provider.sendBackRawResponse { + if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { + return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) + } + } + return &anthropicResponse, rawResponse, latency, nilAlso applies to: 864-865
828-833: Optional: use Accept header for GET and drop Content-TypeContent-Type on GET is unnecessary; consider setting Accept: application/json instead.
core/providers/openai.go (2)
1831-1849: Use provider-specific type + converter for consistency and resilience.You’re unmarshalling directly into BifrostListModelsResponse. For consistency with other providers (and to future‑proof fields), parse OpenAI’s native list response and convert to Bifrost.
Apply this diff:
- response := &schemas.BifrostListModelsResponse{} - - // Use enhanced response handler with pre-allocated response - rawResponse, bifrostErr := handleProviderResponse(responseBody, response, sendBackRawResponse) + // Parse provider-specific response then convert to Bifrost + openAIResp := &openai.OpenAIModelListResponse{} + rawResponse, bifrostErr := handleProviderResponse(responseBody, openAIResp, sendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr } - // Set raw response if enabled - if sendBackRawResponse { - response.ExtraFields.RawResponse = rawResponse - } + response := openAIResp.ToBifrostListModelsResponse() + if response == nil { + return nil, newBifrostOperationError("failed to convert OpenAI model list response", nil, providerName) + } + if sendBackRawResponse { + response.ExtraFields.RawResponse = rawResponse + }
1811-1813: Nit: avoid Content-Type on GET.Setting Content-Type for a GET is unnecessary; consider removing to keep headers minimal.
core/schemas/models.go (1)
15-21: Use int64 for timestamp-like and capacity-like fields.Use int64 for Created and ContextLength to avoid 32‑bit overflow and align with typical Unix timestamp/token counts.
- Created *int `json:"created,omitempty"` + Created *int64 `json:"created,omitempty"` ... - ContextLength *int `json:"context_length,omitempty"` + ContextLength *int64 `json:"context_length,omitempty"`
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (34)
core/bifrost.go(5 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(8 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(1 hunks)core/providers/gemini.go(1 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(1 hunks)core/schemas/bifrost.go(1 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(1 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/openrouter/models.go(1 hunks)core/schemas/providers/openrouter/types.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (26)
core/schemas/providers/anthropic/models.go (2)
core/schemas/providers/anthropic/types.go (1)
AnthropicModelListResponse(279-284)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
core/schemas/providers/azure/models.go (2)
core/schemas/providers/azure/types.go (1)
AzureModelListResponse(28-31)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
core/schemas/providers/mistral/models.go (2)
core/schemas/providers/mistral/types.go (1)
MistralModelListResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
core/providers/openai.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (5)
BifrostError(318-327)OpenAI(35-35)ListModelsRequest(93-93)ModelProvider(32-32)RequestType(79-79)core/schemas/logger.go (1)
Logger(28-55)core/schemas/provider.go (1)
Provider(194-221)
core/schemas/providers/bedrock/models.go (2)
core/schemas/providers/bedrock/types.go (1)
BedrockModelListResponse(452-454)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (1)
BifrostListModelsRequest(3-5)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (2)
ModelProvider(32-32)ListModelsRequest(93-93)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-237)
core/providers/openrouter.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (4)
BifrostError(318-327)RequestType(79-79)ListModelsRequest(93-93)OpenRouter(48-48)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/openrouter/types.go (1)
OpenRouterModelListResponse(94-96)
core/providers/parasail.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/schemas/providers/cohere/models.go (2)
core/schemas/providers/cohere/types.go (1)
CohereModelListResponse(539-542)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(12-29)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (2)
BifrostError(318-327)Groq(43-43)
core/providers/anthropic.go (4)
core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (5)
BifrostError(318-327)Anthropic(37-37)ChatCompletionRequest(84-84)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/anthropic/types.go (2)
AnthropicModelListResponse(279-284)AnthropicError(301-307)
core/providers/ollama.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/providers/azure.go (6)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(27-31)core/schemas/bifrost.go (4)
BifrostError(318-327)Azure(36-36)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(194-221)core/utils.go (1)
Ptr(12-14)core/schemas/models.go (3)
Model(12-29)BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/providers/azure/types.go (1)
AzureModelListResponse(28-31)
core/providers/sgl.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/schemas/providers/openrouter/models.go (2)
core/schemas/providers/openrouter/types.go (1)
OpenRouterModelListResponse(94-96)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
core/providers/cohere.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (6)
BifrostError(318-327)Cohere(39-39)ChatCompletionRequest(84-84)RequestType(79-79)ListModelsRequest(93-93)ErrorField(336-343)core/schemas/provider.go (1)
Provider(194-221)core/schemas/providers/cohere/types.go (2)
CohereModelListResponse(539-542)CohereError(521-525)
core/schemas/provider.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/providers/vertex.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/bifrost.go (4)
core/schemas/account.go (2)
Account(59-75)Key(8-17)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (5)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(318-327)ErrorField(336-343)ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)
core/providers/mistral.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (4)
BifrostError(318-327)RequestType(79-79)ListModelsRequest(93-93)Mistral(41-41)core/schemas/provider.go (1)
Provider(194-221)core/schemas/providers/mistral/types.go (1)
MistralModelListResponse(31-34)
core/providers/bedrock.go (5)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(45-52)core/schemas/bifrost.go (6)
BifrostError(318-327)ErrorField(336-343)RequestCancelled(278-278)Bedrock(38-38)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(194-221)core/schemas/models.go (3)
Model(12-29)BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/providers/bedrock/types.go (1)
BedrockModelListResponse(452-454)
core/schemas/providers/gemini/models.go (2)
core/schemas/providers/gemini/types.go (1)
GeminiModelListResponse(1356-1359)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-29)
core/providers/gemini.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (4)
BifrostError(318-327)Gemini(47-47)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/gemini/types.go (1)
GeminiModelListResponse(1356-1359)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(250-258)
🔇 Additional comments (26)
core/schemas/bifrost.go (1)
93-93: LGTM! ListModelsRequest constant added correctly.The new constant follows the existing naming convention and integrates cleanly with the RequestType enum.
core/schemas/providers/azure/models.go (1)
5-23: LGTM! Azure model conversion implemented correctly.The nil check, field mappings, and slice initialization are all appropriate. The conversion properly maps Azure-specific fields to the standardized Bifrost model format.
core/schemas/providers/cohere/types.go (1)
527-542: LGTM! Cohere model types properly defined.The struct definitions include appropriate JSON tags and field types for modeling the Cohere Models API response.
core/schemas/providers/openai/types.go (1)
124-139: LGTM! OpenAI model types defined correctly.The struct definitions are appropriate with proper JSON tags. The comment noting GROQ-specific fields (
ActiveandContextWindow) is helpful documentation, as multiple providers may use OpenAI-compatible response formats.core/schemas/providers/mistral/types.go (1)
1-34: LGTM! Mistral model types properly structured.The type definitions correctly model the Mistral Models API with appropriate JSON tags and nested structures for capabilities.
core/schemas/providers/azure/types.go (1)
1-31: LGTM! Azure model types well-defined.The struct definitions appropriately model Azure's API response with proper nesting for capabilities and deprecation metadata, and correct use of optional fields.
core/providers/openrouter.go (1)
185-206: LGTM! ListModels implementation follows the established pattern.The method correctly converts the provider response, sets extra fields, and handles raw responses when enabled.
core/providers/cohere.go (2)
825-886: Overall implementation looks solid.The helper method follows the established pattern with proper resource management (fasthttp acquire/release), error handling, and response parsing. Once the endpoint version is confirmed, this will be good to go.
837-837: No changes required—endpoint is correct per official Cohere API documentation.The
/v1/modelsendpoint is confirmed as correct for listing models in the Cohere API. While the Cohere provider does use/v2/endpoints elsewhere (chat, embed), this version inconsistency reflects the actual Cohere API structure where different endpoints exist at different versions. The code at line 837 is correct as written.core/schemas/provider.go (1)
219-220: LGTM! Clean interface extension.The ListModels method addition follows the established pattern of other Provider interface methods with appropriate context, key, and request parameters. The method signature is consistent and well-documented.
core/providers/sgl.go (1)
199-202: LGTM! Consistent with provider's unsupported operations.The ListModels implementation correctly returns an unsupported operation error, matching the pattern used by other unsupported operations (Speech, SpeechStream, Transcription, etc.) in the SGL provider.
core/providers/cerebras.go (1)
183-186: LGTM! Proper delegation to shared OpenAI handler.The ListModels implementation correctly delegates to the shared handleOpenAIModelListRequest helper, passing all necessary parameters including the proper endpoint URL and provider context.
core/providers/vertex.go (1)
636-639: LGTM! Consistent with provider's unsupported operations.The ListModels implementation correctly returns an unsupported operation error, aligning with the pattern used by other unsupported methods (Speech, SpeechStream, Transcription, etc.) in the Vertex provider.
core/providers/groq.go (1)
224-236: LGTM! Proper delegation to shared OpenAI handler.The ListModels implementation correctly delegates to the shared handleOpenAIModelListRequest helper with all necessary parameters, following the established pattern for OpenAI-compatible providers.
core/providers/ollama.go (1)
199-202: LGTM! Proper delegation to shared OpenAI handler.The ListModels implementation correctly delegates to handleOpenAIModelListRequest, which is appropriate since Ollama provides an OpenAI-compatible API. All parameters are properly passed.
core/providers/parasail.go (1)
166-169: LGTM! Consistent with provider's unsupported operations.The ListModels implementation correctly returns an unsupported operation error, maintaining consistency with other unsupported operations in the Parasail provider.
core/schemas/providers/gemini/types.go (1)
1339-1359: LGTM—struct definitions match Gemini API specification.The GeminiModel and GeminiModelListResponse types correctly implement the Google Gemini API models.list response format, with all 13 Model object fields and both top-level response fields (models array and nextPageToken) properly mapped to their corresponding types. No unmarshaling errors should occur.
core/schemas/providers/anthropic/types.go (1)
6-6: Confirm Anthropic created_at wire format matches time.TimeIf Anthropic returns non-RFC3339 (e.g., epoch seconds), json unmarshalling into time.Time will fail. Please confirm the provider payload and adjust (string/int + custom parse) if needed.
Also applies to: 271-285
transports/bifrost-http/handlers/inference.go (1)
277-279: Route registration looks goodGET /v1/models wired with existing middleware chain.
core/providers/gemini.go (1)
946-986: OK to parse directly before releasing fasthttp.ResponseUnmarshal happens before Release; logic and headers are fine.
core/schemas/providers/anthropic/models.go (1)
18-19: Potential 32-bit overflow converting Unix time to intint(model.CreatedAt.Unix()) can overflow on 32-bit platforms. Consider using int64 end-to-end for Created, or guard via a clamped conversion.
Would you like a follow-up PR to switch Model.Created to *int64 across schemas?
core/providers/anthropic.go (1)
800-814: Sanity: nil-safety on conversion resultIf anthropicResponse is unexpectedly nil, response.ToBifrost... would panic. Not observed now, but add a guard.
- response := anthropicResponse.ToBifrostListModelsResponse() + if anthropicResponse == nil { + return nil, newBifrostOperationError("nil provider response", nil, provider.GetProviderKey()) + } + response := anthropicResponse.ToBifrostListModelsResponse()core/schemas/providers/bedrock/types.go (1)
439-454: Bedrock model types look good.Field names/tags align with Bedrock’s model summaries; ready to use with the converter.
core/schemas/providers/openrouter/types.go (1)
3-96: Schema is clear and matches OpenRouter’s docs.Good coverage of pricing, architecture, and limits; no issues.
core/providers/azure.go (1)
442-470: Azure ListModels implementation LGTM.Non‑deployment path, conversion, and extra_fields are correct; raw response gated by flag.
core/providers/bedrock.go (1)
961-999: Bedrock ListModels implementation LGTM.Correct service endpoint (non‑runtime), conversion to Bifrost, and metadata population.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
2c910e9 to
0b3bca9
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/providers/bedrock.go (1)
200-205: Fix error message: reading response, not request.Minor but confusing when debugging.
Apply:
- Message: "error reading request", + Message: "error reading response",
♻️ Duplicate comments (3)
core/providers/azure.go (1)
94-99: Outdated/duplicated Azure API-version default; centralize to avoid drift.Defaulting to "2024-02-01" here and elsewhere is inconsistent with schema docs and may break non-deployment endpoints in some regions. Define one constant and reference it in all call sites.
Apply:
@@ -const AzureAuthorizationTokenKey schemas.BifrostContextKey = "azure-authorization-token" +const AzureAuthorizationTokenKey schemas.BifrostContextKey = "azure-authorization-token" +const defaultAzureAPIVersion = "2024-10-21" // central default @@ - apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr(defaultAzureAPIVersion) + } @@ - apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr(defaultAzureAPIVersion) + } @@ - apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr(defaultAzureAPIVersion) + }What is the latest stable data‑plane API version for Azure OpenAI “openai/models” (resource‑level) as of October 2025?Also applies to: 226-231, 322-327
core/bifrost.go (2)
233-262: Key selection for ListModels incorrectly filters valid keys.Passing an empty model string to
selectKeyFromProviderForModel(line 247) causes the selector to incorrectly filter out keys with explicit Models lists or Azure/Bedrock deployment mappings. At lines 2056-2074, the selector checksslices.Contains(key.Models, model)and deployment mappings—withmodel="", these checks will fail for otherwise valid keys.The selector should treat empty model as a non-model-specific operation and skip model/deployment filtering. Apply the fix suggested in the previous review or introduce a dedicated selector that bypasses these checks for ListModels.
1097-1106: Race condition on providers append—use CAS loop.The load-modify-store pattern here is not atomic. If two concurrent
prepareProvidercalls for different providers execute, one append can be lost. Use a compare-and-swap loop as suggested in the previous review to ensure all provider additions are preserved.Apply the CAS-loop fix from the previous review comment to make the provider list updates atomic.
🧹 Nitpick comments (7)
core/schemas/providers/openrouter/models.go (1)
10-12: Consider preallocating Data slice for efficiency.Since you know the final length, preallocating
Dataavoids incremental growth:bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, + Data: make([]schemas.Model, 0, len(response.Data)), }core/schemas/providers/cohere/models.go (1)
12-14: Consider preallocating Data slice for efficiency.Preallocating with the correct capacity avoids incremental growth:
bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, + Data: make([]schemas.Model, 0, len(response.Models)), }core/schemas/providers/gemini/models.go (1)
10-12: Consider preallocating Data slice for efficiency.You can preallocate with the known capacity:
bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, + Data: make([]schemas.Model, 0, len(response.Models)), }transports/bifrost-http/handlers/inference.go (1)
291-301: Normalize provider input to reduce 400s from trivial user input.Trim/lowercase query value before use to avoid case/whitespace issues.
Apply:
- provider := string(ctx.QueryArgs().Peek("provider")) + provider := strings.ToLower(strings.TrimSpace(string(ctx.QueryArgs().Peek("provider"))))core/schemas/providers/bedrock/models.go (1)
10-12: Preallocate slice for fewer allocs.Small perf win on large model lists.
Apply:
- bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, - } + bifrostResponse := &schemas.BifrostListModelsResponse{ + Data: make([]schemas.Model, 0, len(response.ModelSummaries)), + }Also applies to: 14-24
core/providers/bedrock.go (1)
106-120: Unreachable context.DeadlineExceeded check during JSON marshal.sonic.Marshal doesn’t use context; this branch won’t trigger. Simplify and, if needed, check ctx.Err() separately.
Apply:
- if err != nil { - if errors.Is(err, context.DeadlineExceeded) { - return nil, 0, &schemas.BifrostError{ /* ... */ } - } - return nil, 0, &schemas.BifrostError{ - IsBifrostError: true, - Error: &schemas.ErrorField{ - Message: schemas.ErrProviderJSONMarshaling, - Error: err, - }, - } - } + if err != nil { + return nil, 0, &schemas.BifrostError{ + IsBifrostError: true, + Error: &schemas.ErrorField{ + Message: schemas.ErrProviderJSONMarshaling, + Error: err, + }, + } + }core/schemas/providers/openai/models.go (1)
10-12: Preallocate slice for better performance.The type alignment is already correct (model.ContextWindow is *int, matching schemas.Model.ContextLength). However, the slice should be preallocated to avoid reallocations during the loop:
- bifrostResponse := &schemas.BifrostListModelsResponse{ - Data: []schemas.Model{}, - } + bifrostResponse := &schemas.BifrostListModelsResponse{ + Data: make([]schemas.Model, 0, len(response.Data)), + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (35)
core/bifrost.go(5 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(8 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(2 hunks)core/providers/gemini.go(1 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(1 hunks)core/schemas/bifrost.go(1 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(1 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/openrouter/models.go(1 hunks)core/schemas/providers/openrouter/types.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (17)
- core/schemas/provider.go
- core/schemas/providers/openrouter/types.go
- core/providers/cerebras.go
- core/providers/ollama.go
- core/providers/mistral.go
- core/schemas/providers/gemini/types.go
- core/schemas/providers/anthropic/types.go
- core/schemas/providers/openai/types.go
- core/providers/gemini.go
- core/providers/anthropic.go
- core/schemas/providers/mistral/models.go
- core/schemas/providers/mistral/types.go
- core/schemas/providers/cohere/types.go
- core/providers/parasail.go
- core/schemas/providers/anthropic/models.go
- core/providers/vertex.go
- core/schemas/providers/azure/models.go
🧰 Additional context used
🧬 Code graph analysis (16)
core/schemas/providers/gemini/models.go (2)
core/schemas/providers/gemini/types.go (1)
GeminiModelListResponse(1356-1359)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-28)
core/schemas/providers/bedrock/models.go (2)
core/schemas/providers/bedrock/types.go (1)
BedrockModelListResponse(452-454)core/schemas/models.go (3)
BifrostListModelsResponse(7-10)Model(12-28)Architecture(30-36)
core/providers/azure.go (6)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(27-31)core/schemas/bifrost.go (4)
BifrostError(318-327)Azure(36-36)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(194-221)core/utils.go (1)
Ptr(12-14)core/schemas/models.go (3)
Model(12-28)BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/providers/azure/types.go (1)
AzureModelListResponse(28-31)
core/schemas/providers/openai/models.go (2)
core/schemas/providers/openai/types.go (1)
OpenAIModelListResponse(136-139)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-28)
core/bifrost.go (4)
core/schemas/account.go (2)
Account(59-75)Key(8-17)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (5)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(318-327)ErrorField(336-343)ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (4)
BifrostError(318-327)ModelProvider(32-32)RequestType(79-79)ListModelsRequest(93-93)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIModelListResponse(136-139)core/schemas/provider.go (1)
Provider(194-221)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (1)
BifrostListModelsRequest(3-5)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (2)
ModelProvider(32-32)ListModelsRequest(93-93)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-237)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (2)
BifrostError(318-327)Groq(43-43)
core/providers/cohere.go (4)
core/schemas/models.go (3)
Model(12-28)BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (5)
RequestType(79-79)ChatCompletionRequest(84-84)BifrostError(318-327)ListModelsRequest(93-93)ErrorField(336-343)core/schemas/providers/cohere/types.go (2)
CohereModelListResponse(539-542)CohereError(521-525)
core/providers/sgl.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (1)
BifrostError(318-327)
core/schemas/providers/openrouter/models.go (2)
core/schemas/providers/openrouter/types.go (1)
OpenRouterModelListResponse(94-96)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-28)
core/schemas/models.go (4)
core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(250-258)core/schemas/providers/openrouter/types.go (2)
Architecture(39-51)TopProvider(82-91)core/schemas/providers/gemini/types.go (1)
Modality(700-700)
core/providers/bedrock.go (5)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(45-52)core/schemas/bifrost.go (5)
BifrostError(318-327)ErrorField(336-343)RequestCancelled(278-278)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(194-221)core/schemas/models.go (3)
Model(12-28)BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/providers/bedrock/types.go (1)
BedrockModelListResponse(452-454)
core/schemas/providers/cohere/models.go (2)
core/schemas/providers/cohere/types.go (1)
CohereModelListResponse(539-542)core/schemas/models.go (2)
BifrostListModelsResponse(7-10)Model(12-28)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(12-28)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-5)BifrostListModelsResponse(7-10)core/schemas/bifrost.go (4)
BifrostError(318-327)RequestType(79-79)ListModelsRequest(93-93)OpenRouter(48-48)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)
🔇 Additional comments (16)
core/schemas/bifrost.go (1)
93-93: LGTM!The new
ListModelsRequestconstant follows the established pattern for RequestType values and integrates cleanly with the existing enum.core/schemas/providers/openrouter/models.go (1)
14-22: Pointer capture bug resolved correctly.The use of
schemas.Ptr()helpers correctly avoids the pointer aliasing issue flagged in the previous review. Each model now gets its own distinct pointers.core/schemas/providers/cohere/models.go (1)
16-23: Pointer capture bug resolved correctly.The use of
schemas.Ptr()helpers correctly resolves the pointer aliasing issue from the previous review.core/providers/openrouter.go (1)
217-217: Authentication header correctly uses Authorization Bearer.The code now correctly uses
Authorization: Bearerper the previous review feedback.core/schemas/providers/azure/types.go (1)
1-31: LGTM!The Azure model types are well-structured with appropriate JSON tags and follow the established pattern used by other provider-specific types in the codebase.
core/schemas/models.go (1)
1-53: LGTM!The unified model schema definitions are well-designed with appropriate use of pointers for optional fields, consistent JSON tagging, and clear structure that supports the ListModels API across multiple providers.
core/schemas/providers/gemini/models.go (1)
14-23: Pointer capture bug resolved correctly.The code correctly creates a fresh
contextLengthvariable in each iteration and wraps it withschemas.Ptr(), avoiding the pointer aliasing issue flagged in the previous review.core/providers/sgl.go (1)
199-202: LGTM!The ListModels implementation correctly returns an unsupported operation error, consistent with other unsupported operations in the SGL provider.
transports/bifrost-http/handlers/inference.go (1)
277-279: Route wiring looks correct.GET /v1/models is properly chained through middleware.
core/providers/groq.go (1)
224-236: Good reuse of OpenAI helper for Groq ListModels.URL, headers, provider metadata, and raw-response handling are consistent.
core/providers/openai.go (1)
1780-1785: ListModels path and helper look solid.GET setup, error handling, conversion, and ExtraFields all align with existing provider patterns.
Also applies to: 1787-1847
core/providers/bedrock.go (2)
393-394: Callsite updates to new completeRequest signature look correct.POST vs GET and runtime flag usage are appropriate.
Also applies to: 469-469, 660-660, 868-868, 875-876
958-995: Bedrock ListModels path is correct.Uses control-plane endpoint (/foundation-models), converts, and sets ExtraFields cleanly.
core/providers/azure.go (1)
439-470: Azure ListModels flow looks good.Uses non-deployment URL, parses provider response, and populates ExtraFields.
core/schemas/providers/bedrock/types.go (1)
434-454: LGTM!The new model types are well-structured with appropriate fields and JSON tags that align with AWS Bedrock's API schema for model listing.
core/bifrost.go (1)
1160-1175: LGTM!The
getProviderByKeyhelper is cleanly implemented with proper thread-safe access via atomic.Load and straightforward linear search logic.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
0b3bca9 to
9c368ed
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (5)
core/providers/openrouter.go (1)
231-241: Critical: Bypassing OpenRouter-specific types and converter.This code directly unmarshals into
schemas.BifrostListModelsResponse, bypassing the OpenRouter-specificOpenRouterModelListResponsetype and itsToBifrostListModelsResponseconverter method. This is inconsistent with the pattern used by other providers (Azure, Vertex, Cohere, etc.) in this PR.Apply this fix to use OpenRouter-specific types:
+import ( + openrouter "github.com/maximhq/bifrost/core/schemas/providers/openrouter" +) + func (provider *OpenRouterProvider) handleOpenRouterModelListRequest(ctx context.Context, key schemas.Key) (*schemas.BifrostListModelsResponse, interface{}, time.Duration, *schemas.BifrostError) { // ... (request setup code remains the same) - var openrouterResponse schemas.BifrostListModelsResponse + var openrouterResponse openrouter.OpenRouterModelListResponse if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } var rawResponse interface{} if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } - return &openrouterResponse, rawResponse, latency, nil + bifrostResponse := openrouterResponse.ToBifrostListModelsResponse() + return bifrostResponse, rawResponse, latency, nil }core/providers/cohere.go (1)
799-805: Decision: enforce operation allowlist for ListModels or intentionally omit?Earlier review asked to add checkOperationAllowed for Cohere ListModels; other providers (Azure/Vertex/Anthropic in this PR) omit it. Please confirm the intended policy and align across providers.
#!/bin/bash # Verify consistency of allowlist checks for ListModels across providers rg -nP --type go -C2 'func\s+\(\s*\w+\s+\*\w+Provider\)\s+ListModels\(' core/providers | cut -d: -f1 | sort -u | while read f; do echo "---- $f" rg -n 'checkOperationAllowed\(.+ListModelsRequest' "$f" -n -C1 || echo "no allowlist check" donecore/providers/azure.go (1)
95-99: Default API version mismatches schema docs; update and centralize.Schema comment says default is "2024-08-01-preview" (core/schemas/account.go). Code uses "2024-02-01". Align to avoid surprises; consider centralizing in one const to prevent drift.
- apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr("2024-08-01-preview") // match AzureKeyConfig docs + }Follow-up: also unify other call sites that embed api-version (e.g., streaming helpers) to reuse this default. Based on learnings.
core/bifrost.go (2)
233-262: Key selection for ListModels can incorrectly filter out valid keys (Azure/Bedrock and model-scoped keys).This critical issue was flagged in a previous review. Passing an empty model (
"") toselectKeyFromProviderForModelat line 247 filters bykey.Modelsand deployment mappings. Keys with explicit Models lists or Azure/Bedrock Deployments will be excluded, breaking ListModels for otherwise valid keys.
1097-1106: Possible lost updates when appending to providers slice; use CAS loop.This major issue was flagged in a previous review. Two concurrent
prepareProvidercalls for different providers can race and lose one append. A compare-and-swap loop is needed to atomically append.
🧹 Nitpick comments (7)
core/schemas/providers/gemini/models.go (1)
15-24: Consider adding Object field for consistency.While not required, setting
Object: "model"in the schemas.Model would align with OpenAI's standard response format and improve cross-provider consistency.for _, model := range response.Models { contextLength := model.InputTokenLimit + model.OutputTokenLimit bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ ID: model.Name, + Object: "model", Name: schemas.Ptr(model.DisplayName), Description: schemas.Ptr(model.Description), ContextLength: schemas.Ptr(int(contextLength)), SupportedMethods: model.SupportedGenerationMethods, }) }core/schemas/providers/anthropic/models.go (1)
18-24: Consider adding Object field for API consistency.While the pointer bug from the previous review has been fixed, the
Objectfield is still unset. Setting it to"model"would align with OpenAI's standard format and improve cross-provider consistency.for _, model := range response.Data { bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ ID: model.ID, + Object: "model", Name: schemas.Ptr(model.DisplayName), Created: schemas.Ptr(int(model.CreatedAt.Unix())), }) }core/providers/anthropic.go (3)
825-846: Use encoded query builder instead of manual string concatenation.Avoid injection/encoding bugs for before_id/after_id; set query args via fasthttp.
- // Build URL with query parameters - queryParams := "" - // Add limit parameter (default to 20) - limit := 20 - if request.Limit != nil { - limit = *request.Limit - } - queryParams += fmt.Sprintf("limit=%d", limit) - // Add cursor-based pagination parameters - if request.ExtraParams != nil { - // before id - if beforeID, ok := request.ExtraParams["before_id"].(string); ok { - queryParams += fmt.Sprintf("&before_id=%s", beforeID) - } - // after id - if afterID, ok := request.ExtraParams["after_id"].(string); ok { - queryParams += fmt.Sprintf("&after_id=%s", afterID) - } - } - req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models?" + queryParams) + // Build URL with encoded query parameters + req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models") + args := req.URI().QueryArgs() + limit := 20 + if request.Limit != nil { + limit = *request.Limit + } + args.Set("limit", fmt.Sprintf("%d", limit)) + if request.ExtraParams != nil { + if beforeID, ok := request.ExtraParams["before_id"].(string); ok && beforeID != "" { + args.Set("before_id", beforeID) + } + if afterID, ok := request.ExtraParams["after_id"].(string); ok && afterID != "" { + args.Set("after_id", afterID) + } + }Add if missing at top:
import "fmt"
877-883: Parse raw response only when needed.Skip second unmarshal unless sendBackRawResponse is enabled. Saves CPU/allocs.
- var rawResponse interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { - return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) - } + var rawResponse interface{} + if provider.sendBackRawResponse { + if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { + return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) + } + }
847-851: Drop Content-Type on GET.Not required and sometimes confusing to proxies; consider removing.
core/providers/vertex.go (1)
789-795: Gate raw-response parsing on sendBackRawResponse.Avoid extra unmarshal when raw response isn’t requested.
- var rawResponse interface{} - if err := sonic.Unmarshal(body, &rawResponse); err != nil { - return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) - } + var rawResponse interface{} + if provider.sendBackRawResponse { + if err := sonic.Unmarshal(body, &rawResponse); err != nil { + return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, providerName) + } + }core/providers/cohere.go (1)
830-849: Build query with encoding; avoid manual concatenation.Encode page_token/endpoint/default_only safely; simpler and robust.
- queryParams := "" - limit := 20 - if request.Limit != nil { - limit = *request.Limit - } - queryParams += fmt.Sprintf("page_size=%d", limit) - - if request.PageToken != nil { - queryParams += fmt.Sprintf("&page_token=%s", *request.PageToken) - } - if endpoint, ok := request.ExtraParams["endpoint"].(string); ok && endpoint != "" { - queryParams += fmt.Sprintf("&endpoint=%s", endpoint) - } - if defaultOnly, ok := request.ExtraParams["default_only"].(bool); ok && defaultOnly { - queryParams += fmt.Sprintf("&default_only=%t", defaultOnly) - } - - req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models?" + queryParams) + u, _ := url.Parse(provider.networkConfig.BaseURL + "/v1/models") + q := u.Query() + limit := 20 + if request.Limit != nil { + limit = *request.Limit + } + q.Set("page_size", fmt.Sprintf("%d", limit)) + if request.PageToken != nil && *request.PageToken != "" { + q.Set("page_token", *request.PageToken) + } + if endpoint, ok := request.ExtraParams["endpoint"].(string); ok && endpoint != "" { + q.Set("endpoint", endpoint) + } + if defaultOnly, ok := request.ExtraParams["default_only"].(bool); ok && defaultOnly { + q.Set("default_only", "true") + } + u.RawQuery = q.Encode() + req.SetRequestURI(u.String())Add imports if missing:
import ( + "net/url" + "fmt" )
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (35)
core/bifrost.go(5 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(8 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(2 hunks)core/providers/gemini.go(1 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(3 hunks)core/schemas/bifrost.go(3 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(1 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (12)
- core/schemas/provider.go
- core/providers/sgl.go
- core/schemas/providers/cohere/models.go
- core/schemas/providers/anthropic/types.go
- core/schemas/providers/azure/types.go
- core/providers/ollama.go
- core/schemas/providers/bedrock/models.go
- core/schemas/providers/mistral/models.go
- core/schemas/models.go
- core/schemas/providers/gemini/types.go
- core/providers/openai.go
- core/schemas/providers/mistral/types.go
🧰 Additional context used
🧬 Code graph analysis (20)
core/schemas/providers/azure/models.go (2)
core/schemas/providers/azure/types.go (1)
AzureModelListResponse(28-31)core/schemas/models.go (2)
BifrostListModelsResponse(16-21)Model(23-39)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/schemas/providers/anthropic/models.go (2)
core/schemas/providers/anthropic/types.go (2)
AnthropicModelListResponse(279-284)AnthropicModel(272-277)core/schemas/models.go (2)
BifrostListModelsResponse(16-21)Model(23-39)
core/providers/anthropic.go (3)
core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (1)
ListModelsRequest(93-93)core/schemas/providers/anthropic/types.go (2)
AnthropicModelListResponse(279-284)AnthropicError(301-307)
core/providers/groq.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (2)
BifrostError(320-329)Groq(43-43)
core/providers/vertex.go (5)
core/schemas/models.go (3)
Model(23-39)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(35-39)core/schemas/providers/vertex/types.go (1)
VertexModelListResponse(59-62)core/schemas/bifrost.go (4)
BifrostError(320-329)ListModelsRequest(93-93)ErrorField(338-345)RequestCancelled(280-280)core/schemas/provider.go (4)
Provider(194-221)ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(45-52)core/schemas/bifrost.go (4)
BifrostError(320-329)ErrorField(338-345)RequestCancelled(280-280)ListModelsRequest(93-93)core/schemas/utils.go (1)
Ptr(14-16)core/schemas/provider.go (1)
Provider(194-221)core/schemas/models.go (3)
Model(23-39)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/providers/bedrock/types.go (1)
BedrockModelListResponse(452-454)
core/schemas/bifrost.go (1)
core/schemas/models.go (1)
BifrostListModelsRequest(3-14)
core/providers/parasail.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/providers/azure.go (6)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(27-31)core/schemas/bifrost.go (4)
BifrostError(320-329)Azure(36-36)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (1)
Provider(194-221)core/utils.go (1)
Ptr(12-14)core/schemas/models.go (3)
Model(23-39)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/providers/azure/types.go (1)
AzureModelListResponse(28-31)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)OpenRouter(48-48)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)
core/providers/mistral.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)Mistral(41-41)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/mistral/types.go (1)
MistralModelListResponse(31-34)
core/schemas/providers/vertex/models.go (2)
core/schemas/providers/vertex/types.go (1)
VertexModelListResponse(59-62)core/schemas/models.go (2)
BifrostListModelsResponse(16-21)Model(23-39)
core/schemas/providers/openai/models.go (2)
core/schemas/providers/openai/types.go (2)
OpenAIModelListResponse(136-139)OpenAIModel(125-134)core/schemas/models.go (2)
BifrostListModelsResponse(16-21)Model(23-39)
core/schemas/providers/gemini/models.go (2)
core/schemas/providers/gemini/types.go (1)
GeminiModelListResponse(1356-1359)core/schemas/models.go (2)
BifrostListModelsResponse(16-21)Model(23-39)
core/schemas/providers/bedrock/types.go (1)
ui/lib/constants/logs.ts (1)
ProviderName(20-20)
core/bifrost.go (4)
core/schemas/account.go (2)
Account(59-75)Key(8-17)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (5)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(320-329)ErrorField(338-345)ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (1)
BifrostListModelsRequest(3-14)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (2)
ModelProvider(32-32)ListModelsRequest(93-93)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-237)
core/providers/gemini.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/bifrost.go (3)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (2)
Provider(194-221)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/gemini/types.go (1)
GeminiModelListResponse(1356-1359)
core/providers/cohere.go (4)
core/schemas/models.go (3)
Model(23-39)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-21)core/schemas/provider.go (1)
Provider(194-221)core/schemas/bifrost.go (3)
BifrostError(320-329)ListModelsRequest(93-93)ErrorField(338-345)core/schemas/providers/cohere/types.go (2)
CohereModelListResponse(539-542)CohereError(521-525)
🔇 Additional comments (17)
core/providers/cerebras.go (1)
183-186: LGTM! ListModels implementation follows established patterns.The implementation correctly delegates to the shared OpenAI-compatible handler and constructs the appropriate endpoint URL. This is consistent with how other operations (TextCompletion, ChatCompletion) are handled in this provider.
core/providers/groq.go (1)
224-236: LGTM! Well-structured ListModels implementation.The method correctly leverages the shared OpenAI-compatible handler with proper parameter formatting and explicit provider identification. The multi-line formatting enhances readability.
core/schemas/providers/cohere/types.go (1)
527-542: LGTM! Well-defined Cohere model types.The new
CohereModelandCohereModelListResponsetypes appropriately capture Cohere's model metadata, including deprecation status, endpoints, features, and pagination support viaNextPageToken. The field types and JSON tags are correctly defined.core/schemas/providers/openai/types.go (1)
124-139: LGTM! Proper OpenAI model types with provider-specific field documentation.The new types correctly represent the OpenAI model list structure. The comment documenting GROQ-specific fields (
Active,ContextWindow) is helpful for understanding provider variations while maintaining a common structure.core/schemas/providers/vertex/models.go (1)
5-24: LGTM! Well-implemented conversion method with proper guards.The method includes defensive nil checking, efficient slice pre-allocation, and correct field mappings. The Unix timestamp conversion for the
Createdfield appropriately transformsVersionCreateTimeto an integer timestamp.core/providers/parasail.go (1)
166-169: LGTM! Correctly handles unsupported operation.The method appropriately returns an unsupported operation error, consistent with how other unsupported operations (Embedding, Speech, Transcription) are handled in this provider.
core/schemas/providers/azure/models.go (1)
5-22: LGTM! Proper conversion method with efficient implementation.The method includes appropriate nil checking, efficient slice pre-allocation, and correct field mappings from Azure's model structure (
AzureModel) to Bifrost's standardizedModelformat.core/schemas/bifrost.go (1)
93-93: LGTM! Clean integration of ListModels request type.The new ListModelsRequest constant, documentation, and field addition follow the established patterns. The intentional omission from GetRequestFields/SetProvider/SetModel/SetFallbacks is correct since ListModels operates at the provider level without requiring model-specific routing.
Also applies to: 122-123, 133-133
transports/bifrost-http/handlers/inference.go (1)
277-279: LGTM! Route registration follows existing patterns.core/schemas/providers/vertex/types.go (1)
3-3: LGTM! Clean type definitions for Vertex model listing.The new types follow standard patterns with appropriate field types and JSON tags.
Also applies to: 48-62
core/providers/mistral.go (1)
180-201: LGTM! Clean conversion and extra fields setup.core/schemas/providers/openai/models.go (1)
5-54: LGTM! Excellent bidirectional conversion implementation.Both conversion methods handle nil checks, preallocate slices, and properly manage pointer conversions. The code follows best practices throughout.
core/schemas/providers/anthropic/models.go (1)
9-27: LGTM! Pointer bug from previous review has been fixed.Both conversion methods now correctly use
schemas.Ptr()for optional fields and handle nil cases properly.Also applies to: 29-52
core/providers/gemini.go (1)
920-994: LGTM! Issues from previous review have been resolved.The unused parameter and typo mentioned in the past review have been fixed. The implementation now properly uses the
requestparameter for pagination (lines 958-964) and theresponsevariable is correctly named throughout.core/providers/azure.go (1)
443-469: LGTM on Azure ListModels flow.Correctly uses non-deployment path and maps to Bifrost; raw response is gated.
core/schemas/providers/bedrock/types.go (1)
434-454: Bedrock model types addition looks good.Fields and tags align; safe to proceed.
core/providers/bedrock.go (1)
958-995: LGTM! ListModels implementation is correct.The implementation correctly:
- Uses the
bedrockservice endpoint (notbedrock-runtime) for the/foundation-modelslist operation- Employs a GET request with no body
- Handles response parsing and conversion to the Bifrost format
- Populates extra fields (Provider, Latency, RequestType, RawResponse)
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
5 times, most recently
from
fb009ed to
78425c1
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/providers/bedrock.go (1)
158-167: Address Bearer token authentication in both Bedrock request functions; include makeStreamingRequest.Standard Bedrock data plane requires AWS Signature Version 4, not Bearer tokens. The review comment correctly identifies the issue at lines 158-167 in
completeRequest(), but the same problematic pattern appears again at lines 267-269 inmakeStreamingRequest(). Both functions checkif key.Value != ""and use Bearer auth; both should always use SigV4 signing for standard Bedrock endpoints (bedrock-runtime).Update the suggested fix to cover both locations:
completeRequest()lines 158-167makeStreamingRequest()lines 266-272 (remove the Bearer branch, always sign with SigV4)
♻️ Duplicate comments (4)
core/providers/vertex.go (2)
637-658: Pagination parameters are ignored; handler signature missing request parameter.The
ListModelsmethod receives arequestparameter withLimitandPageTokenfields (line 638), but passes nothing to the handler (line 639). The handler signature (line 660) doesn't accept the request, so pagination cannot be implemented. This breaks the pagination contract defined inBifrostListModelsRequest.As per coding guidelines (from past review comments), update the handler signature and pass pagination to the Vertex API:
-func (provider *VertexProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - vertexResponse, rawResponse, latency, err := provider.handleVertexListModelsRequest(ctx, key) +func (provider *VertexProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { + vertexResponse, rawResponse, latency, err := provider.handleVertexListModelsRequest(ctx, key, request)And update the handler to accept and use pagination parameters (see past review comment for full implementation).
660-795: Implement pagination query parameters.The handler constructs the URL without query parameters (line 678), ignoring
pageSizeandpageTokenthat Vertex AI's publishers.models.list endpoint supports. This prevents clients from paginating through model lists.As confirmed by past review web search, Vertex AI uses
pageSizeandpageTokenquery parameters. Add URL encoding and query parameter handling as detailed in the past review comment above.core/providers/openrouter.go (1)
231-242: Use OpenRouter-specific types + converter instead of Bifrost directly.Unmarshalling directly into schemas.BifrostListModelsResponse bypasses provider-specific parsing and existing converter. Unmarshal into openrouter.OpenRouterModelListResponse and convert via ToBifrostListModelsResponse. Also add the missing import.
Apply:
@@ -import ( +import ( "context" "fmt" "net/http" "strings" "time" "github.com/bytedance/sonic" schemas "github.com/maximhq/bifrost/core/schemas" + openrouter "github.com/maximhq/bifrost/core/schemas/providers/openrouter" "github.com/valyala/fasthttp" ) @@ - var openrouterResponse schemas.BifrostListModelsResponse - if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { + var openrouterResponse openrouter.OpenRouterModelListResponse + if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } @@ - return &openrouterResponse, rawResponse, latency, nil + bifrostResponse := openrouterResponse.ToBifrostListModelsResponse() + return bifrostResponse, rawResponse, latency, nilcore/providers/azure.go (1)
95-99: Outdated, duplicated default API version; centralize and update.Defaulting to "2024-02-01" in three places risks drift and incompatibility with non-deployment endpoints. Centralize a single default (current stable) and reuse everywhere.
Apply (local constant; consider moving to shared schemas for reuse):
@@ package providers @@ import ( "context" "fmt" "net/http" "time" @@ ) + +const defaultAzureAPIVersion = "2024-10-21" // TODO: centralize in schemas and keep updated @@ - apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr(defaultAzureAPIVersion) + } @@ - apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr(defaultAzureAPIVersion) + } @@ - apiVersion := key.AzureKeyConfig.APIVersion - if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") - } + apiVersion := key.AzureKeyConfig.APIVersion + if apiVersion == nil { + apiVersion = schemas.Ptr(defaultAzureAPIVersion) + }To confirm the latest recommended data-plane API version for "models", please verify:
What is the latest stable Azure OpenAI data-plane api-version that supports GET /openai/models as of Oct 2025?Also applies to: 226-229, 323-325
🧹 Nitpick comments (10)
core/schemas/models.go (1)
3-22: Consider using pointer types for Limit and PageToken fields.The
Limit(int) andPageToken(string) fields inBifrostListModelsRequestare not pointer types, which means they cannot distinguish between "not set" and "zero value". This could cause issues when:
Limitis 0: Is it intentionally 0 or not set?PageTokenis "": Is it the first page or an empty string token?This inconsistency is evident in the past review comments on vertex.go, where the handler needs to check
request != nil && request.Limit != nil.Consider updating to pointer types for nullable semantics:
type BifrostListModelsRequest struct { Provider ModelProvider `json:"provider"` // Internal fields - Limit int `json:"-"` + Limit *int `json:"-"` // PageToken: Token received from previous request to retrieve next page - PageToken string `json:"-"` + PageToken *string `json:"-"` // ExtraParams: Additional provider-specific query parameters // This allows for flexibility to pass any custom parameters that specific providers might support ExtraParams map[string]interface{} `json:"-"` } type BifrostListModelsResponse struct { Data []Model `json:"data"` ExtraFields BifrostResponseExtraFields `json:"extra_fields"` - NextPageToken string `json:"-"` // Token to retrieve next page + NextPageToken *string `json:"-"` // Token to retrieve next page }core/schemas/providers/azure/types.go (1)
3-9: Consider using pointer types for AzureModelCapabilities fields.Using non-pointer bool fields means you cannot distinguish between "capability explicitly false" vs "capability not provided by API". If Azure's API omits capabilities that are false, the current structure will incorrectly report them as false rather than unknown.
If the API omits false capabilities:
type AzureModelCapabilities struct { - FineTune bool `json:"fine_tune"` - Inference bool `json:"inference"` - Completion bool `json:"completion"` - ChatCompletion bool `json:"chat_completion"` - Embeddings bool `json:"embeddings"` + FineTune *bool `json:"fine_tune,omitempty"` + Inference *bool `json:"inference,omitempty"` + Completion *bool `json:"completion,omitempty"` + ChatCompletion *bool `json:"chat_completion,omitempty"` + Embeddings *bool `json:"embeddings,omitempty"` }core/schemas/providers/gemini/types.go (1)
1340-1359: Consider pointer types for optional GeminiModel fields.The
GeminiModeltype uses non-pointer types for all fields, which means zero values (0, false, "") are indistinguishable from missing/unset values. This could cause issues when the Gemini API omits optional fields.If the Gemini API omits optional fields (e.g.,
thinking,temperaturesettings), consider using pointers for optional fields:type GeminiModel struct { Name string `json:"name"` - BaseModelId string `json:"baseModelId"` + BaseModelId *string `json:"baseModelId,omitempty"` Version string `json:"version"` DisplayName string `json:"displayName"` - Description string `json:"description"` + Description *string `json:"description,omitempty"` InputTokenLimit int `json:"inputTokenLimit"` OutputTokenLimit int `json:"outputTokenLimit"` SupportedGenerationMethods []string `json:"supportedGenerationMethods"` - Thinking bool `json:"thinking"` - Temperature float64 `json:"temperature"` - MaxTemperature float64 `json:"maxTemperature"` - TopP float64 `json:"topP"` - TopK int `json:"topK"` + Thinking *bool `json:"thinking,omitempty"` + Temperature *float64 `json:"temperature,omitempty"` + MaxTemperature *float64 `json:"maxTemperature,omitempty"` + TopP *float64 `json:"topP,omitempty"` + TopK *int `json:"topK,omitempty"` }core/schemas/providers/bedrock/models.go (1)
14-24: Set Object field for consistency with other providers.While the pointer handling and architecture mapping are correct, the
Objectfield is not set. For consistency with other providers (OpenAI, Anthropic, etc.), consider settingObject: "model".Apply this diff:
for _, model := range response.ModelSummaries { bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ ID: model.ModelID, + Object: "model", Name: schemas.Ptr(model.ModelName), OwnedBy: schemas.Ptr(model.ProviderName), Architecture: &schemas.Architecture{ InputModalities: model.InputModalities, OutputModalities: model.OutputModalities, }, }) }core/schemas/providers/anthropic/models.go (1)
18-24: Set Object field for consistency with other providers.The
Objectfield is not populated in the converted models. For consistency with other providers and to match common API patterns, setObjectto"model".Apply this diff:
for _, model := range response.Data { bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ ID: model.ID, + Object: "model", Name: schemas.Ptr(model.DisplayName), Created: schemas.Ptr(int(model.CreatedAt.Unix())), }) }transports/bifrost-http/integrations/anthropic.go (1)
117-129: Simplify by initializing ExtraParams once.The code repeatedly checks and initializes
ExtraParamsfor each query parameter. Consider initializing the map once at the start of the parameter extraction logic.Apply this diff:
listModelsReq.Provider = schemas.Anthropic // Extract limit from query parameters if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { if limit, err := strconv.Atoi(limitStr); err == nil { listModelsReq.Limit = limit } } + // Initialize ExtraParams if needed + if listModelsReq.ExtraParams == nil { + listModelsReq.ExtraParams = make(map[string]interface{}) + } + if beforeID := string(ctx.QueryArgs().Peek("before_id")); beforeID != "" { - if listModelsReq.ExtraParams == nil { - listModelsReq.ExtraParams = make(map[string]interface{}) - } listModelsReq.ExtraParams["before_id"] = beforeID } if afterID := string(ctx.QueryArgs().Peek("after_id")); afterID != "" { - if listModelsReq.ExtraParams == nil { - listModelsReq.ExtraParams = make(map[string]interface{}) - } listModelsReq.ExtraParams["after_id"] = afterID }core/providers/anthropic.go (1)
828-852: Build query with url.Values to escape parameters safely.Manual concatenation risks invalid URLs with special characters (page_token/before_id/after_id). Use url.Values.
+ // Import at top: + // "net/url" @@ - // Build URL with query parameters - queryParams := "" - - // Add limit parameter (default to 20) - limit := request.Limit - if limit == 0 { - limit = 1000 - } - queryParams += fmt.Sprintf("limit=%d", limit) - if request.PageToken != "" { - queryParams += fmt.Sprintf("&page_token=%s", request.PageToken) - } - - // Add cursor-based pagination parameters - if request.ExtraParams != nil { - // before id - if beforeID, ok := request.ExtraParams["before_id"].(string); ok { - queryParams += fmt.Sprintf("&before_id=%s", beforeID) - } - // after id - if afterID, ok := request.ExtraParams["after_id"].(string); ok { - queryParams += fmt.Sprintf("&after_id=%s", afterID) - } - } - - req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models?" + queryParams) + // Build URL with query parameters + values := url.Values{} + limit := request.Limit + if limit == 0 { limit = 1000 } + values.Set("limit", fmt.Sprintf("%d", limit)) + if request.PageToken != "" { + values.Set("page_token", request.PageToken) + } + if request.ExtraParams != nil { + if beforeID, ok := request.ExtraParams["before_id"].(string); ok && beforeID != "" { + values.Set("before_id", beforeID) + } + if afterID, ok := request.ExtraParams["after_id"].(string); ok && afterID != "" { + values.Set("after_id", afterID) + } + } + req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models?" + values.Encode())core/providers/azure.go (1)
100-116: Minor: extract URL construction to a helper to avoid duplication.Both deployment and non-deployment paths are hand-built in multiple places. A small helper (buildAzureURL(path, key, model, useDeployment)) reduces errors and keeps api-version logic in one place.
Also applies to: 126-143
core/providers/cohere.go (1)
836-856: URL-escape query parameters for list models.Manually concatenated query strings can break with special characters (page_token, endpoint). Use url.Values.
+// import "net/url" @@ - queryParams := "" - limit := request.Limit - if limit == 0 { - limit = 1000 - } - queryParams += fmt.Sprintf("page_size=%d", limit) - - if request.PageToken != "" { - queryParams += fmt.Sprintf("&page_token=%s", request.PageToken) - } - if endpoint, ok := request.ExtraParams["endpoint"].(string); ok && endpoint != "" { - queryParams += fmt.Sprintf("&endpoint=%s", endpoint) - } - if defaultOnly, ok := request.ExtraParams["default_only"].(bool); ok && defaultOnly { - queryParams += fmt.Sprintf("&default_only=%t", defaultOnly) - } - - req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models?" + queryParams) + vals := url.Values{} + limit := request.Limit + if limit == 0 { limit = 1000 } + vals.Set("page_size", fmt.Sprintf("%d", limit)) + if request.PageToken != "" { vals.Set("page_token", request.PageToken) } + if endpoint, ok := request.ExtraParams["endpoint"].(string); ok && endpoint != "" { + vals.Set("endpoint", endpoint) + } + if defaultOnly, ok := request.ExtraParams["default_only"].(bool); ok && defaultOnly { + vals.Set("default_only", "true") + } + req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models?" + vals.Encode())Also applies to: 843-852
core/bifrost.go (1)
1183-1192: Past race condition concern is now resolved by mutex protection.The past review comment flagged a potential race condition with concurrent appends to the providers slice. The current implementation is safe because all call sites (lines 190, 776, 1236) acquire
providerMutex.Lock()before callingprepareProvider, serializing concurrent updates.Optional refinement: The nil check (lines 1185-1189) is unnecessary since
Initalways initializesprovidersto an empty slice (line 97). You could simplify to:// Add provider to list of providers loadedProviders := bifrost.providers.Load() - if loadedProviders == nil { - // Initialize if somehow nil - emptyProviders := make([]schemas.Provider, 0) - loadedProviders = &emptyProviders - } currentProviders := *loadedProviders updatedProviders := append(currentProviders, provider) bifrost.providers.Store(&updatedProviders)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (46)
core/bifrost.go(8 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(8 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(2 hunks)core/providers/gemini.go(2 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(3 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)docs/features/custom-providers.mdx(2 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(7 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (13)
- core/schemas/bifrost.go
- core/providers/parasail.go
- core/schemas/provider.go
- core/providers/cerebras.go
- core/providers/mistral.go
- core/schemas/providers/vertex/types.go
- core/schemas/providers/mistral/models.go
- core/schemas/providers/cohere/models.go
- core/schemas/providers/vertex/models.go
- core/schemas/providers/openai/models.go
- core/schemas/providers/azure/models.go
- core/providers/ollama.go
- transports/bifrost-http/handlers/inference.go
🧰 Additional context used
🧬 Code graph analysis (20)
core/providers/sgl.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (5)
BifrostError(320-329)ErrorField(338-345)RequestCancelled(280-280)Bedrock(38-38)ListModelsRequest(93-93)core/utils.go (1)
Ptr(12-14)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)
transports/bifrost-http/integrations/openai.go (4)
transports/bifrost-http/integrations/utils.go (8)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)GenericRouter(227-232)NewGenericRouter(236-243)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)OpenAI(35-35)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/providers/gemini.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)Gemini(47-47)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1356-1359)
core/schemas/providers/anthropic/models.go (2)
core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicModel(273-278)core/schemas/models.go (2)
BifrostListModelsResponse(17-22)Model(24-41)
core/schemas/providers/bedrock/models.go (2)
core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)core/schemas/models.go (3)
BifrostListModelsResponse(17-22)Model(24-41)Architecture(43-49)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (2)
BifrostError(320-329)Groq(43-43)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (5)
BifrostError(320-329)OpenAI(35-35)ListModelsRequest(93-93)ModelProvider(32-32)RequestType(79-79)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(136-139)core/schemas/provider.go (1)
Provider(197-224)
core/providers/azure.go (6)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/bifrost.go (4)
BifrostError(320-329)Azure(36-36)RequestType(79-79)ListModelsRequest(93-93)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/utils.go (1)
Ptr(12-14)core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/providers/azure/types.go (1)
AzureListModelsResponse(28-31)
core/providers/anthropic.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)Anthropic(37-37)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (1)
Provider(197-224)core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicError(302-308)
transports/bifrost-http/integrations/genai.go (4)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(29-54)
core/providers/openrouter.go (3)
core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)OpenRouter(48-48)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)
core/providers/cohere.go (5)
core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (6)
RequestType(79-79)ChatCompletionRequest(84-84)BifrostError(320-329)Cohere(39-39)ListModelsRequest(93-93)ErrorField(338-345)core/schemas/account.go (1)
Key(8-17)core/schemas/providers/cohere/types.go (2)
CohereListModelsResponse(539-542)CohereError(521-525)
core/providers/vertex.go (5)
core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/providers/vertex/types.go (1)
VertexListModelsResponse(59-62)core/schemas/bifrost.go (4)
BifrostError(320-329)ListModelsRequest(93-93)ErrorField(338-345)RequestCancelled(280-280)core/schemas/provider.go (5)
Provider(197-224)ErrRequestCancelled(23-23)ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/bifrost.go (2)
BifrostRequest(130-140)ListModelsRequest(93-93)core/schemas/models.go (1)
BifrostListModelsResponse(17-22)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(29-52)core/schemas/provider.go (1)
Provider(197-224)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (8)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(320-329)BifrostResponseExtraFields(252-260)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)Model(24-41)
core/schemas/providers/gemini/models.go (2)
core/schemas/providers/gemini/types.go (2)
GeminiListModelsResponse(1356-1359)GeminiModel(1340-1354)core/schemas/models.go (2)
BifrostListModelsResponse(17-22)Model(24-41)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(252-260)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(24-41)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
🔇 Additional comments (33)
core/schemas/providers/mistral/types.go (2)
21-28: LGTM!The
Capabilitiesstruct is well-defined with clear boolean flags for each model capability. The structure is appropriate for feature detection.
31-34: LGTM!The
MistralListModelsResponsestruct follows the standard API response pattern with anObjecttype identifier and aDataarray, consistent with common provider conventions.ui/app/providers/fragments/apiStructureFormFragment.tsx (1)
48-48: LGTM!The
list_modelsfield is properly integrated with consistent optional chaining and default value matching the other allowed request types.ui/app/providers/dialogs/addNewCustomProviderDialog.tsx (2)
24-24: LGTM!The schema definition for
list_modelsis consistent with the other allowed request type validations.
59-59: LGTM!The default value for
list_modelsis properly set totrue, consistent with the other request types.ui/app/providers/fragments/allowedRequestsFields.tsx (1)
21-21: LGTM!The new "List Models" request type is properly added to the REQUEST_TYPES array. Note that this creates a 4-5 column split (4 items in left column, 5 in right), which is expected based on the existing layout logic at lines 25-26.
ui/lib/types/schemas.ts (1)
284-284: LGTM!The
list_modelsfield is properly added to theallowedRequestsSchemawith the correct boolean type, consistent with the other request type fields.ui/lib/types/config.ts (1)
123-123: LGTM!The
list_modelsfield is properly added to theAllowedRequestsinterface with the correct boolean type, maintaining consistency with the existing request type fields.ui/lib/constants/config.ts (1)
68-68: LGTM!The
list_modelsfield is properly added toDEFAULT_ALLOWED_REQUESTSwith a default value oftrue, consistent with the other request types and properly satisfying theRequired<AllowedRequests>type constraint.core/providers/sgl.go (1)
199-202: LGTM!The ListModels implementation correctly returns an unsupported operation error, consistent with other unsupported operations in this provider (Speech, Transcription, etc.).
transports/bifrost-http/integrations/utils.go (1)
118-118: LGTM!The new
ListModelsResponseConvertertype follows the established pattern for response converters.core/schemas/providers/cohere/types.go (1)
527-542: LGTM!The new Cohere model types are well-structured with proper JSON tags and support for pagination via
NextPageToken.transports/bifrost-http/integrations/openai.go (2)
325-360: LGTM!The new
CreateOpenAIListModelsRouteConfigsfunction follows the established pattern and correctly registers GET endpoints for listing models across OpenAI and Azure deployment paths.
362-387: LGTM!The pre-callback correctly chains the Azure endpoint hook with OpenAI provider defaulting, ensuring proper provider selection for both standard and Azure deployment endpoints.
core/schemas/models.go (1)
51-60: LGTM!The Pricing type uses pointer fields for all pricing dimensions, correctly allowing providers to return null for pricing information they don't provide.
transports/bifrost-http/integrations/genai.go (2)
67-89: LGTM! Route config follows established patterns.The new GET endpoint for listing Gemini models is well-structured and consistent with the existing route configuration pattern. The request/response converters and PreCallback are properly wired.
156-177: LGTM! Parameter extraction correctly maps Gemini-specific query params.The function properly sets the provider and maps Gemini's
pageSizeparameter to the internalLimitfield, along with thepageTokenfor pagination support.core/schemas/providers/anthropic/types.go (1)
272-285: LGTM! Type definitions are clean and appropriate.The new model types for Anthropic's list models response are well-structured with appropriate field types and JSON tags.
core/providers/groq.go (1)
224-236: LGTM! Correctly delegates to OpenAI-compatible handler.The ListModels implementation appropriately reuses the shared OpenAI handler, consistent with Groq's OpenAI-compatible API design.
core/schemas/providers/openai/types.go (1)
124-139: LGTM! Model types are well-defined.The OpenAI model types include appropriate fields with proper JSON tags. The optional Groq-specific fields are correctly marked as omitempty.
core/schemas/providers/anthropic/models.go (1)
29-52: LGTM! Reverse conversion handles optional fields correctly.The ToAnthropicListModelsResponse function properly checks for nil pointers and converts Unix timestamps back to time.Time values.
transports/bifrost-http/integrations/anthropic.go (1)
76-102: LGTM! List models route config follows established patterns.The route configuration for Anthropic's list models endpoint is properly structured and consistent with other integrations.
core/providers/gemini.go (2)
937-961: LGTM! ListModels method follows established patterns.The implementation correctly checks operation permissions, delegates to the handler, and enriches the response with extra fields (provider, latency, request type).
963-1015: Well-structured helper function with proper error handling.The handleGeminiListModelsRequest function correctly builds the URL with query parameters, handles errors, and parses both typed and raw responses for optional inclusion in extra fields.
core/providers/anthropic.go (1)
789-814: ListModels flow looks consistent with provider pattern.Checks allowlist, converts provider response to Bifrost, and fills ExtraFields correctly. LGTM.
core/providers/openai.go (1)
1780-1851: Solid OpenAI ListModels implementation.Correct endpoint, auth, error handling, use of provider-specific types and converter, and ExtraFields. LGTM.
core/schemas/providers/gemini/models.go (1)
10-27: Pointer bug fixed; capacity preallocated.Using schemas.Ptr avoids shared-address pitfall; preallocating Data is good. LGTM.
core/providers/cohere.go (1)
798-823: Authorization check added; flow and ExtraFields look good.Consistent with other providers. LGTM.
core/schemas/providers/bedrock/types.go (1)
434-454: Model types LGTM.Fields and JSON tags match Bedrock model listing semantics; ready for converters.
Also applies to: 452-453
core/providers/bedrock.go (1)
958-999: ListModels implementation looks correct.Correct endpoint (/foundation-models), conversion, ExtraFields, and optional RawResponse. LGTM.
core/providers/openrouter.go (1)
184-202: OpenRouter's API does not support pagination parameters—do not implement this suggestion.OpenRouter's GET /v1/models endpoint does not accept pagination query parameters (limit, cursor, page_token). The endpoint returns all available models in the response body without pagination support.
While other providers like Anthropic and Gemini support pagination via query parameters (limit, page_token), OpenRouter's API design does not. The current implementation correctly calls the endpoint without query parameters. Adding the suggested pagination code would be ineffective since the API ignores these parameters.
Likely an incorrect or invalid review comment.
core/bifrost.go (2)
1246-1261: LGTM!The
getProviderByKeymethod correctly retrieves a provider from the thread-safe providers list. Thread-safety is maintained viaatomic.Pointer.Load(), and the nil return case is properly handled by the caller.
2141-2165: Past key selection issue for ListModels is now resolved.The new conditional logic correctly handles
ListModelsRequestby skipping model-specific filtering (lines 2141-2142), while preserving the existing deployment checks for Azure/Bedrock on model-targeted requests (lines 2144-2164). This addresses the past review comment about key selection incorrectly filtering out valid keys for ListModels.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
78425c1 to
6bd0835
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (8)
transports/bifrost-http/handlers/inference.go (1)
289-323: Add pagination and extra query parameters.Currently only provider is read; clients cannot paginate or pass provider-specific options.
func (h *CompletionHandler) listModels(ctx *fasthttp.RequestCtx) { // Get provider from query parameters provider := string(ctx.QueryArgs().Peek("provider")) // Convert context bifrostCtx := lib.ConvertToBifrostContext(ctx, h.handlerStore.ShouldAllowDirectKeys()) if bifrostCtx == nil { SendError(ctx, fasthttp.StatusInternalServerError, "Failed to convert context", h.logger) return } var resp *schemas.BifrostListModelsResponse var bifrostErr *schemas.BifrostError // If provider is empty, list all models from all providers if provider == "" { resp, bifrostErr = h.client.ListAllModels(*bifrostCtx) } else { - bifrostListModelsReq := &schemas.BifrostListModelsRequest{ - Provider: schemas.ModelProvider(provider), - } + bifrostListModelsReq := &schemas.BifrostListModelsRequest{ + Provider: schemas.ModelProvider(provider), + } + // Optional: limit + if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { + if limit, err := strconv.Atoi(limitStr); err == nil && limit > 0 { + bifrostListModelsReq.Limit = limit + } + } + // Optional: page_token + if pageToken := string(ctx.QueryArgs().Peek("page_token")); pageToken != "" { + bifrostListModelsReq.PageToken = pageToken + } + // Optional: provider-specific extras + bifrostListModelsReq.ExtraParams = make(map[string]interface{}) + ctx.QueryArgs().VisitAll(func(key, value []byte) { + k := string(key) + if k != "provider" && k != "limit" && k != "page_token" { + bifrostListModelsReq.ExtraParams[k] = string(value) + } + }) resp, bifrostErr = h.client.ListModelsRequest(*bifrostCtx, bifrostListModelsReq) } if bifrostErr != nil { SendBifrostError(ctx, bifrostErr, h.logger) return } // Send successful response SendJSON(ctx, resp, h.logger) }transports/bifrost-http/integrations/utils.go (1)
250-267: Validate list-models GET routes to prevent nil deref.Ensure ListModelsResponseConverter is configured; otherwise handler will panic at conversion.
- method := strings.ToUpper(route.Method) + method := strings.ToUpper(route.Method) if method != fasthttp.MethodGet { if route.GetRequestTypeInstance == nil { g.logger.Warn("route configuration is invalid: GetRequestTypeInstance cannot be nil for route " + route.Path) continue } if route.RequestConverter == nil { g.logger.Warn("route configuration is invalid: RequestConverter cannot be nil for route " + route.Path) continue } // Test that GetRequestTypeInstance returns a valid instance if testInstance := route.GetRequestTypeInstance(); testInstance == nil { g.logger.Warn("route configuration is invalid: GetRequestTypeInstance returned nil for route " + route.Path) continue } - } + } else { + // GET routes are used for list models; enforce converter presence + if route.ListModelsResponseConverter == nil { + g.logger.Warn("route configuration is invalid: ListModelsResponseConverter cannot be nil for GET route " + route.Path) + continue + } + }core/providers/gemini.go (1)
978-982: Lower the default limit to 50 to match Gemini's API default.The fallback limit is still set to 1000, which diverges from Gemini's documented default page size of 50. This unnecessarily inflates response payloads when clients don't specify a limit.
Apply this diff to align with Gemini's default:
limit := request.Limit if limit == 0 { - limit = 1000 + limit = 50 } queryParams += fmt.Sprintf("pageSize=%d", limit)Based on learnings (past review findings confirmed Gemini's default is 50).
core/providers/mistral.go (1)
180-241: Missing pagination support: wire request parameter through to handler.The
requestparameter containingLimitandPageTokenis never passed tohandleMistralListModelsRequest, so pagination parameters are silently ignored. This breaks pagination functionality for Mistral's models endpoint.Update the handler signature and call site:
-func (provider *MistralProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - mistralResponse, rawResponse, latency, err := provider.handleMistralListModelsRequest(ctx, key) +func (provider *MistralProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { + mistralResponse, rawResponse, latency, err := provider.handleMistralListModelsRequest(ctx, key, request)And add pagination query parameters in the handler:
-func (provider *MistralProvider) handleMistralListModelsRequest(ctx context.Context, key schemas.Key) (*mistral.MistralListModelsResponse, interface{}, time.Duration, *schemas.BifrostError) { +func (provider *MistralProvider) handleMistralListModelsRequest(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*mistral.MistralListModelsResponse, interface{}, time.Duration, *schemas.BifrostError) { // Create request req := fasthttp.AcquireRequest() resp := fasthttp.AcquireResponse() defer fasthttp.ReleaseRequest(req) defer fasthttp.ReleaseResponse(resp) // Set any extra headers from network config setExtraHeaders(req, provider.networkConfig.ExtraHeaders, nil) - req.SetRequestURI(provider.networkConfig.BaseURL + "/v1/models") + // Build URL with query parameters if provided + url := provider.networkConfig.BaseURL + "/v1/models" + if request.Limit > 0 || request.PageToken != "" { + queryArgs := req.URI().QueryArgs() + if request.Limit > 0 { + queryArgs.SetUint("limit", request.Limit) + } + if request.PageToken != "" { + queryArgs.Set("page_token", request.PageToken) + } + } + req.SetRequestURI(url) req.Header.SetMethod("GET")core/providers/openrouter.go (1)
231-241: Critical: Use OpenRouter-specific types and converter.The code unmarshals directly into
schemas.BifrostListModelsResponse, bypassing the OpenRouter-specificOpenRouterModelListResponsetype and itsToBifrostListModelsResponseconverter. This is inconsistent with other providers and leaves the converter unused.Apply this fix:
+import ( + openrouter "github.com/maximhq/bifrost/core/schemas/providers/openrouter" +) - var openrouterResponse schemas.BifrostListModelsResponse - if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { + var openrouterResponse openrouter.OpenRouterModelListResponse + if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } var rawResponse interface{} if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } - return &openrouterResponse, rawResponse, latency, nil + bifrostResponse := openrouterResponse.ToBifrostListModelsResponse() + return bifrostResponse, rawResponse, latency, nilcore/schemas/providers/mistral/types.go (1)
7-7: ChangeCreatedfield frominttoint64.Unix timestamps should be stored as
int64to handle dates beyond 2038 and ensure platform portability (sinceintsize is platform-dependent).Apply this change:
type MistralModel struct { ID string `json:"id"` Object string `json:"object"` - Created int `json:"created"` + Created int64 `json:"created"` OwnedBy string `json:"owned_by"`Based on learnings (past review confirmed Mistral API returns integer Unix timestamps).
core/providers/azure.go (1)
96-99: Update API version default to "2024-10-21" and centralize to prevent drift.The code defaults to "2024-02-01" in three locations (lines 97, 228, 324), creating maintenance burden and version drift. Additionally, schema documentation at
core/schemas/account.go:30claims "2024-08-01-preview" as the default, creating a documentation-code mismatch. The current stable Azure OpenAI data-plane API version is "2024-10-21" as of October 2025.Create a centralized constant and update all three locations:
+// In core/schemas/account.go or core/schemas/azure.go +const DefaultAzureAPIVersion = "2024-10-21" // In core/providers/azure.go (three locations: lines 97, 228, 324) apiVersion := key.AzureKeyConfig.APIVersion if apiVersion == nil { - apiVersion = schemas.Ptr("2024-02-01") + apiVersion = schemas.Ptr(schemas.DefaultAzureAPIVersion) }Also update the comment in
core/schemas/account.goline 23 to match the new default.Based on learnings (past review confirmed 2024-10-21 is current stable version).
core/providers/vertex.go (1)
637-795: Missing pagination support: update handler signature and add query parameters.The
requestparameter is never passed tohandleVertexListModelsRequest, silently discarding pagination parameters. The handler also constructs URLs without pageSize/pageToken query parameters, preventing pagination functionality.Update the handler signature and call:
func (provider *VertexProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - vertexResponse, rawResponse, latency, err := provider.handleVertexListModelsRequest(ctx, key) + vertexResponse, rawResponse, latency, err := provider.handleVertexListModelsRequest(ctx, key, request) if err != nil { return nil, err }Update the handler signature and add pagination support:
+import ( + "net/url" +) -func (provider *VertexProvider) handleVertexListModelsRequest(ctx context.Context, key schemas.Key) (*vertex.VertexListModelsResponse, interface{}, time.Duration, *schemas.BifrostError) { +func (provider *VertexProvider) handleVertexListModelsRequest(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*vertex.VertexListModelsResponse, interface{}, time.Duration, *schemas.BifrostError) { providerName := provider.GetProviderKey() if key.VertexKeyConfig == nil { return nil, nil, 0, newConfigurationError("vertex key config is not set", providerName) } projectID := key.VertexKeyConfig.ProjectID if projectID == "" { return nil, nil, 0, newConfigurationError("project ID is not set", providerName) } region := key.VertexKeyConfig.Region if region == "" { return nil, nil, 0, newConfigurationError("region is not set in key config", providerName) } // Construct URL for listing models - url := fmt.Sprintf("https://%s-aiplatform.googleapis.com/v1/projects/%s/locations/%s/models", region, projectID, region) + baseURL := fmt.Sprintf("https://%s-aiplatform.googleapis.com/v1/projects/%s/locations/%s/publishers/google/models", region, projectID, region) + u, err := url.Parse(baseURL) + if err != nil { + return nil, nil, 0, newBifrostOperationError(schemas.ErrProviderRequest, err, providerName) + } + q := u.Query() + if request.Limit > 0 { + q.Set("pageSize", fmt.Sprintf("%d", request.Limit)) + } + if request.PageToken != "" { + q.Set("pageToken", request.PageToken) + } + u.RawQuery = q.Encode() // Create request - req, err := http.NewRequestWithContext(ctx, "GET", url, nil) + req, err := http.NewRequestWithContext(ctx, "GET", u.String(), nil)Based on learnings (past review confirmed Vertex AI uses pageSize/pageToken query parameters).
🧹 Nitpick comments (6)
ui/app/providers/fragments/allowedRequestsFields.tsx (1)
21-22: Add “List Models” entry looks good; minor UX/robustness polish.
- Balance columns dynamically (5/5 when 10 items) instead of hardcoded 4/6.
- Coerce possibly undefined values to boolean to avoid controlled/uncontrolled warnings.
Apply:
-const leftColumn = REQUEST_TYPES.slice(0, 4); -const rightColumn = REQUEST_TYPES.slice(4); +const mid = Math.ceil(REQUEST_TYPES.length / 2); +const leftColumn = REQUEST_TYPES.slice(0, mid); +const rightColumn = REQUEST_TYPES.slice(mid);And:
-<Switch checked={field.value} onCheckedChange={field.onChange} size="md" /> +<Switch checked={Boolean(field.value)} onCheckedChange={field.onChange} size="md" />Also applies to: 25-26, 48-49, 67-68
core/schemas/providers/gemini/types.go (2)
1341-1354: Go initialism style: preferIDoverId.Rename
BaseModelId→BaseModelIDfor consistency with Go conventions and other types in repo (e.g.,ModelID). This improves readability and linters’ happiness. Non-breaking for JSON due to tags.-type GeminiModel struct { - Name string `json:"name"` - BaseModelId string `json:"baseModelId"` +type GeminiModel struct { + Name string `json:"name"` + BaseModelID string `json:"baseModelId"` Version string `json:"version"` DisplayName string `json:"displayName"` Description string `json:"description"` InputTokenLimit int `json:"inputTokenLimit"` OutputTokenLimit int `json:"outputTokenLimit"` SupportedGenerationMethods []string `json:"supportedGenerationMethods"` Thinking bool `json:"thinking"` Temperature float64 `json:"temperature"` MaxTemperature float64 `json:"maxTemperature"` TopP float64 `json:"topP"` TopK int `json:"topK"` }
1356-1359: Avoid emitting empty pagination token.Mark
NextPageTokenas omitempty to trim empty strings in JSON output.-type GeminiListModelsResponse struct { - Models []GeminiModel `json:"models"` - NextPageToken string `json:"nextPageToken"` -} +type GeminiListModelsResponse struct { + Models []GeminiModel `json:"models"` + NextPageToken string `json:"nextPageToken,omitempty"` +}transports/bifrost-http/integrations/genai.go (1)
156-177: Capture provider-specific extras and validate pageSize.Expose unknown query params via ExtraParams and guard negative values.
func extractGeminiListModelsParams(ctx *fasthttp.RequestCtx, req interface{}) error { if listModelsReq, ok := req.(*schemas.BifrostListModelsRequest); ok { // Set provider to Gemini listModelsReq.Provider = schemas.Gemini // Extract pageSize from query parameters (Gemini uses pageSize instead of limit) if pageSizeStr := string(ctx.QueryArgs().Peek("pageSize")); pageSizeStr != "" { - if pageSize, err := strconv.Atoi(pageSizeStr); err == nil { - listModelsReq.Limit = pageSize + if pageSize, err := strconv.Atoi(pageSizeStr); err == nil { + if pageSize < 0 { + pageSize = 0 + } + listModelsReq.Limit = pageSize } } // Extract pageToken from query parameters if pageToken := string(ctx.QueryArgs().Peek("pageToken")); pageToken != "" { listModelsReq.PageToken = pageToken } + // Collect any additional provider-specific query params + if listModelsReq.ExtraParams == nil { + listModelsReq.ExtraParams = make(map[string]interface{}) + } + ctx.QueryArgs().VisitAll(func(k, v []byte) { + key := string(k) + if key != "pageSize" && key != "pageToken" { + listModelsReq.ExtraParams[key] = string(v) + } + }) return nil } return errors.New("invalid request type for Gemini list models") }core/schemas/providers/vertex/models.go (1)
14-21: Optional: fallback Name when DisplayName is empty.Avoid nil Name by falling back to ID.
- bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ - ID: model.Name, - Name: schemas.Ptr(model.DisplayName), - Description: schemas.Ptr(model.Description), - Created: schemas.Ptr(int(model.VersionCreateTime.Unix())), - }) + namePtr := func() *string { + if model.DisplayName != "" { + return schemas.Ptr(model.DisplayName) + } + return schemas.Ptr(model.Name) + }() + bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ + ID: model.Name, + Name: namePtr, + Description: schemas.Ptr(model.Description), + Created: schemas.Ptr(int(model.VersionCreateTime.Unix())), + })transports/bifrost-http/integrations/utils.go (1)
113-115: Fix RequestConverter docstring (no ctx param).Clarify that URL/query extraction should happen in PreCallback.
-// It takes the parsed request object and the request context for accessing URL/query params. +// It takes the parsed request object. Use PreRequestCallback to access URL/query params.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (46)
core/bifrost.go(8 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(8 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(2 hunks)core/providers/gemini.go(2 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(3 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)docs/features/custom-providers.mdx(2 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(7 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- core/providers/groq.go
🚧 Files skipped from review as they are similar to previous changes (21)
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- core/schemas/providers/bedrock/models.go
- ui/lib/constants/config.ts
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- core/schemas/providers/mistral/models.go
- core/providers/sgl.go
- core/providers/anthropic.go
- core/providers/cerebras.go
- core/schemas/providers/azure/types.go
- core/schemas/providers/gemini/models.go
- core/providers/cohere.go
- core/schemas/providers/anthropic/types.go
- transports/bifrost-http/integrations/openai.go
- transports/bifrost-http/integrations/anthropic.go
- core/schemas/providers/openai/types.go
- core/schemas/providers/azure/models.go
- core/schemas/provider.go
- ui/lib/types/schemas.ts
- core/schemas/providers/anthropic/models.go
- core/providers/parasail.go
- docs/features/custom-providers.mdx
🧰 Additional context used
🧬 Code graph analysis (17)
core/schemas/providers/openai/models.go (2)
core/schemas/providers/openai/types.go (2)
OpenAIListModelsResponse(136-139)OpenAIModel(125-134)core/schemas/models.go (2)
BifrostListModelsResponse(17-22)Model(24-41)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (6)
BifrostError(320-329)ErrorField(338-345)RequestCancelled(280-280)Bedrock(38-38)ListModelsRequest(93-93)RequestType(79-79)core/utils.go (1)
Ptr(12-14)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/bifrost.go (2)
BifrostRequest(130-140)ListModelsRequest(93-93)core/schemas/models.go (1)
BifrostListModelsResponse(17-22)
core/providers/gemini.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)Gemini(47-47)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1356-1359)
core/providers/ollama.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/providers/mistral.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)Mistral(41-41)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)
core/schemas/providers/vertex/models.go (2)
core/schemas/providers/vertex/types.go (1)
VertexListModelsResponse(59-62)core/schemas/models.go (2)
BifrostListModelsResponse(17-22)Model(24-41)
core/schemas/providers/cohere/models.go (2)
core/schemas/providers/cohere/types.go (1)
CohereListModelsResponse(539-542)core/schemas/models.go (2)
BifrostListModelsResponse(17-22)Model(24-41)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(29-54)core/schemas/provider.go (1)
Provider(197-224)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)OpenRouter(48-48)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/bifrost.go (4)
BifrostError(320-329)OpenAI(35-35)ListModelsRequest(93-93)ModelProvider(32-32)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(136-139)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/bifrost.go (2)
core/schemas/models.go (1)
BifrostListModelsRequest(3-15)core/schemas/provider.go (1)
Provider(197-224)
core/providers/azure.go (5)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/utils.go (1)
Ptr(12-14)core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/providers/azure/types.go (1)
AzureListModelsResponse(28-31)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(252-260)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (8)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(320-329)BifrostResponseExtraFields(252-260)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)Model(24-41)
core/providers/vertex.go (5)
core/schemas/models.go (3)
Model(24-41)BifrostListModelsRequest(3-15)BifrostListModelsResponse(17-22)core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/providers/vertex/types.go (1)
VertexListModelsResponse(59-62)core/schemas/bifrost.go (4)
BifrostError(320-329)ListModelsRequest(93-93)ErrorField(338-345)RequestCancelled(280-280)core/schemas/provider.go (5)
Provider(197-224)ErrRequestCancelled(23-23)ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(17-22)BifrostListModelsRequest(3-15)core/schemas/bifrost.go (3)
BifrostError(320-329)ModelProvider(32-32)ListModelsRequest(93-93)core/schemas/provider.go (1)
Provider(197-224)
🔇 Additional comments (19)
core/schemas/providers/bedrock/types.go (1)
434-454: Model types LGTM.Fields and JSON tags match Bedrock summaries well. No issues spotted.
core/providers/ollama.go (1)
199-202: Endpoint is valid and properly implemented; request parameter follows established pattern.Ollama exposes GET /v1/models through its OpenAI-compatible layer, confirming the endpoint is correct. The implementation aligns with other OpenAI-compatible providers (Groq, Cerebras, OpenAI)—all use the same shared handler and similarly do not forward the
requestparameter. This is a consistent design choice, not a defect specific to Ollama.ui/lib/types/config.ts (1)
113-124: All defaults and schema updates forlist_modelsare properly in place.Verification confirms:
DEFAULT_ALLOWED_REQUESTSincludeslist_models: true(ui/lib/constants/config.ts:68)- Schema definitions include
list_models: z.boolean()across ui/lib/types/schemas.ts, ui/app/providers/dialogs/addNewCustomProviderDialog.tsx, and ui/lib/schemas/providerForm.ts- Form creation sets
list_models: trueby default (ui/app/providers/dialogs/addNewCustomProviderDialog.tsx:59)- Backward compatibility reads use
?? truefallback (ui/app/providers/fragments/apiStructureFormFragment.tsx:48)- TypeScript constraint
satisfies Required<AllowedRequests>ensures completeness- UI field list includes the new field (ui/app/providers/fragments/allowedRequestsFields.tsx:21)
No runtime
undefinedrisk detected.transports/bifrost-http/integrations/genai.go (1)
67-90: LGTM: Gemini list-models route wiring is correct.Request/response conversion and PreCallback are properly set.
transports/bifrost-http/handlers/inference.go (1)
275-279: LGTM: Route added.GET /v1/models registration is correct and chained through middleware.
core/schemas/providers/cohere/types.go (1)
527-542: LGTM: Cohere model types added.Fields and tags align with usage in converters.
core/schemas/providers/vertex/types.go (1)
50-62: LGTM: Vertex model/list types added.Schema matches converter expectations.
core/schemas/bifrost.go (3)
93-95: LGTM: RequestType for list models.
133-140: LGTM: BifrostRequest extended with ListModelsRequest.
253-256: LGTM: Extra fields made optional.omitempty on Provider/ModelRequested is appropriate for list-models.
core/schemas/providers/cohere/models.go (1)
7-26: LGTM: Cohere → Bifrost list-models mapping.Correct field mapping; avoids range-variable address capture via schemas.Ptr.
core/schemas/providers/openai/models.go (1)
5-54: LGTM!The bidirectional conversion logic is well-implemented with proper nil checks, pre-allocated slices, and correct pointer handling for optional fields. The Created and OwnedBy fields are appropriately wrapped/unwrapped using pointers.
core/schemas/models.go (1)
1-77: LGTM!The model listing schema is well-structured with:
- Appropriate use of internal fields (json:"-") for Limit, PageToken, ExtraParams
- Pointer types for optional fields with omitempty
- Clean separation between request/response structures
- Comprehensive Model type supporting pricing, architecture, and capability metadata
core/providers/bedrock.go (1)
958-999: LGTM! ListModels implementation follows provider patterns.The implementation correctly:
- Uses the bedrock service endpoint (not bedrock-runtime) for listing foundation models
- Handles nil request body with the updated completeRequest signature
- Converts Bedrock-specific response to Bifrost format
- Populates metadata (provider, latency, request type)
core/providers/openai.go (1)
1780-1851: LGTM! ListModels follows OpenAI provider patterns.The implementation correctly mirrors other OpenAI methods:
- Uses fasthttp client with proper request/response pooling
- GET request to
/v1/modelswith optional Authorization header- Converts OpenAI response format to Bifrost schema
- Proper error handling via
parseOpenAIError- Sets metadata fields (provider, request type, latency)
core/bifrost.go (4)
36-38: LGTM! Thread-safe provider registry added.The addition of
providers atomic.Pointer[[]schemas.Provider]at line 38 and its initialization inInit(lines 96-97) provides a thread-safe way to track available providers. This enables the newListModelsRequestto look up providers by key.Also applies to: 96-97
211-274: LGTM! ListModelsRequest correctly handles provider lookup and key selection.The implementation properly:
- Validates request and provider (lines 213-228)
- Forwards all request parameters including Limit, PageToken, and ExtraParams (lines 230-235) ✅ Past review addressed
- Determines base provider type for custom providers (lines 247-252) ✅ Past review addressed
- Selects keys using the request-type-aware selector (line 257) ✅ Past review addressed
1252-1267: LGTM! getProviderByKey safely retrieves providers from the registry.The implementation correctly:
- Loads the atomic pointer
- Handles nil case
- Iterates through providers to find a match by key
- Returns nil if not found
2127-2127: LGTM! Key selection now supports ListModelsRequest without model filtering.The updated signature adds
requestType schemas.RequestType(line 2127), and the logic at lines 2147-2171 correctly handles ListModelsRequest by skipping model/deployment filtering. This resolves the issue raised in past reviews where keys with explicit Models lists or Azure/Bedrock Deployments were incorrectly filtered out for ListModels calls.✅ Past review addressed
Also applies to: 2147-2171
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
6bd0835 to
8618bf8
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
transports/bifrost-http/integrations/utils.go (1)
299-324: Defensive check for nil request instance.createHandler calls GetRequestTypeInstance without nil guard. Add a quick check to avoid panic if a misconfigured route slips through.
req := config.GetRequestTypeInstance() + if req == nil { + g.sendError(ctx, config.ErrorConverter, newBifrostError(nil, "route misconfiguration: GetRequestTypeInstance returned nil")) + return + }
♻️ Duplicate comments (7)
core/providers/gemini.go (1)
978-983: Reduce default page size to 50 (align with Gemini defaults).1000 inflates responses; 50 is a safer default and closer to service defaults.
- pageSize := request.PageSize - if pageSize == 0 { - pageSize = 1000 - } + pageSize := request.PageSize + if pageSize == 0 { + pageSize = 50 + }core/providers/mistral.go (1)
180-203: Request parameters are ignored; add provider-side pagination.ListModels drops the request and paginates locally. Pass request to the handler and translate PageSize/PageToken into query params.
-func (provider *MistralProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - mistralResponse, rawResponse, latency, err := provider.handleMistralListModelsRequest(ctx, key) +func (provider *MistralProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { + mistralResponse, rawResponse, latency, err := provider.handleMistralListModelsRequest(ctx, key, request)And update local handler use accordingly.
transports/bifrost-http/integrations/utils.go (2)
383-403: Handle missing ListModelsResponseConverter at runtime.Avoid a nil-call panic by checking converter and returning a clear error.
- response, err = config.ListModelsResponseConverter(listModelsResponse) + if config.ListModelsResponseConverter == nil { + g.sendError(ctx, config.ErrorConverter, newBifrostError(nil, "route misconfiguration: ListModelsResponseConverter is nil")) + return + } + response, err = config.ListModelsResponseConverter(listModelsResponse)
250-268: Validate required fields for GET routes to prevent nil panics.GET routes skip the validation in RegisterRoutes (line 251:
if method != fasthttp.MethodGet) but createHandler unconditionally callsGetRequestTypeInstance()(line 304) andRequestConverter()(line 340) for all routes. ListModels GET routes also requireListModelsResponseConverter, which is called without checking (handleNonStreamingRequest line 405). Add explicit validation for GET routes.method := strings.ToUpper(route.Method) if method != fasthttp.MethodGet { if route.GetRequestTypeInstance == nil { g.logger.Warn("route configuration is invalid: GetRequestTypeInstance cannot be nil for route " + route.Path) continue } if route.RequestConverter == nil { g.logger.Warn("route configuration is invalid: RequestConverter cannot be nil for route " + route.Path) continue } // Test that GetRequestTypeInstance returns a valid instance if testInstance := route.GetRequestTypeInstance(); testInstance == nil { g.logger.Warn("route configuration is invalid: GetRequestTypeInstance returned nil for route " + route.Path) continue } + } else { + // GET routes also require these (used by createHandler) + if route.GetRequestTypeInstance == nil { + g.logger.Warn("route configuration is invalid: GetRequestTypeInstance cannot be nil for GET route " + route.Path) + continue + } + if route.RequestConverter == nil { + g.logger.Warn("route configuration is invalid: RequestConverter cannot be nil for GET route " + route.Path) + continue + } + // ListModels routes require response converter + if route.ListModelsResponseConverter == nil { + g.logger.Warn("route configuration is invalid: ListModelsResponseConverter cannot be nil for GET route " + route.Path) + continue + } }core/providers/openrouter.go (1)
237-247: Critical: Bypassing OpenRouter-specific types and converter.The code directly unmarshals into
schemas.BifrostListModelsResponse, but OpenRouter has its own response types (OpenRouterModelListResponse) and converter (ToBifrostListModelsResponse). This bypasses the converter logic and is inconsistent with other providers (OpenAI, Cohere, Bedrock, Azure, Vertex).Apply this fix to use OpenRouter-specific types:
+import ( + "context" + "fmt" + "net/http" + "strings" + "time" + + "github.com/bytedance/sonic" + schemas "github.com/maximhq/bifrost/core/schemas" + openrouter "github.com/maximhq/bifrost/core/schemas/providers/openrouter" + "github.com/valyala/fasthttp" +)- var openrouterResponse schemas.BifrostListModelsResponse - if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { + var openrouterResponse openrouter.OpenRouterModelListResponse + if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } var rawResponse interface{} if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } - return &openrouterResponse, rawResponse, latency, nil + bifrostResponse := openrouterResponse.ToBifrostListModelsResponse() + return bifrostResponse, rawResponse, latency, nilcore/bifrost.go (2)
1198-1207: Race condition on provider append still present despite previous review.Two concurrent
prepareProvidercalls can race when appending to the providers slice. The Load → append → Store sequence is not atomic, which can cause one provider append to overwrite another, resulting in lost updates. This issue was previously flagged but remains unaddressed.Apply the CAS loop fix suggested in the previous review:
- // Add provider to list of providers - loadedProviders := bifrost.providers.Load() - if loadedProviders == nil { - // Initialize if somehow nil - emptyProviders := make([]schemas.Provider, 0) - loadedProviders = &emptyProviders - } - currentProviders := *loadedProviders - updatedProviders := append(currentProviders, provider) - bifrost.providers.Store(&updatedProviders) + // Atomically append provider to the providers slice + for { + oldPtr := bifrost.providers.Load() + var oldSlice []schemas.Provider + if oldPtr != nil { + oldSlice = *oldPtr + } + newSlice := make([]schemas.Provider, len(oldSlice)+1) + copy(newSlice, oldSlice) + newSlice[len(oldSlice)] = provider + if bifrost.providers.CompareAndSwap(oldPtr, &newSlice) { + break + } + }
307-332: Pagination safeguard still missing despite previous review.The pagination loop lacks a maximum iteration count, which could cause infinite loops if a provider implementation continuously returns
NextPageTokenvalues. This issue was previously flagged but remains unaddressed.The fix suggested in the previous review should be applied:
+ const maxPaginationRequests = 100 // Safety limit per provider + for _, providerKey := range providerKeys { if strings.TrimSpace(string(providerKey)) == "" { continue } // Create request for this provider with limit of 1000 providerRequest := &schemas.BifrostListModelsRequest{ Provider: providerKey, PageSize: 1000, } // Get all pages for this provider + paginationCount := 0 for { + paginationCount++ + if paginationCount > maxPaginationRequests { + bifrost.logger.Warn(fmt.Sprintf("reached maximum pagination requests (%d) for provider %s", maxPaginationRequests, providerKey)) + break + } + response, bifrostErr := bifrost.ListModelsRequest(ctx, providerRequest) if bifrostErr != nil { // Log the error but continue with other providers bifrost.logger.Warn(fmt.Sprintf("failed to list models for provider %s: %v", providerKey, bifrostErr.Error.Message)) if firstError == nil { firstError = bifrostErr } break } if response != nil { if len(response.Data) > 0 { allModels = append(allModels, response.Data...) } } // Check if there are more pages if response.NextPageToken == "" { break } // Set the page token for the next request providerRequest.PageToken = response.NextPageToken } }
🧹 Nitpick comments (10)
core/schemas/account.go (1)
63-63: Don’t use pointer to context.Context in public interfaces.Go convention is to pass context by value; pointer to interface is an anti‑pattern. Plan a non‑breaking migration.
Proposed signature for next minor with deprecation of the old one:
- GetKeysForProvider(ctx *context.Context, providerKey ModelProvider) ([]Key, error) + GetKeysForProvider(ctx context.Context, providerKey ModelProvider) ([]Key, error)Migration idea:
- Add GetKeysForProviderV2 with value context, keep old method temporarily, and update call sites internally first.
core/providers/ollama.go (1)
198-202: Confirm Ollama supports /v1/models (fallback to /api/tags if not).Not all Ollama builds expose an OpenAI-compatible models endpoint; many only support GET /api/tags. If /v1/models 404s, list will fail. Consider a fallback path that, on 404, hits /api/tags and maps tags to Bifrost models; or document the requirement to enable the OAI shim.
core/schemas/bifrost.go (2)
133-140: Wire ListModels into GetRequestFields/SetProvider for consistent metadata.BifrostRequest now has ListModelsRequest, but GetRequestFields/SetProvider don’t consider it. This can drop provider attribution in logs/metrics for list flows.
Minimal additions:
@@ func (br *BifrostRequest) GetRequestFields() (provider ModelProvider, model string, fallbacks []Fallback) { - switch { + switch { + case br.ListModelsRequest != nil: + return br.ListModelsRequest.Provider, "", nil case br.TextCompletionRequest != nil: @@ @@ func (br *BifrostRequest) SetProvider(provider ModelProvider) { - switch { + switch { + case br.ListModelsRequest != nil: + br.ListModelsRequest.Provider = provider case br.TextCompletionRequest != nil: @@ @@ func (br *BifrostRequest) SetModel(model string) { - switch { + switch { + case br.ListModelsRequest != nil: + // no-op for list models + return case br.TextCompletionRequest != nil:
254-256: omitempty on Provider/ModelRequested: validate no breaking change in existing outputs.Making these optional aligns with ListModels, but confirm all existing response paths still populate them so clients don’t see unexpected omissions.
core/schemas/providers/cohere/models.go (1)
17-24: Converter looks correct; pointer safety is addressed.Prealloc, provider-prefixed IDs, and Ptr helpers are good. Optionally populate OwnedBy if available.
transports/bifrost-http/integrations/openai.go (2)
335-357: Capture pagination and extra query params in PreCallback.Expose page_size/page_token and collect unknown query args into ExtraParams so downstream providers can use them consistently.
Apply within createOpenAIListModelsPreCallback:
- if listModelsReq, ok := req.(*schemas.BifrostListModelsRequest); ok { + if listModelsReq, ok := req.(*schemas.BifrostListModelsRequest); ok { // Set provider to OpenAI (may be overridden by Azure hook) if listModelsReq.Provider == "" { listModelsReq.Provider = schemas.OpenAI } + + // Pagination + if v := string(ctx.QueryArgs().Peek("page_size")); v != "" { + if n, err := strconv.Atoi(v); err == nil { + listModelsReq.PageSize = n + } + } + if v := string(ctx.QueryArgs().Peek("page_token")); v != "" { + listModelsReq.PageToken = v + } + // Extra params (exclude known) + if listModelsReq.ExtraParams == nil { + listModelsReq.ExtraParams = make(map[string]interface{}) + } + ctx.QueryArgs().VisitAll(func(k, val []byte) { + ks := string(k) + if ks != "page_size" && ks != "page_token" && ks != "provider" { + listModelsReq.ExtraParams[ks] = string(val) + } + }) return nil }
362-387: Comment vs behavior mismatch about Azure override.The comment says provider “may be overridden by Azure hook,” but AzureEndpointPreHook doesn’t set listModelsReq.Provider; it only attaches a direct key. Either update the comment or set Provider based on Azure headers/path if that’s intended.
transports/bifrost-http/handlers/inference.go (1)
292-328: Support ExtraParams and sensible defaults.
- Collect unknown query params into ExtraParams for provider-specific filtering.
- Optionally set a conservative default page_size when absent.
pageSize, _ := strconv.Atoi(string(ctx.QueryArgs().Peek("page_size"))) pageToken := string(ctx.QueryArgs().Peek("page_token")) bifrostListModelsReq := &schemas.BifrostListModelsRequest{ Provider: schemas.ModelProvider(provider), PageSize: pageSize, PageToken: pageToken, } + // ExtraParams: include all unknown query args + bifrostListModelsReq.ExtraParams = map[string]interface{}{} + ctx.QueryArgs().VisitAll(func(k, v []byte) { + ks := string(k) + if ks != "provider" && ks != "page_size" && ks != "page_token" { + bifrostListModelsReq.ExtraParams[ks] = string(v) + } + }) + // Optional: set a safe default if not provided + if bifrostListModelsReq.PageSize <= 0 { + bifrostListModelsReq.PageSize = 100 + }transports/bifrost-http/integrations/anthropic.go (1)
104-134: Also extract page_token for Anthropic.Carry the page_token query arg through BifrostListModelsRequest for cursor flows.
if listModelsReq, ok := req.(*schemas.BifrostListModelsRequest); ok { @@ - if afterID := string(ctx.QueryArgs().Peek("after_id")); afterID != "" { + if afterID := string(ctx.QueryArgs().Peek("after_id")); afterID != "" { if listModelsReq.ExtraParams == nil { listModelsReq.ExtraParams = make(map[string]interface{}) } listModelsReq.ExtraParams["after_id"] = afterID } + + // Optional page_token passthrough + if v := string(ctx.QueryArgs().Peek("page_token")); v != "" { + listModelsReq.PageToken = v + }core/providers/anthropic.go (1)
831-837: Lower default limit to reduce payloads.Use a conservative default (e.g., 100) when limit is not provided.
- pageSize := request.PageSize - if pageSize == 0 { - pageSize = 1000 - } + pageSize := request.PageSize + if pageSize == 0 { + pageSize = 100 + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (49)
core/bifrost.go(8 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(10 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(2 hunks)core/providers/gemini.go(2 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(3 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/schemas/utils.go(2 hunks)docs/features/custom-providers.mdx(2 hunks)tests/core-providers/config/account.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(7 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- tests/core-providers/config/account.go
🚧 Files skipped from review as they are similar to previous changes (17)
- ui/lib/types/config.ts
- ui/lib/types/schemas.ts
- ui/lib/constants/config.ts
- core/schemas/provider.go
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- core/schemas/providers/cohere/types.go
- core/providers/cerebras.go
- core/schemas/providers/mistral/types.go
- core/schemas/providers/azure/types.go
- docs/features/custom-providers.mdx
- core/schemas/providers/openai/models.go
- core/schemas/providers/bedrock/types.go
- core/schemas/providers/openai/types.go
- ui/app/providers/fragments/allowedRequestsFields.tsx
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- core/schemas/providers/gemini/models.go
- core/providers/vertex.go
🧰 Additional context used
🧬 Code graph analysis (27)
core/providers/openrouter.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostError(320-329)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(93-93)core/schemas/utils.go (1)
ApplyPagination(627-681)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/bifrost.go (2)
BifrostRequest(130-140)ListModelsRequest(93-93)core/schemas/models.go (1)
BifrostListModelsResponse(16-25)
core/providers/sgl.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
transports/bifrost-http/integrations/openai.go (5)
transports/bifrost-http/integrations/utils.go (8)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)OpenAI(35-35)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/providers/parasail.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(35-60)core/schemas/provider.go (1)
Provider(197-224)
core/providers/mistral.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (3)
BifrostError(320-329)ListModelsRequest(93-93)Mistral(41-41)core/schemas/utils.go (1)
ApplyPagination(627-681)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)
core/schemas/providers/vertex/models.go (4)
core/schemas/providers/vertex/types.go (1)
VertexListModelsResponse(59-62)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Vertex(40-40)core/schemas/utils.go (1)
Ptr(17-19)
core/schemas/providers/mistral/models.go (4)
core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Mistral(41-41)core/schemas/utils.go (1)
Ptr(17-19)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)BifrostListModelsRequest(3-14)core/schemas/bifrost.go (3)
BifrostError(320-329)ModelProvider(32-32)ListModelsRequest(93-93)core/schemas/provider.go (1)
Provider(197-224)
core/providers/ollama.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/providers/groq.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (2)
BifrostError(320-329)Groq(43-43)
core/schemas/providers/anthropic/models.go (4)
core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicModel(273-278)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
Ptr(17-19)
core/providers/anthropic.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (3)
BifrostError(320-329)Anthropic(37-37)ListModelsRequest(93-93)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicError(302-308)
core/schemas/bifrost.go (2)
core/schemas/models.go (1)
BifrostListModelsRequest(3-14)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(32-61)core/schemas/provider.go (1)
Provider(197-224)
core/providers/openai.go (7)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (5)
BifrostError(320-329)OpenAI(35-35)ListModelsRequest(93-93)ModelProvider(32-32)RequestType(79-79)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(136-139)core/schemas/utils.go (1)
ApplyPagination(627-681)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/azure/models.go (4)
core/schemas/providers/azure/types.go (1)
AzureListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Azure(36-36)core/schemas/utils.go (1)
Ptr(17-19)
core/schemas/utils.go (1)
core/schemas/models.go (2)
Model(27-44)BifrostListModelsResponse(16-25)
core/providers/gemini.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostError(320-329)Gemini(47-47)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (1)
Provider(197-224)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1356-1359)
core/schemas/providers/cohere/models.go (4)
core/schemas/providers/cohere/types.go (1)
CohereListModelsResponse(539-542)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
Ptr(17-19)
core/bifrost.go (5)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (8)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(320-329)BifrostResponseExtraFields(252-260)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
ApplyPagination(627-681)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (4)
BifrostError(320-329)ErrorField(338-345)Bedrock(38-38)ListModelsRequest(93-93)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)core/schemas/utils.go (1)
ApplyPagination(627-681)
core/providers/azure.go (5)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (1)
Provider(197-224)core/schemas/utils.go (2)
Ptr(17-19)ApplyPagination(627-681)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(252-260)
core/providers/cohere.go (4)
core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (6)
RequestType(79-79)ChatCompletionRequest(84-84)BifrostError(320-329)Cohere(39-39)ListModelsRequest(93-93)ErrorField(338-345)core/schemas/providers/cohere/types.go (2)
CohereListModelsResponse(539-542)CohereError(521-525)
core/schemas/providers/bedrock/models.go (4)
core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (3)
BifrostListModelsResponse(16-25)Model(27-44)Architecture(46-52)core/schemas/utils.go (1)
Ptr(17-19)
🔇 Additional comments (28)
core/schemas/account.go (1)
24-24: Runtime defaulting and constant reference are already implemented.Verification confirms:
DefaultAzureAPIVersionconstant defined atcore/schemas/providers/azure/types.go:4with value"2024-10-21"✓- Runtime enforcement already in place at three locations in
core/providers/azure.go(lines 96-98, 226-228, 322-324), each correctly checking for nil and usingschemas.Ptr(azure.DefaultAzureAPIVersion)✓- Constant reference is already the implementation; no hardcoding ✓
- Single source of truth maintained ✓
The doc comment update at line 24 is accurate. No additional code changes are required beyond what is already shown in the PR.
transports/bifrost-http/integrations/genai.go (2)
67-89: LGTM! Well-structured list models route.The route configuration correctly:
- Uses GET method for the list models endpoint
- Wraps
BifrostListModelsRequestinto theBifrostRequestcontainer- Applies consistent error mapping via
gemini.ToGeminiError- Leverages the
extractGeminiListModelsParamsPreCallback to populate provider-specific parameters
156-177: LGTM! Query parameter extraction is correct.The function properly:
- Sets the provider to Gemini
- Parses the
pageSizequery parameter with error tolerance (ignores parse errors)- Extracts the
pageTokenquery parameter- Returns descriptive errors for invalid request types
core/schemas/providers/anthropic/types.go (1)
272-285: LGTM! Anthropic model types are well-defined.The types correctly:
- Include proper JSON struct tags matching Anthropic's API format
- Use
time.Timefor thecreated_attimestamp- Define pagination fields (
first_id,has_more,last_id) consistent with Anthropic's response structurecore/schemas/providers/azure/models.go (1)
5-22: LGTM! Correct conversion implementation.The method correctly:
- Guards against nil input
- Preallocates the
Dataslice for efficiency- Uses
schemas.Ptr()to safely create pointers (avoiding loop variable capture)- Prefixes model IDs with the provider namespace (
"Azure/")core/schemas/providers/mistral/models.go (1)
14-23: LGTM! Pointer capture bug has been resolved.The implementation now correctly uses
schemas.Ptr()for all pointer fields, avoiding the loop variable capture issue flagged in the previous review. The conversion properly maps:
- Model ID with provider prefix
- String fields via
schemas.Ptr()- Integer fields with type conversions wrapped in
schemas.Ptr()core/schemas/providers/vertex/types.go (1)
48-62: LGTM! Vertex model types are properly defined.The types correctly:
- Include JSON struct tags matching Vertex AI's API format
- Use
time.TimeforversionCreateTime- Support pagination with
next_page_token- Follow naming conventions consistent with Vertex AI's response structure
core/schemas/providers/anthropic/models.go (2)
9-30: LGTM! Correct bidirectional conversion with proper pointer handling.The
ToBifrostListModelsResponsemethod correctly:
- Guards against nil input
- Uses the flexible
providerKeyparameter for ID prefixing- Leverages
schemas.Ptr()to avoid pointer capture bugs- Converts
CreatedAttimestamp to Unix epoch- Propagates pagination fields (
FirstID,LastID,HasMore)
32-61: LGTM! Reverse conversion handles optional fields correctly.The
ToAnthropicListModelsResponsefunction correctly:
- Guards against nil input
- Handles optional fields with nil checks before dereferencing
- Converts Unix epoch back to
time.Time- Preserves pagination semantics
- Preallocates slices for efficiency
core/providers/sgl.go (1)
199-202: LGTM! Appropriate stub for unsupported operation.The method correctly returns an unsupported operation error, consistent with the pattern used for other unsupported operations in this provider (e.g.,
Speech,Transcription). This maintains API surface consistency while clearly communicating that SGL doesn't support model listing.core/schemas/providers/gemini/types.go (2)
1356-1359: Critical: Missing JSON struct tags on GeminiListModelsResponse.The
GeminiListModelsResponsestruct lacks JSON tags forNextPageToken, causing it to be marshaled as"NextPageToken"instead of the conventional"nextPageToken"or"next_page_token".Apply this diff:
type GeminiListModelsResponse struct { - Models []GeminiModel `json:"models"` - NextPageToken string `json:"nextPageToken"` + Models []GeminiModel `json:"models"` + NextPageToken string `json:"next_page_token"` }Note: Verify whether Gemini uses
"next_page_token"or"nextPageToken".Likely an incorrect or invalid review comment.
1340-1354: ****The review comment claims the
GeminiModelstruct lacks JSON tags, but the provided code snippet clearly shows all JSON tags are present and correct. The tags already match the actual Gemini API field names in camelCase format (baseModelId, displayName, inputTokenLimit, outputTokenLimit, supportedGenerationMethods, maxTemperature, topP, topK). No changes are needed.Likely an incorrect or invalid review comment.
core/providers/parasail.go (1)
166-169: Unsupported ListModels: ensure consistent HTTP semantics.Returning newUnsupportedOperationError is fine; confirm the transport maps it to 501 Not Implemented (or 405) and still sets ExtraFields.RequestType=ListModelsRequest for observability.
core/schemas/bifrost.go (1)
93-94: New RequestType looks good.
No issues with the ListModelsRequest constant.transports/bifrost-http/handlers/inference.go (1)
277-279: Route addition looks good.
Registers GET /v1/models with existing middleware chain.core/providers/groq.go (1)
224-237: LGTM on Groq ListModels.Delegation to shared OpenAI list models handler is consistent with other providers.
core/providers/gemini.go (1)
939-961: ListModels flow wiring looks correct.
The earlier compile-time issues appear resolved; response fields populated and returned properly.core/providers/anthropic.go (1)
789-814: ListModels orchestration looks good.
Converts provider response, sets ExtraFields, and returns.core/schemas/providers/vertex/models.go (1)
5-25: LGTM!The conversion logic is correct and follows the established cross-provider pattern. Preallocation, nil-safety, and ID prefixing are properly implemented.
core/schemas/providers/bedrock/models.go (1)
5-27: LGTM!The conversion correctly uses
schemas.Ptrto avoid range variable aliasing issues (past review addressed). Architecture field with modalities provides useful model capabilities metadata.core/providers/cohere.go (1)
798-902: LGTM!The ListModels implementation correctly includes operation authorization check (past review addressed) and properly handles Cohere-specific query parameters (
endpoint,default_only) via ExtraParams. The flow uses provider-specific types and converters consistently.core/providers/bedrock.go (2)
89-167: LGTM!The updated
completeRequestsignature withmethodanduseRuntimeEndpointparameters provides the flexibility needed for ListModels (GET to non-runtime endpoint) while maintaining backward compatibility. Conditional marshaling and body handling for non-GET requests is correct.
948-991: LGTM!ListModels correctly uses
completeRequestwithGET,nilbody, anduseRuntimeEndpoint=falseto query the bedrock service (not bedrock-runtime)/foundation-modelsendpoint. Response conversion and pagination are properly implemented.core/schemas/models.go (1)
1-80: LGTM!The ListModels type definitions are well-structured with comprehensive model metadata fields. Good use of
omitemptyfor optional fields andjson:"-"for ExtraParams. Provider-specific fields (e.g., Anthropic pagination) are properly isolated as optional pointers.core/providers/azure.go (3)
96-98: Good: API version centralized to constant.Using
azure.DefaultAzureAPIVersionconstant addresses the past review concern about duplicated hardcoded values and version drift. This ensures consistency across the codebase.
67-167: LGTM!The updated
completeRequestsignature withmethodanduseDeploymentparameters correctly handles both deployment-based and resource-level endpoints. Conditional marshaling and body handling for non-GET requests is appropriate.
439-472: LGTM!ListModels correctly queries the resource-level
/openai/modelsendpoint withuseDeployment=false. Response conversion via Azure-specific types, pagination, and metadata population are properly implemented.core/providers/openai.go (1)
1780-1854: LGTM!The ListModels implementation follows OpenAI provider patterns consistently: operation authorization check, provider-specific types with converter (
OpenAIListModelsResponse.ToBifrostListModelsResponse), proper error handling, and metadata population. Clean separation between public method and internal handler.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
8618bf8 to
40d6d8f
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
transports/bifrost-http/integrations/utils.go (1)
305-331: Avoid panic on GET: don’t call GetRequestTypeInstance when nil.Guard the factory for GET routes; allow nil req for Pre/Post/Converter if needed.
Apply this diff:
@@ func (g *GenericRouter) createHandler(config RouteConfig) fasthttp.RequestHandler { - req := config.GetRequestTypeInstance() + var req interface{} + if method != fasthttp.MethodGet { + if config.GetRequestTypeInstance == nil { + g.sendError(ctx, config.ErrorConverter, newBifrostError(nil, "invalid route config: GetRequestTypeInstance is nil for non-GET")) + return + } + req = config.GetRequestTypeInstance() + } else if config.GetRequestTypeInstance != nil { + // Optional: allow GET routes to provide a request instance for PreCallback + req = config.GetRequestTypeInstance() + } @@ - if method != fasthttp.MethodGet { + if method != fasthttp.MethodGet { if config.RequestParser != nil {
♻️ Duplicate comments (5)
core/providers/openrouter.go (1)
237-247: Consider consistency: use OpenRouter-specific types and converter if they exist.The code currently unmarshals directly into
schemas.BifrostListModelsResponse. A past review comment suggested that OpenRouter should follow the same pattern as other providers (like Gemini, Cohere, etc.) which have their own provider-specific response types and converter methods.If
core/schemas/providers/openrouter/types.godefinesOpenRouterModelListResponseandcore/schemas/providers/openrouter/models.gohas aToBifrostListModelsResponseconverter, please use them for consistency:+import ( + openrouter "github.com/maximhq/bifrost/core/schemas/providers/openrouter" +) + - var openrouterResponse schemas.BifrostListModelsResponse - if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { + var openrouterResponse openrouter.OpenRouterModelListResponse + if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } var rawResponse interface{} if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { return nil, nil, latency, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } - return &openrouterResponse, rawResponse, latency, nil + bifrostResponse := openrouterResponse.ToBifrostListModelsResponse() + return bifrostResponse, rawResponse, latency, nilThis maintains consistency with the conversion patterns used by other providers in this PR.
core/providers/mistral.go (1)
180-203: ListModels: add nil-guard and document client-side pagination.
Avoid a possible nil deref if callers pass nil request; also add a brief comment noting Mistral lacks server pagination (handled client-side).Apply this diff:
func (provider *MistralProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - mistralResponse, rawResponse, latency, err := provider.handleMistralListModelsRequest(ctx, key) + if request == nil { + request = &schemas.BifrostListModelsRequest{} + } + // Note: Mistral /v1/models has no pagination params; we fetch all and apply client-side pagination. + mistralResponse, rawResponse, latency, err := provider.handleMistralListModelsRequest(ctx, key) @@ - response = schemas.ApplyPagination(response, request.PageSize, request.PageToken) + response = schemas.ApplyPagination(response, request.PageSize, request.PageToken)Based on learnings.
core/schemas/providers/anthropic/models.go (1)
21-27: Set Object to "model" for consistency.Other providers set Object on model list items; add it here.
Apply this diff:
@@ - for _, model := range response.Data { - bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ - ID: string(providerKey) + "/" + model.ID, - Name: schemas.Ptr(model.DisplayName), - Created: schemas.Ptr(int(model.CreatedAt.Unix())), - }) - } + for _, model := range response.Data { + bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ + ID: string(providerKey) + "/" + model.ID, + Object: "model", + Name: schemas.Ptr(model.DisplayName), + Created: schemas.Ptr(int(model.CreatedAt.Unix())), + }) + }core/schemas/providers/mistral/types.go (1)
7-7: ChangeCreatedfield type frominttoint64for Unix timestamps.Unix timestamps should be
int64in Go to avoid the Year 2038 problem on 32-bit systems and ensure cross-platform portability. The Mistral API returns integer timestamps (seconds since epoch), which should be stored asint64.Apply this diff:
- Created int `json:"created"` + Created int64 `json:"created"`core/bifrost.go (1)
1206-1215: Critical: Race condition in provider append - use CAS loop.The current load-append-store pattern is not atomic. Concurrent
prepareProvidercalls can lose updates when two threads:
- Both load the current providers slice
- Each appends their provider
- Both store their updated slice (last write wins, first update lost)
This can cause providers to silently disappear when multiple providers are prepared concurrently during initialization or runtime updates.
Replace with an atomic compare-and-swap loop:
- // Add provider to list of providers - loadedProviders := bifrost.providers.Load() - if loadedProviders == nil { - // Initialize if somehow nil - emptyProviders := make([]schemas.Provider, 0) - loadedProviders = &emptyProviders - } - currentProviders := *loadedProviders - updatedProviders := append(currentProviders, provider) - bifrost.providers.Store(&updatedProviders) + // Atomically append provider to the providers slice using CAS loop + for { + oldPtr := bifrost.providers.Load() + var oldSlice []schemas.Provider + if oldPtr != nil { + oldSlice = *oldPtr + } + // Create new slice with appended provider + newSlice := make([]schemas.Provider, len(oldSlice)+1) + copy(newSlice, oldSlice) + newSlice[len(oldSlice)] = provider + // Attempt atomic swap; retry if another thread modified it + if bifrost.providers.CompareAndSwap(oldPtr, &newSlice) { + break + } + }
🧹 Nitpick comments (8)
transports/bifrost-http/handlers/inference.go (1)
289-328: Consider extracting extra query parameters for provider-specific options.The past review comment suggested extracting extra query parameters into
ExtraParamsto support provider-specific parameters. While the handler now extractspage_sizeandpage_token, it doesn't capture additional custom parameters that providers might support.If provider-specific parameters are needed, consider adding:
bifrostListModelsReq := &schemas.BifrostListModelsRequest{ Provider: schemas.ModelProvider(provider), PageSize: pageSize, PageToken: pageToken, } + + // Extract any extra query parameters (excluding known ones) + bifrostListModelsReq.ExtraParams = make(map[string]interface{}) + ctx.QueryArgs().VisitAll(func(key, value []byte) { + k := string(key) + if k != "provider" && k != "page_size" && k != "page_token" { + bifrostListModelsReq.ExtraParams[k] = string(value) + } + })Note: Only add this if providers require custom query parameters for listing models.
transports/bifrost-http/integrations/genai.go (1)
163-167: Consider handling invalid pageSize values.The
strconv.Atoierror is silently ignored, meaning invalidpageSizevalues (e.g.,"abc") will default to0. While this may be intentionally permissive, consider:
- Returning an error for clearly invalid input
- Or at minimum, logging a warning when parsing fails
- Or documenting this lenient behavior if intentional
Apply this diff to return an error for invalid pageSize:
// Extract pageSize from query parameters (Gemini uses pageSize instead of limit) if pageSizeStr := string(ctx.QueryArgs().Peek("pageSize")); pageSizeStr != "" { - if pageSize, err := strconv.Atoi(pageSizeStr); err == nil { + pageSize, err := strconv.Atoi(pageSizeStr) + if err != nil { + return fmt.Errorf("invalid pageSize parameter: %w", err) + } - listModelsReq.PageSize = pageSize - } + listModelsReq.PageSize = pageSize }Alternatively, if the lenient behavior is intended, consider adding a comment explaining that invalid values are silently ignored and treated as unlimited.
core/schemas/providers/mistral/models.go (1)
19-20: Optional: guard against int width narrowing for Created/MaxContextLength.
If upstream types are int64, consider keeping int64 in schema to avoid narrowing on 32‑bit builds; otherwise document Created as Unix seconds int.transports/bifrost-http/integrations/openai.go (2)
325-361: OpenAI list-models route registered, but query params aren’t parsed.
Clients can’t influence pagination (page_size/page_token) via query yet. Add lightweight extraction to honor PR goals.Apply this diff to parse common params via PreCallback:
func CreateOpenAIListModelsRouteConfigs(pathPrefix string, handlerStore lib.HandlerStore) []RouteConfig { @@ - PreCallback: createOpenAIListModelsPreCallback(handlerStore), + PreCallback: createOpenAIListModelsPreCallback(handlerStore), }) }(See next comment for pre-callback changes.)
--- `362-387`: **Pre-callback should parse page_size/page_token; Azure path provider override needs confirmation.** Currently it only defaults Provider to openai. Consider: - Read page_size and page_token from query args. - If this route is used for Azure deployments path, decide whether Provider should be set to Azure when x-bf-azure-endpoint or deployment-id is present; otherwise requests may route to OpenAI. Proposed pre-callback tweak: ```diff func createOpenAIListModelsPreCallback(handlerStore lib.HandlerStore) PreRequestCallback { azureHook := AzureEndpointPreHook(handlerStore) return func(ctx *fasthttp.RequestCtx, req interface{}) error { if azureHook != nil { if err := azureHook(ctx, req); err != nil { return err } } - // Then extract query parameters for list models + // Then extract query parameters for list models if listModelsReq, ok := req.(*schemas.BifrostListModelsRequest); ok { - // Set provider to OpenAI (may be overridden by Azure hook) - if listModelsReq.Provider == "" { - listModelsReq.Provider = schemas.OpenAI - } + // Default provider if not provided elsewhere + if listModelsReq.Provider == "" { + listModelsReq.Provider = schemas.OpenAI + } + // Common pagination params + if v := string(ctx.QueryArgs().Peek("page_size")); v != "" { + if n, err := strconv.Atoi(v); err == nil { + listModelsReq.PageSize = n + } + } + if v := string(ctx.QueryArgs().Peek("page_token")); v != "" { + listModelsReq.PageToken = v + } + // Optional: allow explicit provider override via query (?provider=...) + if v := string(ctx.QueryArgs().Peek("provider")); v != "" { + listModelsReq.Provider = schemas.ModelProvider(v) + } + // TODO: if Azure deployment headers/path are present for this route, + // confirm whether Provider should be set to Azure to route correctly. return nil } return nil } }Would you like me to adjust this further to force Azure provider when deployment-id/x-bf-azure-endpoint are present?
core/providers/mistral.go (2)
227-229: Error parsing uses OpenAI parser; consider provider-specific parser.
Mistral error shapes may differ; parsing via parseOpenAIError could misclassify. Prefer a Mistral-aware parser or a generic HTTP error mapper.
215-219: Nit: content-type on GET.
Setting Content-Type for a GET isn’t needed; safe to drop.core/bifrost.go (1)
2144-2207: LGTM! Key selection correctly handles ListModelsRequest.The updated
selectKeyFromProviderForModelsignature includesrequestTypeand correctly branches logic:
- For
ListModelsRequest: Uses all keys without model/deployment filtering (line 2165) since listing models doesn't target a specific model- For other request types: Applies model matching and Azure/Bedrock deployment checks as before
The implementation is correct and handles the new use case appropriately.
Consider adding a brief comment before line 2164 to clarify the intent:
+ // For ListModels, all keys are eligible since no specific model is required. + // For other requests, filter by model support and deployment mappings. if requestType == schemas.ListModelsRequest { supportedKeys = keys } else {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (49)
core/bifrost.go(8 hunks)core/providers/anthropic.go(1 hunks)core/providers/azure.go(10 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(2 hunks)core/providers/gemini.go(2 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(3 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/schemas/utils.go(2 hunks)docs/features/custom-providers.mdx(2 hunks)tests/core-providers/config/account.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(7 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (15)
- ui/lib/types/config.ts
- core/schemas/providers/cohere/models.go
- core/providers/parasail.go
- core/schemas/providers/openai/types.go
- core/schemas/providers/openai/models.go
- core/schemas/bifrost.go
- core/schemas/providers/bedrock/models.go
- core/schemas/providers/vertex/models.go
- core/providers/sgl.go
- ui/lib/constants/config.ts
- core/schemas/providers/bedrock/types.go
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- ui/lib/types/schemas.ts
- core/schemas/account.go
- ui/app/providers/fragments/allowedRequestsFields.tsx
🧰 Additional context used
🧬 Code graph analysis (27)
core/providers/gemini.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (3)
BifrostError(320-329)Gemini(47-47)ListModelsRequest(93-93)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1356-1359)
core/schemas/providers/gemini/models.go (4)
core/schemas/providers/gemini/types.go (2)
GeminiListModelsResponse(1356-1359)GeminiModel(1340-1354)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
Ptr(17-19)
transports/bifrost-http/integrations/openai.go (5)
transports/bifrost-http/integrations/utils.go (8)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)OpenAI(35-35)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/schemas/providers/azure/models.go (4)
core/schemas/providers/azure/types.go (1)
AzureListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Azure(36-36)core/schemas/utils.go (1)
Ptr(17-19)
core/providers/openrouter.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostError(320-329)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(93-93)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)
core/schemas/providers/anthropic/models.go (4)
core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicModel(273-278)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
Ptr(17-19)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (2)
BifrostError(320-329)Groq(43-43)
core/providers/ollama.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(27-44)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
core/providers/azure.go (4)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (1)
Provider(197-224)core/schemas/utils.go (2)
Ptr(17-19)ApplyPagination(627-684)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(35-60)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/integrations/anthropic.go (5)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(32-61)
core/providers/vertex.go (6)
core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/providers/vertex/types.go (1)
VertexListModelsResponse(59-62)core/schemas/bifrost.go (4)
BifrostError(320-329)ListModelsRequest(93-93)ErrorField(338-345)RequestCancelled(280-280)core/schemas/utils.go (2)
ApplyPagination(627-684)Ptr(17-19)core/schemas/provider.go (5)
Provider(197-224)ErrRequestCancelled(23-23)ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(252-260)
core/schemas/utils.go (1)
core/schemas/models.go (2)
Model(27-44)BifrostListModelsResponse(16-25)
core/providers/cohere.go (4)
core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (5)
RequestType(79-79)BifrostError(320-329)Cohere(39-39)ListModelsRequest(93-93)ErrorField(338-345)core/schemas/providers/cohere/types.go (2)
CohereListModelsResponse(539-542)CohereError(521-525)
core/providers/mistral.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)Mistral(41-41)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)
core/bifrost.go (5)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (8)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(320-329)BifrostResponseExtraFields(252-260)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
ApplyPagination(627-684)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (5)
BifrostError(320-329)ErrorField(338-345)Bedrock(38-38)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)core/schemas/utils.go (1)
ApplyPagination(627-684)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/bifrost.go (2)
BifrostRequest(130-140)ListModelsRequest(93-93)core/schemas/models.go (1)
BifrostListModelsResponse(16-25)
core/providers/anthropic.go (4)
core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (3)
BifrostError(320-329)Anthropic(37-37)ListModelsRequest(93-93)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicError(302-308)
transports/bifrost-http/handlers/inference.go (5)
transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)BifrostListModelsRequest(3-14)core/schemas/bifrost.go (3)
BifrostError(320-329)ModelProvider(32-32)ListModelsRequest(93-93)core/schemas/provider.go (1)
Provider(197-224)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/schemas/provider.go (3)
core/schemas/bifrost.go (1)
ListModelsRequest(93-93)core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)
core/schemas/providers/mistral/models.go (4)
core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Mistral(41-41)core/schemas/utils.go (1)
Ptr(17-19)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(136-139)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/config/account.go (2)
core/schemas/utils.go (1)
Ptr(17-19)core/utils.go (1)
Ptr(12-14)
🔇 Additional comments (36)
ui/app/providers/fragments/apiStructureFormFragment.tsx (1)
48-48: LGTM!The addition of
list_modelsto the form default values is consistent with the existing pattern for other allowed request types, and the default value oftruealigns with the other fields.core/schemas/providers/gemini/types.go (1)
1339-1359: LGTM!The new model types are well-defined with appropriate JSON tags and field types for the Gemini list models response structure.
core/schemas/providers/vertex/types.go (1)
50-62: LGTM!The new Vertex model types are properly defined with appropriate field types including the
time.TimeforVersionCreateTime, and the necessarytimeimport has been added.core/schemas/providers/cohere/types.go (1)
527-542: LGTM!The Cohere model types are properly structured with appropriate field types and JSON tags, consistent with the patterns used in other provider implementations.
core/schemas/providers/gemini/models.go (2)
9-33: LGTM!The pointer bug from the previous review has been properly addressed. The
contextLengthvariable is now declared inside the loop (line 20), ensuring each model gets its own distinct pointer value. TheDataslice is also correctly preallocated with the appropriate capacity.
35-60: LGTM!The reverse conversion function is well-implemented with proper nil checks, preallocation, and conditional field assignments.
core/providers/openrouter.go (1)
184-208: LGTM!The
ListModelsmethod properly handles pagination, sets extra fields including provider, request type, and latency, and conditionally includes the raw response when enabled.tests/core-providers/config/account.go (1)
163-165: The API version "2024-10-21" is confirmed as a valid GA (stable) Azure OpenAI API version and is the latest GA release. The code update is correct.core/schemas/utils.go (1)
552-684: LGTM! Pagination implementation is correct.The pagination logic properly handles:
- Empty page returns when offset exceeds data bounds (no wrap-around to page 1)
- Cursor validation with LastID integrity checks
- Next token generation only when more data exists
- Graceful handling of invalid/empty tokens
The past review concerns about looping back to page 1 and debug prints have been properly addressed.
core/schemas/models.go (1)
1-80: LGTM! Well-structured schema definitions.The models are well-designed with:
- Appropriate use of pointer types and
omitemptyfor optional fields- Proper JSON exclusion for internal fields (ExtraParams with
json:"-")- Comprehensive metadata coverage across providers
- Consistent naming conventions
core/providers/bedrock.go (3)
89-118: LGTM! Request marshaling and error handling are correct.The updates properly address the past review concern:
- Removed the unreachable
context.DeadlineExceededcheck on JSON marshaling errors- Marshaling now conditional on
requestBody != nil(appropriate for GET requests)- Error handling returns proper BifrostError with
IsBifrostError: true
120-127: LGTM! URL construction correctly distinguishes runtime vs control plane endpoints.The conditional endpoint selection:
bedrock-runtimefor model invocations (whenuseRuntimeEndpoint = true)bedrockfor control plane operations like listing models (whenuseRuntimeEndpoint = false)This aligns with AWS Bedrock's API architecture.
948-991: LGTM! ListModels implementation follows best practices.The implementation is well-structured:
- Proper operation permission checks
- Correct use of control plane endpoint (
useRuntimeEndpoint = false)- Standard response parsing and conversion pipeline
- Appropriate pagination application
- Consistent metadata population (Provider, Latency, RequestType)
- Optional raw response exposure
docs/features/custom-providers.mdx (1)
71-72: LGTM! Documentation correctly includes the new list_models option.The
list_modelsfield has been properly added to theallowed_requestsconfiguration examples with valid JSON syntax. The past trailing comma issues have been resolved.Also applies to: 103-104
core/providers/cerebras.go (1)
183-186: LGTM! ListModels delegation is appropriate.The implementation correctly delegates to the shared OpenAI-compatible handler, consistent with:
- Cerebras's OpenAI-compatible API design
- Other methods in this provider (TextCompletion, ChatCompletion, etc.)
- The established pattern for OpenAI-compatible providers
core/schemas/providers/anthropic/types.go (1)
272-286: LGTM! Type definitions align with Anthropic's API.The new structs are well-defined:
AnthropicModelcaptures essential model metadataAnthropicListModelsResponseincludes proper pagination fields (FirstID, HasMore, LastID)- Use of
time.TimeforCreatedAtis appropriate for type safetytransports/bifrost-http/integrations/genai.go (1)
67-90: LGTM! List models route is properly configured.The route configuration correctly:
- Uses GET method for the /v1beta/models endpoint
- Employs appropriate request/response converters
- Uses
gemini.ToGeminiErrorfor error conversion (past issue resolved)- Includes PreCallback for query parameter extraction
Consistent with Gemini's API patterns and the existing chat completions route.
core/schemas/providers/azure/models.go (1)
14-20: I need to see the full ID assignment logic for the remaining providers to confirm consistency.Now I need to search the codebase directly for the remaining provider ID assignments:
Based on my script results, I have concrete evidence of the provider ID prefixing pattern. Let me provide the rewritten review comment:
Confirm ID format consistency across all provider implementations.
The conversion prefixes model IDs with provider names (e.g.,
"Azure/","Mistral/","Vertex/"). The current Azure implementation is consistent with how Mistral and Vertex handle model IDs in their respective converters. However, verify that OpenAI, Bedrock, and Gemini converters follow the same provider-prefixed naming convention to ensure uniform model identification across all providers in aggregated responses.core/schemas/providers/mistral/models.go (1)
15-22: Good fix: no loop-variable pointer capture; pre-sized slice.
Using schemas.Ptr avoids the classic range-pointer bug; capacity pre-allocation is correct.core/schemas/provider.go (1)
90-101: AllowedRequests: list_models gate integrated correctly.
Switch case and JSON field look consistent with existing ops.Also applies to: 108-132
transports/bifrost-http/integrations/openai.go (2)
104-104: RouteConfig type annotations added.
These Type fields improve introspection/routing and are fine.Also applies to: 143-143, 183-183, 223-223, 253-253, 289-289
392-393: Router wiring includes list-models routes.
Appends list-models alongside existing routes; looks correct.core/providers/ollama.go (1)
199-202: ListModels delegates to shared OpenAI-style handler — good.
URL, headers, provider key, and raw-response flags are passed correctly.transports/bifrost-http/integrations/anthropic.go (3)
23-43: Type annotation for route config aligns with other integrations.
No issues.
76-102: Anthropic list-models route wiring looks correct.
GET /anthropic/v1/models with proper converters and pre-callback.
104-134: Param extraction covers limit/before_id/after_id.
Provider is set; PageSize mapping is correct; ExtraParams prepared for backend.If Anthropic supports page_token in addition to before_id/after_id, we can add it similarly; confirm desired behavior.
core/providers/groq.go (1)
224-237: LGTM — ListModels wiring is consistent and minimal.core/providers/openai.go (1)
1780-1854: LGTM! ListModels implementation follows established patterns.The implementation correctly:
- Checks operation permissions via
checkOperationAllowed- Uses GET method for the
/v1/modelsendpoint- Conditionally sets Authorization header only when
key.Valueis non-empty- Handles errors via
parseOpenAIError(consistent with other operations)- Converts OpenAI response to Bifrost format via
ToBifrostListModelsResponse- Applies pagination with
schemas.ApplyPagination- Populates extra fields (Provider, RequestType, Latency, optional RawResponse)
core/schemas/providers/azure/types.go (1)
1-34: LGTM! Azure model types are well-structured.The type definitions correctly model Azure's models API response structure with appropriate JSON tags. The
DefaultAzureAPIVersionconstant provides centralized API version management as discussed in previous reviews.core/providers/azure.go (3)
67-167: LGTM! RefactoredcompleteRequestfor flexibility.The changes correctly generalize
completeRequestto support:
- Different HTTP methods (GET, POST, etc.) via the
methodparameter- Non-deployment endpoints (like models listing) via
useDeployment=false- Optional request body (marshaled only when non-nil)
The conditional body/content-type setting (lines 139-142) for non-GET requests is appropriate. URL construction properly branches between deployment-based and non-deployment paths.
179-394: LGTM! Call sites correctly updated.All
completeRequestinvocations properly specify:
"POST"method for operations with request bodiesuseDeployment=truefor deployment-based endpoints (completions, chat/completions, embeddings)- Consistent use of
azure.DefaultAzureAPIVersionfor API version defaults
439-472: LGTM! Azure ListModels implementation is correct.The method correctly:
- Uses
GETwithnilbody for the models endpoint- Sets
useDeployment=falsefor the resource-level/openai/modelspath (not deployment-specific)- Parses the Azure-specific response via
AzureListModelsResponse- Converts to Bifrost format with
ToBifrostListModelsResponse()- Applies pagination via
schemas.ApplyPagination- Populates extra fields (Provider, Latency, RequestType, optional RawResponse)
core/bifrost.go (4)
38-38: LGTM! Providers field initialization is correct.The
providersfield is properly declared asatomic.Pointer[[]schemas.Provider]and initialized with an empty slice duringInit.Also applies to: 96-97
211-274: LGTM! ListModelsRequest implementation is solid.The method correctly:
- Validates request (non-nil check, provider required)
- Copies all request fields (Provider, PageSize, PageToken, ExtraParams)
- Retrieves provider instance via
getProviderByKey- Determines base provider type for custom providers (for key requirement checks)
- Conditionally obtains API key only when
providerRequiresKeyreturns true- Passes
schemas.ListModelsRequestasrequestTypetoselectKeyFromProviderForModel(important for key filtering logic)- Delegates to provider-specific
ListModelsimplementation
276-368: LGTM! ListAllModels correctly aggregates models with pagination safety.The method properly:
- Retrieves all configured providers
- Implements per-provider pagination with page size 1000
- Includes safety limit (
maxIterations = 20) to prevent infinite loops- Logs warnings when hitting pagination limits or provider errors
- Continues with other providers on individual failures (tracks first error)
- Aggregates all models and sorts alphabetically by ID for consistency
- Measures total elapsed time across all provider requests
- Applies final pagination on the aggregated result set
1269-1284: LGTM! Provider lookup is thread-safe and correct.The
getProviderByKeyhelper safely loads from the atomicproviderspointer and performs a simple linear search by provider key. This is appropriate for the expected number of providers (typically < 20).
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
40d6d8f to
4b86a8b
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
transports/bifrost-http/integrations/utils.go (1)
308-332: Critical: GET routes will panic at line 312.The validation (lines 252-263) allows GET routes to have nil GetRequestTypeInstance, but line 312 unconditionally calls it without checking. This will cause a nil pointer panic for GET routes.
Apply this fix:
func (g *GenericRouter) createHandler(config RouteConfig) fasthttp.RequestHandler { return func(ctx *fasthttp.RequestCtx) { method := string(ctx.Method()) - // Parse request body into the integration-specific request type - // Note: config validation is performed at startup in RegisterRoutes - req := config.GetRequestTypeInstance() + var req interface{} // Parse request body based on configuration if method != fasthttp.MethodGet { + // Parse request body into the integration-specific request type + // Note: config validation is performed at startup in RegisterRoutes + req = config.GetRequestTypeInstance() + if config.RequestParser != nil { // Use custom parser (e.g., for multipart/form-data) if err := config.RequestParser(ctx, req); err != nil {
♻️ Duplicate comments (2)
core/providers/vertex.go (1)
637-660: Critical: Pagination params not wired to Vertex API; breaks pagination.This is the same issue flagged in past reviews and remains unaddressed:
- Line 639:
handleVertexListModelsRequest(ctx, key)doesn't receive therequestparameter, so PageSize and PageToken never reach Vertex API- Line 647: Local re-pagination with
ApplyPaginationbreaks NextPageToken semantics and wastes bandwidth by fetching all modelsThis requires the fix from the past review (see past review comments section). You must:
- Pass
requesttohandleVertexListModelsRequest(update both the call at line 639 and the signature at line 662)- Remove the local
ApplyPaginationcall (line 647)- Build proper query parameters with
pageSizeandpageTokenin the handler (see past review for specific diff)- Add
net/urlimport for proper query encodingThe past review comment provides the complete diff - please apply it.
core/bifrost.go (1)
1206-1215: Race condition: concurrent provider appends can lose updates.The current load-append-store pattern can drop concurrent updates when multiple providers are initialized simultaneously. Although
prepareProvideris called within aproviderMutexlock, that mutex is per-provider (keyed byproviderKey). Concurrent calls for different providers (e.g., provider A and provider B) use different mutexes and can race on the sharedbifrost.providersatomic pointer:
- Thread 1: prepares provider A → loads providers → appends A
- Thread 2: prepares provider B → loads providers → appends B
- Both threads store their slices; one overwrites the other
Apply this diff to use a compare-and-swap loop:
for range providerConfig.ConcurrencyAndBufferSize.Concurrency { currentWaitGroup.Add(1) go bifrost.requestWorker(provider, providerConfig, queue) } - // Add provider to list of providers - loadedProviders := bifrost.providers.Load() - if loadedProviders == nil { - // Initialize if somehow nil - emptyProviders := make([]schemas.Provider, 0) - loadedProviders = &emptyProviders - } - currentProviders := *loadedProviders - updatedProviders := append(currentProviders, provider) - bifrost.providers.Store(&updatedProviders) + // Atomically append provider using CAS loop + for { + oldPtr := bifrost.providers.Load() + var oldSlice []schemas.Provider + if oldPtr != nil { + oldSlice = *oldPtr + } + newSlice := make([]schemas.Provider, len(oldSlice)+1) + copy(newSlice, oldSlice) + newSlice[len(oldSlice)] = provider + if bifrost.providers.CompareAndSwap(oldPtr, &newSlice) { + break + } + }
🧹 Nitpick comments (5)
core/schemas/providers/vertex/types.go (1)
48-62: LGTM! Consider adding field documentation.The new model listing types are well-structured and follow Go conventions. The use of value types for response structs is appropriate, and the
NextPageTokenfield properly supports pagination.Consider adding godoc comments for the struct fields to improve maintainability:
type VertexModel struct { - Name string `json:"name"` - VersionId string `json:"versionId"` - VersionAliases []string `json:"versionAliases"` - VersionCreateTime time.Time `json:"versionCreateTime"` - DisplayName string `json:"displayName"` - Description string `json:"description"` + Name string `json:"name"` // Resource name of the model + VersionId string `json:"versionId"` // Version identifier + VersionAliases []string `json:"versionAliases"` // List of version aliases + VersionCreateTime time.Time `json:"versionCreateTime"` // Timestamp when the version was created + DisplayName string `json:"displayName"` // Human-readable display name + Description string `json:"description"` // Model description } type VertexListModelsResponse struct { - Models []VertexModel `json:"models"` - NextPageToken string `json:"next_page_token"` + Models []VertexModel `json:"models"` // List of models + NextPageToken string `json:"next_page_token"` // Token for fetching the next page of results }core/schemas/providers/bedrock/types.go (1)
434-434: Fix the section header to use singular form.The section header uses "MODELS TYPES" but all other sections in this file use singular form ("REQUEST TYPES", "RESPONSE TYPES", etc.).
Apply this diff:
-// ==================== MODELS TYPES ==================== +// ==================== MODEL TYPES ====================core/schemas/providers/gemini/types.go (1)
1356-1359: Consider adding documentation for consistency.The struct lacks a godoc comment. While the purpose is clear, adding documentation would maintain consistency with other types in this file. Also consider
omitemptyonNextPageTokensince it may be empty on the final page.transports/bifrost-http/integrations/genai.go (1)
156-177: Consider validation for pageSize parameter.The function silently ignores
pageSizeparsing errors, which means invalid values (e.g., non-numeric or negative) default to 0. While this is lenient and acceptable for optional parameters, explicit validation could improve the user experience by returning clear error messages for malformed requests.If you'd like to add validation:
// Extract pageSize from query parameters (Gemini uses pageSize instead of limit) if pageSizeStr := string(ctx.QueryArgs().Peek("pageSize")); pageSizeStr != "" { - if pageSize, err := strconv.Atoi(pageSizeStr); err == nil { + pageSize, err := strconv.Atoi(pageSizeStr) + if err != nil { + return fmt.Errorf("invalid pageSize parameter: %v", err) + } + if pageSize <= 0 { + return fmt.Errorf("pageSize must be positive") + } - listModelsReq.PageSize = pageSize - } + listModelsReq.PageSize = pageSize }transports/bifrost-http/handlers/inference.go (1)
289-328: LGTM with optional improvement suggestion.The handler correctly branches between listing all models (when provider is empty) and listing models for a specific provider. The
pageSizeparsing silently ignores errors and defaults to 0, which is lenient but acceptable for optional parameters.Optional: If you want to support provider-specific query parameters, consider populating
ExtraParams:bifrostListModelsReq := &schemas.BifrostListModelsRequest{ Provider: schemas.ModelProvider(provider), PageSize: pageSize, PageToken: pageToken, } + + // Extract any extra query parameters for provider-specific features + bifrostListModelsReq.ExtraParams = make(map[string]interface{}) + ctx.QueryArgs().VisitAll(func(key, value []byte) { + k := string(key) + if k != "provider" && k != "page_size" && k != "page_token" { + bifrostListModelsReq.ExtraParams[k] = string(value) + } + })This would allow providers like Gemini (which uses
pageSizeinstead ofpage_size) or others to access custom query parameters.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (49)
core/bifrost.go(8 hunks)core/providers/anthropic.go(2 hunks)core/providers/azure.go(10 hunks)core/providers/bedrock.go(6 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(3 hunks)core/providers/gemini.go(3 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(1 hunks)core/providers/openai.go(1 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(3 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(2 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/schemas/utils.go(2 hunks)docs/features/custom-providers.mdx(2 hunks)tests/core-providers/config/account.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(7 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (2)
- core/schemas/providers/gemini/models.go
- core/schemas/account.go
🚧 Files skipped from review as they are similar to previous changes (18)
- core/providers/sgl.go
- ui/lib/constants/config.ts
- ui/app/providers/fragments/allowedRequestsFields.tsx
- core/schemas/providers/cohere/models.go
- core/schemas/bifrost.go
- tests/core-providers/config/account.go
- core/schemas/providers/cohere/types.go
- core/schemas/providers/anthropic/types.go
- core/providers/parasail.go
- docs/features/custom-providers.mdx
- core/providers/gemini.go
- ui/lib/types/schemas.ts
- core/schemas/providers/vertex/models.go
- core/schemas/providers/mistral/types.go
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- core/schemas/provider.go
- core/schemas/providers/bedrock/models.go
🧰 Additional context used
🧬 Code graph analysis (24)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(38-67)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)BifrostListModelsRequest(3-14)core/schemas/bifrost.go (3)
BifrostError(320-329)ModelProvider(32-32)ListModelsRequest(93-93)core/schemas/provider.go (1)
Provider(197-224)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/schemas/providers/azure/models.go (4)
core/schemas/providers/azure/types.go (1)
AzureListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Azure(36-36)core/schemas/utils.go (1)
Ptr(17-19)
core/schemas/providers/mistral/models.go (4)
core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/bifrost.go (1)
Mistral(41-41)core/schemas/utils.go (1)
Ptr(17-19)
core/providers/openrouter.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostError(320-329)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(93-93)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)
core/providers/vertex.go (6)
core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/providers/vertex/types.go (1)
VertexListModelsResponse(59-62)core/schemas/bifrost.go (4)
BifrostError(320-329)ListModelsRequest(93-93)ErrorField(338-345)RequestCancelled(280-280)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (4)
Provider(197-224)ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)
core/schemas/providers/openai/models.go (4)
core/schemas/providers/openai/types.go (2)
OpenAIListModelsResponse(136-139)OpenAIModel(125-134)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
Ptr(17-19)
transports/bifrost-http/integrations/openai.go (5)
transports/bifrost-http/integrations/utils.go (6)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)OpenAI(35-35)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/providers/anthropic.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (3)
BifrostError(320-329)Anthropic(37-37)ListModelsRequest(93-93)core/schemas/provider.go (3)
Provider(197-224)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicError(302-308)
core/providers/mistral.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostError(320-329)RequestType(79-79)ListModelsRequest(93-93)Mistral(41-41)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (2)
Provider(197-224)ErrProviderResponseUnmarshal(25-25)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(252-260)
core/providers/cohere.go (4)
core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (4)
RequestType(79-79)BifrostError(320-329)Cohere(39-39)ListModelsRequest(93-93)core/schemas/providers/cohere/types.go (2)
CohereListModelsResponse(539-542)CohereError(521-525)
core/providers/azure.go (5)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/utils.go (2)
Ptr(17-19)ApplyPagination(627-684)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)
core/providers/openai.go (7)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (5)
BifrostError(320-329)OpenAI(35-35)ListModelsRequest(93-93)ModelProvider(32-32)RequestType(79-79)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(136-139)core/schemas/utils.go (1)
ApplyPagination(627-684)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/anthropic/models.go (4)
core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(280-285)AnthropicModel(273-278)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
Ptr(17-19)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(93-93)BifrostError(320-329)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(35-60)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(27-44)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/bifrost.go (2)
BifrostRequest(130-140)ListModelsRequest(93-93)core/schemas/models.go (1)
BifrostListModelsResponse(16-25)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (2)
BifrostError(320-329)Groq(43-43)
core/bifrost.go (5)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (9)
KeySelector(16-16)ListModelsRequest(93-93)BifrostError(320-329)ErrorField(338-345)BifrostResponseExtraFields(252-260)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)Model(27-44)core/schemas/utils.go (1)
ApplyPagination(627-684)
core/providers/ollama.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/bifrost.go (1)
BifrostError(320-329)
core/schemas/utils.go (1)
core/schemas/models.go (2)
Model(27-44)BifrostListModelsResponse(16-25)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (5)
BifrostError(320-329)ErrorField(338-345)Bedrock(38-38)ListModelsRequest(93-93)RequestType(79-79)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
Model(27-44)BifrostListModelsRequest(3-14)BifrostListModelsResponse(16-25)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(452-454)core/schemas/utils.go (1)
ApplyPagination(627-684)
🔇 Additional comments (49)
ui/lib/types/config.ts (1)
123-123: LGTM! Backend schema alignment verified.The addition of
list_models: booleanto the TypeScriptAllowedRequestsinterface is correctly aligned with the Go backend. The Go schema includesListModels boolwith JSON tag"list_models", which matches the TypeScript addition perfectly.core/schemas/providers/vertex/types.go (1)
3-3: LGTM!The
timeimport is necessary for theVersionCreateTimefield in the newVertexModelstruct.core/schemas/providers/gemini/types.go (1)
1339-1339: LGTM!The section header follows the existing pattern in the file for organizing type definitions.
core/schemas/providers/openai/types.go (1)
124-139: LGTM with observation: GROQ-specific fields in OpenAI types.The structure is correct, and the optional GROQ fields (
Active,ContextWindow) won't affect OpenAI usage due toomitempty. This pattern works for OpenAI-compatible providers that share the same response structure with provider-specific extensions.transports/bifrost-http/integrations/genai.go (1)
67-89: LGTM!The route configuration is complete with all required converters properly wired. The ErrorConverter correctly uses
gemini.ToGeminiErrorfor consistency with other Gemini routes.core/providers/mistral.go (2)
180-203: LGTM!The implementation correctly applies client-side pagination using
schemas.ApplyPagination, which is appropriate since Mistral's/v1/modelsendpoint doesn't support pagination query parameters. All required fields are properly populated.Based on learnings.
205-243: LGTM with minor observation.The implementation correctly constructs and sends the HTTP request to Mistral's
/v1/modelsendpoint. The double unmarshal pattern (lines 232-240) is consistent with other providers in this PR, though it could be optimized by unmarshaling once into raw interface{} and then type-asserting or re-marshaling for the typed structure.core/providers/groq.go (1)
224-237: LGTM!Clean delegation to the shared OpenAI-compatible list models handler with all parameters correctly passed. Consistent with the pattern used by other OpenAI-compatible providers.
core/providers/openrouter.go (1)
184-208: LGTM!The method correctly prefixes model IDs with "openrouter/" for disambiguation, applies client-side pagination, and populates all required extra fields.
core/schemas/providers/azure/types.go (1)
1-34: LGTM!The Azure model types are well-structured with comprehensive metadata. The
DefaultAzureAPIVersionconstant provides a clear default, and the capabilities/deprecation structures offer good granularity for Azure-specific model information.transports/bifrost-http/handlers/inference.go (1)
277-278: LGTM!Route registration is clean and correct, properly chaining middlewares for the GET
/v1/modelsendpoint.core/providers/cerebras.go (1)
183-186: LGTM!Clean delegation to the shared OpenAI-compatible list models handler. All parameters are correctly passed, consistent with other OpenAI-compatible providers in this PR.
core/schemas/providers/azure/models.go (1)
5-22: LGTM!The conversion logic is clean and follows the established pattern. Nil-check, pre-allocation, and proper use of
schemas.Ptr()helper prevent pointer capture issues.core/schemas/providers/mistral/models.go (1)
5-27: LGTM!The pointer capture bug from the previous review has been correctly addressed. All pointer fields now use
schemas.Ptr()helper, ensuring each model gets its own pointer values.core/providers/ollama.go (1)
199-202: LGTM!The delegation to
handleOpenAIListModelsRequestis appropriate since Ollama exposes an OpenAI-compatible API. The implementation follows the same pattern as other methods in this provider.transports/bifrost-http/integrations/anthropic.go (1)
76-102: LGTM!The route configuration follows the established pattern and properly integrates list models support into the Anthropic integration.
core/providers/anthropic.go (2)
791-816: LGTM!The ListModels implementation follows the established provider pattern. Authorization check uses the correct
schemas.ListModelsRequestoperation type, and ExtraFields are properly populated.
856-860: No issues found—nil check concern is invalid; page_token mapping is correct.The nil check at line 857 is safe. In Go, reading from a nil map returns the zero value and
falsefor the second argument—no panic occurs. The code correctly maps the genericPageTokento Anthropic'safter_idparameter (following the same pattern as other providers like Gemini and Cohere), using it as a fallback only whenafter_idis not already in ExtraParams. Anthropic's API uses cursor-based pagination withafter_idandbefore_id, and theNextPageTokenfrom responses correctly feeds back asPageTokenin subsequent requests.core/providers/cohere.go (2)
800-825: LGTM!Authorization check has been correctly added. The implementation follows the established pattern and properly uses URL-encoded query parameters.
856-858: Code is correct as-is; no changes needed.The Cohere API treats the
default_onlyparameter the same whether it's absent or explicitly set to false, so the current implementation that only setsdefault_only=truewhen the boolean is true handles all cases correctly.core/schemas/providers/openai/models.go (1)
5-54: LGTM!Both conversion directions are implemented correctly with proper nil-checks, pre-allocation, and careful pointer handling. The bidirectional mapping preserves data integrity.
transports/bifrost-http/integrations/openai.go (1)
325-360: LGTM!The route configurations properly support multiple OpenAI-compatible paths including Azure deployment-specific endpoints. The implementation follows the established pattern for route registration.
core/schemas/utils.go (4)
552-578: LGTM! Well-designed pagination cursor.The pagination cursor implementation is clean and robust:
- Opaque base64-encoded tokens with LastID validation
- Appropriate handling of zero/negative offsets
- URL-safe encoding without padding
580-603: LGTM! Safe error handling in decode.All error paths correctly return a zero cursor, providing safe defaults and preventing runtime panics.
605-622: LGTM! Cursor validation correctly prevents tampering.The validation logic properly checks that the LastID in the cursor matches the actual last item ID at the expected position, preventing cursor manipulation.
624-684: LGTM! Pagination logic correctly handles all cases.The implementation properly addresses the past review feedback:
- No longer loops back to page 1 when offset >= totalItems
- Returns empty page with no NextPageToken at end
- Correctly generates NextPageToken only when more data exists
- Proper bounds checking and slice operations
transports/bifrost-http/integrations/utils.go (2)
250-275: Route validation correctly requires RequestConverter for all routes.The validation logic properly addresses the past review feedback - RequestConverter is now validated for all routes (lines 272-275), not just non-GET routes. The separation of concerns is appropriate:
- Lines 252-263: Non-GET routes require GetRequestTypeInstance
- Lines 265-269: GET routes require ListModelsResponseConverter
- Lines 272-275: All routes require RequestConverter
391-410: LGTM! ListModels handling is consistent with other request types.The implementation properly follows the same pattern as other request types:
- Invokes the client method
- Applies optional PostCallback
- Validates non-nil response after callback
- Converts using the configured converter
core/schemas/providers/anthropic/models.go (2)
9-36: LGTM! Pointer capture bug correctly fixed.The conversion properly uses
schemas.Ptr(model.DisplayName)instead of&model.DisplayName, avoiding the range-variable pointer capture bug that was flagged in the past review.The ID prefixing with
providerKeyensures model IDs are globally unique across providers.Note: The past review mentioned adding an
Objectfield, but the currentModelstruct (core/schemas/models.go lines 27-44) doesn't include an Object field, so this is not applicable.
38-67: LGTM! Reverse conversion properly handles optional fields.The bidirectional conversion correctly handles nullable fields with conditional checks and appropriate conversions (e.g., Unix timestamp to time.Time).
core/providers/bedrock.go (3)
95-118: LGTM! Conditional marshaling correctly supports GET requests.The updated signature and conditional marshaling logic properly handles requests with and without bodies. The error handling correctly returns a provider JSON marshaling error.
Past review feedback about removing unreachable context deadline checks has been addressed.
120-135: LGTM! URL construction properly routes between bedrock-runtime and bedrock endpoints.The conditional URL construction correctly distinguishes between:
- Runtime endpoints (bedrock-runtime) for model inference
- Control plane endpoints (bedrock) for operations like listing models
Request creation appropriately includes or omits the body based on whether requestBody is provided.
948-991: LGTM! ListModels implementation follows provider patterns.The implementation correctly:
- Uses GET to the bedrock (non-runtime) endpoint
- Converts Bedrock response to Bifrost format
- Applies pagination using ApplyPagination utility
- Populates ExtraFields with provider, latency, and request type
- Optionally includes raw response
core/schemas/models.go (5)
3-14: LGTM! Request schema cleanly separates public and provider-specific parameters.The use of
json:"-"for ExtraParams is a good pattern - it allows provider-specific customization without exposing internal details in the API contract.
16-25: LGTM! Response schema cleanly abstracts provider-specific pagination.The use of
json:"-"for Anthropic-specific pagination fields allows internal state tracking without exposing provider-specific details in the public API. The standard NextPageToken provides a uniform pagination interface across all providers.
27-44: LGTM! Model schema flexibly accommodates diverse provider capabilities.Making only ID required (non-pointer) while all other fields are optional allows providers to expose varying levels of model metadata without forcing every provider to populate every field.
46-69: LGTM! Supporting structs maintain schema flexibility.All fields being optional (pointers or slices) maintains the flexible design pattern, allowing providers to expose only the metadata they support.
71-80: LGTM! Final structs maintain consistent optional field pattern.Consistent use of optional pointer fields throughout all supporting structures maintains the flexible schema design.
core/providers/vertex.go (1)
489-494: LGTM! Embedding handler signature cleanup.The removal of the unused
paramsparameter simplifies the signature and the call site is updated consistently.core/providers/azure.go (3)
74-167: LGTM! completeRequest properly supports GET and non-deployment endpoints.The refactored signature and implementation correctly:
- Conditionally marshals request body only when provided
- Uses centralized
azure.DefaultAzureAPIVersionconstant (addresses past review)- Routes between deployment and non-deployment URLs based on
useDeploymentflag- Omits body and content-type for GET requests
- Maintains backward compatibility for existing POST/deployment flows
179-179: LGTM! All completeRequest call sites correctly updated.The call sites properly use:
"POST"withuseDeployment=truefor inference operations (text, chat, embeddings)"GET"withuseDeployment=falsefor control plane operations (ListModels)This correctly routes requests between deployment endpoints and resource-level endpoints.
Also applies to: 270-270, 394-394, 443-443
439-472: LGTM! Azure ListModels correctly uses resource-level endpoint.The implementation properly:
- Uses GET to the non-deployment "models" endpoint (resource-level operation)
- Converts Azure response to Bifrost format
- Applies pagination using ApplyPagination utility
- Populates ExtraFields with provider, latency, and request type
- Optionally includes raw response
core/bifrost.go (5)
96-97: LGTM: Providers slice initialization.Correctly initializes an empty slice for runtime provider tracking.
211-274: LGTM: ListModelsRequest implementation.The method correctly:
- Validates the request and provider existence
- Forwards all request parameters (Provider, PageSize, PageToken, ExtraParams)
- Derives the base provider type for custom providers
- Selects an appropriate key using the ListModelsRequest type
- Delegates to the provider's ListModels implementation
Past review comments have been addressed.
276-368: LGTM: ListAllModels aggregation logic.The method correctly:
- Fetches models from all configured providers with a 1000-item page size
- Includes a safety limit (maxIterations) to prevent infinite pagination loops
- Aggregates and sorts models alphabetically by ID
- Tracks total elapsed time and applies client-side pagination
- Handles per-provider errors gracefully while continuing with other providers
Past review comments have been addressed.
1269-1284: LGTM: getProviderByKey implementation.Correctly loads the providers slice atomically and performs a linear search by provider key.
2144-2144: LGTM: Key selection logic for ListModelsRequest.The updated signature includes a
requestTypeparameter, and the key filtering logic correctly branches based on the request type:
- ListModelsRequest: Uses all available keys without model/deployment filtering (since no specific model is being requested)
- Other requests: Applies existing model-specific and deployment-specific filtering
This resolves the issue flagged in past reviews where ListModelsRequest with an empty model string would incorrectly filter out keys with explicit Models lists or Azure/Bedrock deployment mappings.
Also applies to: 2164-2188
core/providers/openai.go (2)
1780-1789: LGTM: ListModels method follows established patterns.The method correctly checks operation permissions and delegates to the internal handler, consistent with other provider operations.
1791-1854: LGTM: OpenAI ListModels handler correctly implements the flow.The handler correctly:
- Sends a GET request to
/v1/modelswith authorization- Handles errors through the standard OpenAI error parser
- Converts the OpenAI response format to Bifrost format
- Applies client-side pagination (appropriate since OpenAI doesn't support server-side pagination for this endpoint)
- Populates extra fields (provider, request type, latency, raw response)
Note:
request.ExtraParamsare not used, which is correct for OpenAI's/v1/modelsendpoint (it doesn't accept query parameters). Other providers might use ExtraParams for provider-specific filtering.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
2 times, most recently
from
21ecf9b to
e3d2ee8
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
tests/core-providers/config/account.go (2)
169-173: Unify Azure API version defaultsTwo Azure key configs use different defaults ("2024-08-01-preview" here vs "2024-10-21" below). Pick one default (prefer stable) to avoid inconsistent behavior.
Apply minimal fix:- // Note: This is a preview API version that may change over time. Update as needed. - // Set AZURE_API_VERSION environment variable to override the default. - APIVersion: bifrost.Ptr(getEnvWithDefault("AZURE_API_VERSION", "2024-08-01-preview")), + // Set AZURE_API_VERSION to override; default to current stable for consistency with other Azure key config. + APIVersion: bifrost.Ptr(getEnvWithDefault("AZURE_API_VERSION", "2024-10-21")),
284-296: Enable ListModels on the custom OpenAI provider’s AllowedRequestsScenarios enable ListModels for ProviderOpenAICustom, but AllowedRequests doesn’t. This can block the route.
Apply:AllowedRequests: &schemas.AllowedRequests{ + ListModels: true, TextCompletion: false, ChatCompletion: true, ChatCompletionStream: true, Embedding: false, Speech: false, SpeechStream: false, Transcription: false, TranscriptionStream: false, },
♻️ Duplicate comments (5)
core/schemas/providers/gemini/models.go (1)
55-80: Missing NextPageToken propagation breaks pagination.The
ToGeminiListModelsResponsefunction converts from Bifrost to Gemini format but doesn't propagate theNextPageTokenfield. This was raised in a previous review but appears to have been incorrectly dismissed. Without propagating this field, pagination cannot work in the reverse direction—clients won't receive the token needed to fetch subsequent pages.Apply this diff to fix the pagination:
func ToGeminiListModelsResponse(resp *schemas.BifrostListModelsResponse) *GeminiListModelsResponse { if resp == nil { return nil } geminiResponse := &GeminiListModelsResponse{ - Models: make([]GeminiModel, 0, len(resp.Data)), + Models: make([]GeminiModel, 0, len(resp.Data)), + NextPageToken: resp.NextPageToken, }transports/bifrost-http/integrations/anthropic.go (1)
105-137: Validate limit > 0 and accept page_token as an alias for after_id.
- Reject non-positive limit; return 400-style error.
- Parse page_token into request.PageToken to support generic pagination; URL builder already maps it to after_id when not present.
@@ - if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { - if limit, err := strconv.Atoi(limitStr); err == nil { - listModelsReq.PageSize = limit - } else { - return fmt.Errorf("invalid limit parameter: %w", err) - } - } + if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { + limit, err := strconv.Atoi(limitStr) + if err != nil { + return fmt.Errorf("invalid limit parameter: %w", err) + } + if limit <= 0 { + return fmt.Errorf("invalid limit parameter: must be > 0") + } + listModelsReq.PageSize = limit + } @@ + // Support generic page_token for forward pagination + if pt := string(ctx.QueryArgs().Peek("page_token")); pt != "" { + listModelsReq.PageToken = pt + }core/providers/gemini.go (3)
371-383: Escape model path segment; trim provider-specific prefixes.Model names may include prefixes or special chars. Normalize (trim "gemini/" and "models/") and url.PathEscape before composing the URL. This was flagged earlier.
@@ - req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+request.Model+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody)) + m := sanitizeGeminiModelID(request.Model) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+url.PathEscape(m)+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody))Add helper and import once in this file:
@@ -import ( +import ( "bufio" "bytes" "context" "errors" "fmt" "io" "net/http" + "net/url" "strconv" "strings" "time" )// Near other helpers in this file func sanitizeGeminiModelID(model string) string { m := strings.TrimPrefix(model, "gemini/") return strings.TrimPrefix(m, "models/") }Also applies to: 376-379
649-657: Apply same model ID normalization/escaping for TranscriptionStream.- req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+request.Model+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody)) + m := sanitizeGeminiModelID(request.Model) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+url.PathEscape(m)+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody))Also applies to: 654-657
912-915: Also escape model path in non-streaming generateContent helper.- req.SetRequestURI(provider.networkConfig.BaseURL + "/models/" + model + endpoint) + m := sanitizeGeminiModelID(model) + req.SetRequestURI(provider.networkConfig.BaseURL + "/models/" + url.PathEscape(m) + endpoint)
🧹 Nitpick comments (7)
core/schemas/provider.go (1)
90-90: Address field ordering per previous review feedback.A previous reviewer (Pratham-Mishra04) requested keeping
list_modelsfirst in all schema definitions. However, theListModelsfield is placed after the struct comment at line 90, while other operation fields follow. Consider moving it to be the first field in the struct for consistency with the reviewer's guidance.Apply this diff to move ListModels to the first position:
type AllowedRequests struct { ListModels bool `json:"list_models"` TextCompletion bool `json:"text_completion"` TextCompletionStream bool `json:"text_completion_stream"`Note: Since this was explicitly requested in a previous review, it's worth confirming whether the ordering requirement applies to this struct.
core/schemas/models.go (1)
142-159: Consider simplifying the offset bound check.The validation at line 149 checks
cursor.Offset <= 0, but line 135 indecodePaginationCursoralready ensurescursor.Offset >= 0by returning a zero cursor for negative offsets. The<= 0check is thus redundant—if offset is 0, it represents the start of the data, which is valid. Consider changing line 149 to only check the upper bound for clarity.Suggested simplification:
- if cursor.Offset <= 0 || cursor.Offset > len(data) { + if cursor.Offset > len(data) { return false }However, the current logic is defensive and returns
trueat line 158 for edge cases, so the existing code is safe—this is purely a clarity improvement.core/providers/groq.go (1)
69-82: Groq ListModels wiring looks goodDelegation to the shared OpenAI-style handler with trimmed BaseURL is correct.
For consistency with other methods, consider using provider.GetProviderKey() instead of schemas.Groq in the call.core/schemas/bifrost.go (1)
82-82: Schemas updated for list_models
- New RequestType and BifrostRequest field look consistent.
- Omitempty on Provider/ModelRequested reduces noise in list_models responses.
Optional: add ListModelsRequest handling in GetRequestFields/SetProvider for unified logging and future hooks.Also applies to: 122-140, 255-258
core/providers/anthropic.go (1)
191-256: Map Anthropic pagination to NextPageToken and drop Content-Type on GET (optional).
- Propagate NextPageToken when HasMore is true by using last_id. This aligns with the standard /v1/models JSON (clients expect next_page_token).
- Content-Type on GET is unnecessary; safe to remove.
Apply:
@@ // Parse Anthropic's response var anthropicResponse anthropic.AnthropicListModelsResponse rawResponse, bifrostErr := handleProviderResponse(resp.Body(), &anthropicResponse, provider.sendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr } @@ // Create final response response := anthropicResponse.ToBifrostListModelsResponse(providerName) + // Propagate next page token using Anthropic pagination semantics + if anthropicResponse.HasMore && anthropicResponse.LastID != nil && *anthropicResponse.LastID != "" { + response.NextPageToken = *anthropicResponse.LastID + } @@ - req.Header.SetContentType("application/json") + // GET: no need to send Content-Typecore/providers/openrouter.go (1)
66-121: Normalize error with provider, use shared response handler, and avoid Content-Type on GET.
- Attach ExtraFields.Provider to errors from parseOpenAIError.
- Replace double sonic.Unmarshal with handleProviderResponse for typed+raw handling and consistency.
- Content-Type header on GET is unnecessary.
@@ if resp.StatusCode() != fasthttp.StatusOK { provider.logger.Debug(fmt.Sprintf("error from %s provider: %s", schemas.OpenRouter, string(resp.Body()))) - return nil, parseOpenAIError(resp) + err := parseOpenAIError(resp) + if err != nil { + err.ExtraFields.Provider = provider.GetProviderKey() + } + return nil, err } - var openrouterResponse schemas.BifrostListModelsResponse - if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { - return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) - } - - var rawResponse interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { - return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) - } + var openrouterResponse schemas.BifrostListModelsResponse + rawResponse, bifrostErr := handleProviderResponse(resp.Body(), &openrouterResponse, provider.sendBackRawResponse) + if bifrostErr != nil { + return nil, bifrostErr + } @@ - req.Header.SetContentType("application/json") + // GET: Content-Type header not neededcore/providers/cohere.go (1)
98-158: Cohere ListModels path — solid.Gating, URL builder, error conversion, and ExtraFields enrichment look correct. Optional: omit Content-Type on GET.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (88)
core/bifrost.go(8 hunks)core/changelog.md(1 hunks)core/providers/anthropic.go(4 hunks)core/providers/azure.go(10 hunks)core/providers/bedrock.go(7 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(6 hunks)core/providers/gemini.go(5 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(2 hunks)core/providers/openai.go(12 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(4 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(1 hunks)core/schemas/providers/gemini/chat.go(1 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/version(1 hunks)docs/apis/openapi.json(5 hunks)docs/features/custom-providers.mdx(2 hunks)docs/features/unified-interface.mdx(1 hunks)framework/changelog.md(1 hunks)framework/version(1 hunks)plugins/governance/changelog.md(1 hunks)plugins/governance/version(1 hunks)plugins/jsonparser/changelog.md(1 hunks)plugins/jsonparser/version(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/version(1 hunks)plugins/maxim/changelog.md(1 hunks)plugins/maxim/version(1 hunks)plugins/mocker/changelog.md(1 hunks)plugins/mocker/version(1 hunks)plugins/otel/changelog.md(1 hunks)plugins/otel/version(1 hunks)plugins/semanticcache/changelog.md(1 hunks)plugins/semanticcache/version(1 hunks)plugins/telemetry/changelog.md(1 hunks)plugins/telemetry/version(1 hunks)tests/core-providers/anthropic_test.go(1 hunks)tests/core-providers/azure_test.go(1 hunks)tests/core-providers/bedrock_test.go(1 hunks)tests/core-providers/cohere_test.go(1 hunks)tests/core-providers/config/account.go(14 hunks)tests/core-providers/gemini_test.go(1 hunks)tests/core-providers/groq_test.go(1 hunks)tests/core-providers/mistral_test.go(1 hunks)tests/core-providers/ollama_test.go(1 hunks)tests/core-providers/openai_test.go(1 hunks)tests/core-providers/openrouter_test.go(1 hunks)tests/core-providers/parasail_test.go(1 hunks)tests/core-providers/scenarios/list_models.go(1 hunks)tests/core-providers/sgl_test.go(1 hunks)tests/core-providers/tests.go(2 hunks)tests/core-providers/vertex_test.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(8 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)transports/changelog.md(1 hunks)transports/version(1 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (16)
- plugins/mocker/version
- plugins/maxim/changelog.md
- plugins/mocker/changelog.md
- plugins/maxim/version
- plugins/jsonparser/version
- plugins/governance/version
- plugins/telemetry/version
- plugins/telemetry/changelog.md
- plugins/semanticcache/changelog.md
- framework/changelog.md
- transports/version
- framework/version
- plugins/logging/version
- plugins/governance/changelog.md
- plugins/otel/changelog.md
- plugins/semanticcache/version
🚧 Files skipped from review as they are similar to previous changes (22)
- tests/core-providers/mistral_test.go
- tests/core-providers/azure_test.go
- ui/lib/types/schemas.ts
- tests/core-providers/gemini_test.go
- ui/lib/constants/config.ts
- tests/core-providers/openai_test.go
- core/schemas/providers/openai/models.go
- core/schemas/account.go
- core/schemas/providers/cohere/types.go
- docs/features/custom-providers.mdx
- tests/core-providers/ollama_test.go
- tests/core-providers/parasail_test.go
- core/schemas/providers/anthropic/models.go
- docs/features/unified-interface.mdx
- transports/bifrost-http/integrations/genai.go
- core/providers/mistral.go
- tests/core-providers/openrouter_test.go
- transports/bifrost-http/integrations/utils.go
- ui/lib/types/config.ts
- tests/core-providers/tests.go
- core/schemas/providers/mistral/models.go
- ui/app/providers/fragments/allowedRequestsFields.tsx
🧰 Additional context used
🧬 Code graph analysis (29)
core/schemas/providers/gemini/models.go (3)
core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/gemini/types.go (3)
DefaultPageSize(1254-1254)GeminiListModelsResponse(1273-1276)GeminiModel(1256-1270)core/schemas/bifrost.go (1)
ModelProvider(32-32)
core/providers/groq.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (2)
BifrostError(322-331)Groq(43-43)
core/schemas/providers/azure/models.go (3)
core/schemas/providers/azure/types.go (1)
AzureListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (1)
Azure(36-36)
core/providers/anthropic.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (3)
BifrostError(322-331)Anthropic(37-37)ListModelsRequest(82-82)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsURL(11-43)core/schemas/providers/anthropic/types.go (2)
AnthropicError(305-311)AnthropicListModelsResponse(283-288)core/schemas/provider.go (1)
Provider(197-224)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenAI(35-35)ListModelsRequest(82-82)ModelProvider(32-32)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(140-143)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/config/account.go (1)
core/utils.go (1)
Ptr(12-14)
core/schemas/bifrost.go (2)
core/schemas/models.go (1)
BifrostListModelsRequest(10-21)core/schemas/provider.go (1)
Provider(197-224)
core/providers/gemini.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (3)
BifrostError(322-331)Gemini(47-47)ListModelsRequest(82-82)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsURL(11-27)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1273-1276)core/schemas/provider.go (1)
Provider(197-224)
core/providers/sgl.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/providers/azure.go (5)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/utils.go (1)
Ptr(12-14)core/schemas/providers/azure/types.go (1)
DefaultAzureAPIVersion(4-4)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)
core/schemas/providers/cohere/models.go (3)
core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/cohere/types.go (2)
DefaultPageSize(529-529)CohereListModelsResponse(542-545)core/schemas/bifrost.go (1)
ModelProvider(32-32)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(23-32)BifrostListModelsRequest(10-21)core/schemas/bifrost.go (3)
BifrostError(322-331)ModelProvider(32-32)ListModelsRequest(82-82)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/mistral/types.go (1)
core/schemas/providers/gemini/types.go (1)
Type(769-769)
core/schemas/provider.go (3)
core/schemas/bifrost.go (2)
ListModelsRequest(82-82)BifrostError(322-331)core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)
core/providers/bedrock.go (6)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (4)
BifrostError(322-331)ErrorField(340-347)Bedrock(38-38)ListModelsRequest(82-82)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(453-455)core/schemas/providers/bedrock/embedding.go (1)
ToBedrockCohereEmbeddingRequest(71-83)
core/schemas/providers/bedrock/models.go (3)
core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(453-455)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (3)
BifrostListModelsResponse(23-32)Model(34-51)Architecture(53-59)
core/schemas/providers/vertex/models.go (3)
core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/vertex/types.go (2)
DefaultPageSize(50-50)VertexListModelsResponse(61-64)core/schemas/bifrost.go (1)
Vertex(40-40)
core/providers/vertex.go (7)
core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (5)
BifrostError(322-331)ErrorField(340-347)RequestCancelled(282-282)ListModelsRequest(82-82)Vertex(40-40)core/schemas/providers/vertex/models.go (1)
ToVertexListModelsURL(10-26)core/schemas/provider.go (5)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)ErrProviderJSONMarshaling(26-26)core/schemas/providers/vertex/types.go (2)
VertexListModelsResponse(61-64)VertexEmbeddingRequest(21-24)core/schemas/embedding.go (1)
BifrostEmbeddingResponse(17-23)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(74-103)core/schemas/providers/anthropic/chat.go (1)
ToAnthropicChatCompletionError(986-1007)
tests/core-providers/scenarios/list_models.go (5)
tests/core-providers/config/account.go (1)
ComprehensiveTestConfig(44-60)core/schemas/provider.go (1)
Provider(197-224)core/schemas/models.go (1)
BifrostListModelsRequest(10-21)core/schemas/bifrost.go (2)
ListModelsRequest(82-82)RequestType(79-79)tests/core-providers/scenarios/utils.go (1)
GetErrorMessage(550-574)
core/providers/parasail.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(254-262)
core/providers/ollama.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(34-51)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
transports/bifrost-http/integrations/openai.go (6)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (5)
Azure(36-36)BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)OpenAI(35-35)transports/bifrost-http/integrations/utils.go (6)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/providers/cohere.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (3)
BifrostError(322-331)Cohere(39-39)ListModelsRequest(82-82)core/schemas/providers/cohere/models.go (1)
ToCohereListModelsURL(10-34)core/schemas/providers/cohere/types.go (2)
CohereError(521-525)CohereListModelsResponse(542-545)core/schemas/provider.go (1)
Provider(197-224)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (8)
KeySelector(16-16)ListModelsRequest(82-82)BifrostError(322-331)BifrostResponseExtraFields(254-262)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(82-82)core/schemas/provider.go (2)
ErrProviderResponseUnmarshal(25-25)Provider(197-224)
🪛 LanguageTool
transports/changelog.md
[grammar] ~5-~5: Ensure spelling is correct
Context: .../v1/models endpoint to list models of conigured providers
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🔇 Additional comments (60)
core/schemas/providers/mistral/types.go (1)
1-34: LGTM! Well-structured type definitions.The types correctly model the Mistral API response structure with appropriate field types and JSON tags. The previous issue with the
Createdfield type has been addressed (nowint64).One minor note for consistency:
MaxContextLengthusesintwhileCreatedusesint64. While context lengths are unlikely to exceed int32 limits, usingint64would provide platform-independence and consistency. However, this is a very minor style point and the current implementation is functionally correct.plugins/jsonparser/changelog.md (1)
4-4: LGTM! Routine version bump.The changelog entry correctly reflects the version updates for core (1.2.14) and framework (1.1.16) to align with the ListModels feature additions.
core/schemas/providers/gemini/chat.go (1)
442-444: LGTM! Bug fix correctly applied.The previous issue where successfully parsed arguments were being discarded has been properly resolved. The code now only resets
argsMapto an empty map whenjson.Unmarshalfails, preserving the parsed arguments on success.ui/app/providers/fragments/apiStructureFormFragment.tsx (1)
48-48: LGTM! Consistent with existing pattern.The
list_modelsfield is added to the form's default values following the same pattern as otherallowed_requestsfields, with a sensible default oftrue.plugins/otel/version (1)
1-1: LGTM! Routine version bump.The version increment from 1.0.15 to 1.0.16 aligns with the PR's feature additions.
tests/core-providers/groq_test.go (1)
50-50: LGTM! Test configuration updated.The
ListModels: trueflag appropriately enables model listing tests for the Groq provider, consistent with the PR's objective to add ListModels support across providers.tests/core-providers/sgl_test.go (1)
43-43: LGTM! Correctly disables unsupported feature.The
ListModels: falseflag appropriately disables model listing tests for SGL, consistent with the provider's lack of support for this operation (as confirmed in core/providers/sgl.go).tests/core-providers/cohere_test.go (1)
44-44: LGTM! Test configuration updated.The
ListModels: trueflag enables model listing tests for the Cohere provider, aligning with the PR's ListModels feature rollout.core/providers/sgl.go (1)
71-74: LGTM! Appropriate stub for unsupported operation.The
ListModelsmethod correctly returns an unsupported operation error for the SGL provider, consistent with the test configuration (ListModels: falsein tests/core-providers/sgl_test.go) and following the same pattern as other unsupported operations in this provider.core/schemas/providers/openai/types.go (1)
128-143: LGTM!The model types are well-structured and follow OpenAI API conventions. The GROQ-specific fields are properly marked as optional.
core/providers/cerebras.go (1)
64-67: LGTM!The ListModels implementation correctly delegates to the shared OpenAI-compatible handler with the appropriate Cerebras endpoint.
core/schemas/provider.go (2)
109-110: LGTM!The
ListModelsRequestcase is correctly integrated into the operation allowance logic.
200-201: LGTM!The
ListModelsmethod signature is well-defined and, per previous review discussions, all provider implementations have been confirmed to include this method.core/schemas/providers/cohere/models.go (2)
10-34: LGTM!The URL construction logic correctly handles pagination parameters with appropriate defaults and optional ExtraParams.
46-53: LGTM!The previous pointer capture bug has been correctly fixed using
schemas.Ptr()to create distinct pointers for each iteration. This ensures each model gets its own Name and ContextLength pointers rather than sharing references to the last loop variable.core/schemas/providers/gemini/models.go (2)
11-27: LGTM!The URL construction correctly handles pagination parameters with proper defaults.
29-53: LGTM!The pointer capture issue from previous reviews has been properly addressed using
schemas.Ptr(). The conversion logic correctly maps Gemini model fields to the Bifrost format.core/schemas/models.go (4)
10-51: LGTM!The request and response types are well-designed with clear separation of concerns. The Model struct provides comprehensive metadata fields with appropriate optionality using pointers.
53-93: LGTM!The supporting types and internal pagination cursor structure are appropriately defined. The cursor encodes both offset and LastID for validation.
95-140: LGTM!The cursor encoding and decoding logic is robust:
- Uses URL-safe base64 encoding for opaque tokens
- Handles errors gracefully by returning empty/zero values
- Validates offset bounds during decode
164-221: LGTM!The pagination implementation is well-designed:
- Validates cursor integrity using LastID to detect data drift
- Resets to beginning on invalid cursors (defensive behavior)
- Correctly handles edge cases (empty data, last page, out of bounds)
- Properly encodes NextPageToken with LastID for validation
core/schemas/providers/gemini/types.go (1)
1253-1276: LGTM!The model types are well-defined with comprehensive fields. Previous review feedback regarding naming conventions (
BaseModelID) and godoc comments has been properly addressed.plugins/logging/changelog.md (1)
4-4: LGTM!Standard changelog entry for the version update.
core/version (1)
1-1: LGTM! Version bump appropriate for feature addition.The version increment from 1.2.13 to 1.2.14 is consistent with the addition of the ListModels feature.
core/changelog.md (1)
4-5: LGTM! Changelog accurately documents the feature.The changelog entries clearly describe the ListModels additions to the Provider interface and core tracking capabilities.
tests/core-providers/bedrock_test.go (1)
48-48: LGTM! Test configuration updated for ListModels.Enabling the ListModels scenario for Bedrock tests is consistent with the PR's objective to add model listing across all providers.
tests/core-providers/anthropic_test.go (1)
46-46: LGTM! Test configuration updated for ListModels.Enabling the ListModels scenario for Anthropic tests aligns with the PR's goal to support model listing across providers.
ui/app/providers/dialogs/addNewCustomProviderDialog.tsx (1)
24-24: LGTM! UI schema updated for ListModels support.The
list_modelsfield is properly added to theallowedRequestsSchemaand initialized with a sensible default oftrue, consistent with other allowed request types.Also applies to: 59-59
tests/core-providers/vertex_test.go (1)
43-43: LGTM! Test configuration updated for ListModels.Enabling the ListModels scenario for Vertex tests is consistent with the comprehensive ListModels rollout across all providers.
core/schemas/providers/azure/models.go (1)
5-22: LGTM! Clean conversion implementation.The
ToBifrostListModelsResponsemethod is well-structured:
- Proper nil handling prevents panics
- Preallocates the Data slice for performance
- Converts Azure-specific model format to Bifrost's standardized format
- Uses appropriate helpers (
schemas.Ptr) for pointer conversions- ID namespacing with
azure/prefix ensures model IDs are globally unique across providersThe implementation follows the same pattern used in other provider conversions (e.g., Bedrock).
core/schemas/providers/vertex/types.go (1)
52-59: LGTM: Vertex model types and pagination tagModel fields and time.Time look fine, and NextPageToken uses the correct camelCase tag. No issues spotted.
Also applies to: 61-64
core/providers/ollama.go (1)
71-74: Ollama ListModels route hooked correctlyCall shape matches other handlers. Verify the shared handler skips Authorization when key.Value is empty (common for local Ollama), otherwise requests may 401.
core/schemas/providers/anthropic/types.go (1)
274-289: Anthropic list-models schema: nullable IDs handledSwitching FirstID/LastID to *string and adding DefaultPageSize is correct. time.Time for created_at is appropriate.
tests/core-providers/config/account.go (2)
40-41: TestScenarios: ListModels flag additionField addition is fine and aligns with new tests.
503-504: ListModels scenarios enabled across providersEnabling these looks good given the generic /v1/models router fans out per provider. No additional per-provider HTTP routes required.
Also applies to: 531-532, 559-560, 587-588, 615-616, 643-644, 670-671, 698-699, 726-727, 754-755, 785-786, 813-814
core/schemas/providers/bedrock/models.go (1)
5-27: Bedrock → Bifrost mapper: safe pointers and preallocationPre-sizing Data and using schemas.Ptr avoids range-variable pointer bugs. Mapping ID as provider/model is consistent across providers.
transports/bifrost-http/handlers/inference.go (1)
277-279: Route addedGET /v1/models registration via middleware chain looks correct.
core/providers/anthropic.go (3)
158-162: Use of http.MethodPost constant — good.Consistent with style elsewhere and avoids string literals.
415-416: Switched to http.MethodPost in streaming request — good.
663-664: Switched to http.MethodPost for ResponsesStream — good.core/providers/parasail.go (1)
69-72: Explicit unsupported ListModels — OK.Matches provider behavior; consistent with other Parasail ops.
transports/bifrost-http/integrations/anthropic.go (1)
77-103: New route wiring for GET /v1/models — looks good.Converter/error handling align with other Anthropic routes.
core/providers/gemini.go (2)
74-127: Gemini ListModels implementation — LGTM.Uses URL builder, proper auth, and shared response handler.
171-172: Use of http.MethodPost — good.core/schemas/providers/vertex/models.go (2)
10-26: URL builder is correct and encoded; default aligns with Vertex.
28-48: Conversion to Bifrost models — LGTM.ID prefixing, name/desc/created mapping are correct.
core/providers/cohere.go (4)
235-237: Use of http.MethodPost — good.
311-312: Use of http.MethodPost for streaming — good.
593-594: Use of http.MethodPost for ResponsesStream — good.
787-789: Use of http.MethodPost for embeddings — good.core/bifrost.go (6)
95-97: LGTM: Providers slice initialization.Correct initialization of the atomic providers pointer with an empty slice. Providers will be populated during initialization in
prepareProvider.
211-274: LGTM: Well-structured ListModelsRequest implementation.The method correctly:
- Validates request and provider inputs
- Copies all request fields (Provider, PageSize, PageToken, ExtraParams)
- Determines the base provider type for custom providers (consistent with
requestWorker)- Selects an appropriate API key when required, passing
schemas.ListModelsRequestas the request type- Delegates to the provider's
ListModelsimplementation with proper error handling
276-374: LGTM: Robust ListAllModels implementation with proper safeguards.The method correctly aggregates models across all configured providers with:
- Nil request handling (creates default request)
- Pagination safety limit (
maxIterations = 20) to prevent infinite loops- Resilient error handling (logs failures, continues with other providers, returns first error only if no models retrieved)
- Nil response checks before accessing fields
- Consistent sorting by model ID
- Proper pagination: fetches up to 1000 models per provider request, aggregates all results, then applies caller's pagination to the final sorted dataset
- Accurate latency tracking across all provider requests
1213-1226: LGTM: Atomic provider append correctly implemented.The compare-and-swap loop ensures that concurrent calls to
prepareProvidernever lose updates. This addresses the previous review concern about potential lost updates when appending to the providers slice.
1280-1295: LGTM: Simple and correct provider lookup.The helper method safely retrieves a provider instance by key using an atomic load and linear search. For typical deployments with a small number of providers, this approach is efficient and straightforward.
2159-2227: LGTM: Request-type-aware key selection correctly implemented.The updated method properly branches on request type:
For ListModelsRequest (lines 2179-2185): Skips model-specific and deployment-specific filtering, only validating that keys have non-empty values (or the provider explicitly allows empty values). This is correct since listing models requires any valid key for authentication, not a model-specific key.
For other request types (lines 2186-2207): Applies full model and deployment validation as before, ensuring the selected key supports the requested model and has the necessary deployment mappings for Azure/Bedrock.
This resolves the previous review concern about incorrectly filtering out valid keys for ListModels requests while preserving the existing validation logic for model-specific requests.
docs/apis/openapi.json (2)
291-349: Documentation for /v1/models reads cleanThe endpoint description, tag wiring, and pagination params match the behavior in core; thanks for capturing the provider optionality right away.
6560-6807: Model metadata schema looks comprehensiveNice job mirroring the Bifrost model struct—pricing, architecture, limits, and the new request_type enum all line up for clients.
core/providers/azure.go (2)
64-142: Generic completeRequest helper is spot onBeing able to send GETs by skipping marshaling while still reusing auth/header handling will keep things tidy across future Azure endpoints.
168-201: ListModels implementation aligns with Azure APITargeting
/openai/modelswith the shared converter and local pagination gives us parity with other providers without new config knobs.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
aef38d3 to
6675c82
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/providers/bedrock.go (1)
186-196: Fix error message wording.This path reads the HTTP response body; message should say “response”, not “request”.
- "message": "error reading request", + "message": "error reading response",
♻️ Duplicate comments (2)
transports/bifrost-http/handlers/inference.go (1)
319-329: Compile-time bug: fasthttp QueryArgs has no All(); use VisitAllReplace iteration with VisitAll; current code won’t compile.
Apply:
- // Pass-through unknown query params for provider-specific features - extraParams := map[string]interface{}{} - for k, v := range ctx.QueryArgs().All() { - s := string(k) - if s != "provider" && s != "page_size" && s != "page_token" { - extraParams[s] = string(v) - } - } + // Pass-through unknown query params for provider-specific features + extraParams := make(map[string]interface{}) + ctx.QueryArgs().VisitAll(func(k, v []byte) { + s := string(k) + if s != "provider" && s != "page_size" && s != "page_token" { + extraParams[s] = string(v) + } + })core/schemas/providers/gemini/models.go (1)
55-80: Preserve NextPageToken in reverse mapping.Set NextPageToken on GeminiListModelsResponse to keep pagination round-trippable.
func ToGeminiListModelsResponse(resp *schemas.BifrostListModelsResponse) *GeminiListModelsResponse { if resp == nil { return nil } geminiResponse := &GeminiListModelsResponse{ - Models: make([]GeminiModel, 0, len(resp.Data)), + Models: make([]GeminiModel, 0, len(resp.Data)), + NextPageToken: resp.NextPageToken, }
🧹 Nitpick comments (14)
core/schemas/providers/bedrock/types.go (1)
435-450: Add documentation comments for the remaining types.Good progress on adding the doc comment for
BedrockListModelsResponse. However,BedrockModelLifecycleandBedrockModelstill lack documentation comments. All other exported types in this file follow the Go convention of having comprehensive doc comments.Apply this diff to add the missing doc comments:
// ==================== MODELS TYPES ==================== + +// BedrockModelLifecycle represents the lifecycle status of a Bedrock model type BedrockModelLifecycle struct { Status string `json:"status"` } +// BedrockModel represents a foundation model available in AWS Bedrock. +// It includes metadata about the model's capabilities, modalities, and configuration. type BedrockModel struct { CustomizationsSupported []string `json:"customizationsSupported,omitempty"` InferenceTypesSupported []string `json:"inferenceTypesSupported,omitempty"` InputModalities []string `json:"inputModalities,omitempty"` ModelArn string `json:"modelArn"` ModelID string `json:"modelId"` ModelLifecycle BedrockModelLifecycle `json:"modelLifecycle,omitempty"` ModelName string `json:"modelName"` OutputModalities []string `json:"outputModalities,omitempty"` ProviderName string `json:"providerName"` ResponseStreamingSupported bool `json:"responseStreamingSupported"` }core/providers/openrouter.go (1)
66-121: ListModels implementation is correct; consider optimizing double unmarshal.The ListModels implementation correctly:
- Constructs and executes the GET request to
/v1/modelswith proper authentication- Handles errors and logs debug information
- Prefixes model IDs with the provider identifier for uniqueness
- Applies pagination using the request parameters
- Populates ExtraFields with provider metadata and latency
Minor optimization opportunity: Lines 94-102 unmarshal
resp.Body()twice—once into the typed response and once into a raw interface. Consider unmarshaling once and reusing the result ifsendBackRawResponseis enabled.Optional refactor to reduce duplicate unmarshaling:
- var openrouterResponse schemas.BifrostListModelsResponse - if err := sonic.Unmarshal(resp.Body(), &openrouterResponse); err != nil { - return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) - } - var rawResponse interface{} if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) } + + // Re-marshal and unmarshal into typed response + responseBytes, err := sonic.Marshal(rawResponse) + if err != nil { + return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) + } + var openrouterResponse schemas.BifrostListModelsResponse + if err := sonic.Unmarshal(responseBytes, &openrouterResponse); err != nil { + return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) + }Note: This optimization has trade-offs (additional marshal step), so the current implementation is acceptable as-is.
core/providers/cerebras.go (1)
64-67: LGTM - ListModels implementation is correct.The
ListModelsmethod properly delegates tohandleOpenAIListModelsRequestwith all necessary parameters and follows the established pattern for OpenAI-compatible providers.For improved readability, consider formatting line 66 with arguments on separate lines:
func (provider *CerebrasProvider) ListModels(ctx context.Context, key schemas.Key, request *schemas.BifrostListModelsRequest) (*schemas.BifrostListModelsResponse, *schemas.BifrostError) { - return handleOpenAIListModelsRequest(ctx, provider.client, request, provider.networkConfig.BaseURL+"/v1/models", key, provider.networkConfig.ExtraHeaders, provider.GetProviderKey(), provider.sendBackRawResponse, provider.logger) + return handleOpenAIListModelsRequest( + ctx, + provider.client, + request, + provider.networkConfig.BaseURL+"/v1/models", + key, + provider.networkConfig.ExtraHeaders, + provider.GetProviderKey(), + provider.sendBackRawResponse, + provider.logger, + ) }core/schemas/providers/gemini/models.go (2)
11-27: Pass through provider-specific query params (ExtraParams).Include request.ExtraParams in the query string (at least string values) so callers can opt into provider flags without changing the signature.
func ToGeminiListModelsURL(request *schemas.BifrostListModelsRequest, baseURL string) string { @@ // Build query parameters params := url.Values{} params.Set("pageSize", strconv.Itoa(pageSize)) if request.PageToken != "" { params.Set("pageToken", request.PageToken) } + + // Optional provider-specific params (string-only for safety) + if request.ExtraParams != nil { + for k, v := range request.ExtraParams { + if s, ok := v.(string); ok && s != "" { + params.Set(k, s) + } + } + } return baseURL + "?" + params.Encode() }
39-49: Surface per-request token limits.Expose Gemini’s InputTokenLimit/OutputTokenLimit via Model.PerRequestLimits for richer client UX.
for _, model := range response.Models { - contextLength := model.InputTokenLimit + model.OutputTokenLimit + contextLength := model.InputTokenLimit + model.OutputTokenLimit + inLimit := model.InputTokenLimit + outLimit := model.OutputTokenLimit // Remove prefix models/ from model.Name modelName := strings.TrimPrefix(model.Name, "models/") bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ ID: string(providerKey) + "/" + modelName, Name: schemas.Ptr(model.DisplayName), Description: schemas.Ptr(model.Description), ContextLength: schemas.Ptr(int(contextLength)), + PerRequestLimits: &schemas.PerRequestLimits{ + PromptTokens: schemas.Ptr(inLimit), + CompletionTokens: schemas.Ptr(outLimit), + }, SupportedMethods: model.SupportedGenerationMethods, }) }docs/apis/openapi.json (3)
331-349: Add 401 response to /v1/models.Endpoint is auth-protected elsewhere; document Unauthorized for client codegen and integrations.
"responses": { "200": { "description": "Successful response with list of models", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ListModelsResponse" } } } }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, "400": { "$ref": "#/components/responses/BadRequest" }, "500": { "$ref": "#/components/responses/InternalServerError" } }
3983-3986: Make latency an integer in milliseconds (schema consistency).Code emits int64 ms; declare latency as integer (format: int64) to avoid float codegen and round-off issues.
"latency": { - "type": "number", + "type": "integer", + "format": "int64", "description": "Request latency in milliseconds", "example": 1234 },Apply in:
- components.schemas.BifrostResponseExtraFields.latency (Lines 3983-3986)
- components.schemas.ListModelsResponse.properties.extra_fields.properties.latency (Lines 6588-6591)
Also applies to: 6588-6591 --- `6810-6811`: **Define ModelDefaultParameters properties.** Expose known defaults to help UIs set sensible sliders. ```diff - "ModelDefaultParameters": { - "type": "object" - } + "ModelDefaultParameters": { + "type": "object", + "properties": { + "temperature": { "type": "number", "minimum": 0.0, "maximum": 2.0, "example": 0.7 }, + "top_p": { "type": "number", "minimum": 0.0, "maximum": 1.0, "example": 0.9 }, + "frequency_penalty": { "type": "number", "minimum": -2.0, "maximum": 2.0, "example": 0.0 } + } + }core/providers/bedrock.go (1)
120-127: Avoid shadowing the url package.Local variable named url shadows import net/url; use a clearer name.
- var url string + var endpointURL string if useRuntimeEndpoint { - url = fmt.Sprintf("https://bedrock-runtime.%s.amazonaws.com/model/%s", region, path) + endpointURL = fmt.Sprintf("https://bedrock-runtime.%s.amazonaws.com/model/%s", region, path) } else { - url = fmt.Sprintf("https://bedrock.%s.amazonaws.com%s", region, path) + endpointURL = fmt.Sprintf("https://bedrock.%s.amazonaws.com%s", region, path) } @@ - if requestBody != nil { - req, err = http.NewRequestWithContext(ctx, method, url, bytes.NewBuffer(jsonBody)) + if requestBody != nil { + req, err = http.NewRequestWithContext(ctx, method, endpointURL, bytes.NewBuffer(jsonBody)) } else { - req, err = http.NewRequestWithContext(ctx, method, url, nil) + req, err = http.NewRequestWithContext(ctx, method, endpointURL, nil) }core/providers/cohere.go (2)
118-120: Minor: avoid setting Content-Type on GET.Content-Type is unnecessary for GET; consider removing to keep headers minimal.
811-835: Skip raw-response unmarshal when not requested.You always unmarshal resp.Body() into rawResponse even if sendBackRawResponse is false. Wrap it to save CPU on large responses.
Apply this diff:
- // Parse raw response for consistent format - var rawResponse interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { - return nil, newBifrostOperationError("error parsing raw response for embedding", err, providerName) - } + var rawResponse interface{} + if provider.sendBackRawResponse { + if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { + return nil, newBifrostOperationError("error parsing raw response for embedding", err, providerName) + } + }core/providers/vertex.go (1)
184-185: Minor: Content-Type on GET is unnecessary.Consider omitting Content-Type for GET requests.
core/bifrost.go (2)
211-274: Optional: lazily prepare provider if not yet initialized.If config changes at runtime and a provider wasn’t prepped, getProviderByKey returns nil. You could fall back to preparing it on demand.
Apply this diff:
- provider := bifrost.getProviderByKey(req.Provider) - if provider == nil { - return nil, &schemas.BifrostError{ - IsBifrostError: false, - Error: &schemas.ErrorField{ - Message: "provider not found for list models request", - }, - } - } + provider := bifrost.getProviderByKey(req.Provider) + if provider == nil { + // Try to initialize provider lazily + if _, err := bifrost.getProviderQueue(req.Provider); err == nil { + provider = bifrost.getProviderByKey(req.Provider) + } + if provider == nil { + return nil, &schemas.BifrostError{ + IsBifrostError: false, + Error: &schemas.ErrorField{ + Message: "provider not found for list models request", + }, + } + } + }
276-374: Speed up ListAllModels by parallelizing provider fetches.Current loop is sequential; running providers concurrently can reduce wall time significantly.
Here’s a minimal pattern (conceptual):
- for _, providerKey := range providerKeys { - ... - for { - response, bifrostErr := bifrost.ListModelsRequest(ctx, providerRequest) - ... - } - } + var wg sync.WaitGroup + mu := &sync.Mutex{} + for _, providerKey := range providerKeys { + pk := providerKey + wg.Add(1) + go func() { + defer wg.Done() + providerRequest := &schemas.BifrostListModelsRequest{Provider: pk, PageSize: 1000} + iterations := 0 + for { + iterations++ + if iterations > maxIterations { bifrost.logger.Warn(...); break } + response, bifrostErr := bifrost.ListModelsRequest(ctx, providerRequest) + if bifrostErr != nil || response == nil { break } + if len(response.Data) > 0 { + mu.Lock(); allModels = append(allModels, response.Data...); mu.Unlock() + } + if response.NextPageToken == "" { break } + providerRequest.PageToken = response.NextPageToken + } + }() + } + wg.Wait()
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (88)
core/bifrost.go(8 hunks)core/changelog.md(1 hunks)core/providers/anthropic.go(4 hunks)core/providers/azure.go(10 hunks)core/providers/bedrock.go(7 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(6 hunks)core/providers/gemini.go(5 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(2 hunks)core/providers/openai.go(12 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(4 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(1 hunks)core/schemas/providers/gemini/chat.go(1 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/version(1 hunks)docs/apis/openapi.json(5 hunks)docs/features/custom-providers.mdx(2 hunks)docs/features/unified-interface.mdx(1 hunks)framework/changelog.md(1 hunks)framework/version(1 hunks)plugins/governance/changelog.md(1 hunks)plugins/governance/version(1 hunks)plugins/jsonparser/changelog.md(1 hunks)plugins/jsonparser/version(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/version(1 hunks)plugins/maxim/changelog.md(1 hunks)plugins/maxim/version(1 hunks)plugins/mocker/changelog.md(1 hunks)plugins/mocker/version(1 hunks)plugins/otel/changelog.md(1 hunks)plugins/otel/version(1 hunks)plugins/semanticcache/changelog.md(1 hunks)plugins/semanticcache/version(1 hunks)plugins/telemetry/changelog.md(1 hunks)plugins/telemetry/version(1 hunks)tests/core-providers/anthropic_test.go(1 hunks)tests/core-providers/azure_test.go(1 hunks)tests/core-providers/bedrock_test.go(1 hunks)tests/core-providers/cohere_test.go(1 hunks)tests/core-providers/config/account.go(14 hunks)tests/core-providers/gemini_test.go(1 hunks)tests/core-providers/groq_test.go(1 hunks)tests/core-providers/mistral_test.go(1 hunks)tests/core-providers/ollama_test.go(1 hunks)tests/core-providers/openai_test.go(1 hunks)tests/core-providers/openrouter_test.go(1 hunks)tests/core-providers/parasail_test.go(1 hunks)tests/core-providers/scenarios/list_models.go(1 hunks)tests/core-providers/sgl_test.go(1 hunks)tests/core-providers/tests.go(2 hunks)tests/core-providers/vertex_test.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(8 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)transports/changelog.md(1 hunks)transports/version(1 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (4)
- plugins/otel/changelog.md
- transports/version
- plugins/maxim/changelog.md
- transports/changelog.md
🚧 Files skipped from review as they are similar to previous changes (42)
- ui/lib/constants/config.ts
- core/schemas/providers/cohere/models.go
- plugins/otel/version
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- plugins/maxim/version
- tests/core-providers/bedrock_test.go
- plugins/mocker/version
- tests/core-providers/azure_test.go
- core/schemas/providers/azure/models.go
- core/schemas/providers/mistral/models.go
- framework/changelog.md
- docs/features/custom-providers.mdx
- core/providers/mistral.go
- tests/core-providers/anthropic_test.go
- framework/version
- core/schemas/provider.go
- core/schemas/providers/vertex/models.go
- core/schemas/providers/azure/types.go
- plugins/governance/version
- core/schemas/providers/bedrock/models.go
- core/providers/ollama.go
- core/schemas/providers/anthropic/models.go
- plugins/semanticcache/version
- ui/app/providers/fragments/allowedRequestsFields.tsx
- tests/core-providers/mistral_test.go
- tests/core-providers/vertex_test.go
- core/schemas/providers/mistral/types.go
- core/schemas/bifrost.go
- tests/core-providers/ollama_test.go
- core/schemas/providers/anthropic/types.go
- ui/lib/types/schemas.ts
- tests/core-providers/parasail_test.go
- plugins/semanticcache/changelog.md
- tests/core-providers/tests.go
- core/version
- plugins/logging/changelog.md
- tests/core-providers/openrouter_test.go
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- tests/core-providers/gemini_test.go
- tests/core-providers/groq_test.go
- core/schemas/providers/openai/models.go
- core/schemas/account.go
🧰 Additional context used
🧬 Code graph analysis (22)
tests/core-providers/config/account.go (1)
core/utils.go (1)
Ptr(12-14)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(82-82)core/schemas/provider.go (2)
ErrProviderResponseUnmarshal(25-25)Provider(197-224)
core/providers/bedrock.go (5)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (4)
BifrostError(322-331)ErrorField(340-347)Bedrock(38-38)ListModelsRequest(82-82)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(453-455)
transports/bifrost-http/integrations/utils.go (1)
core/schemas/models.go (1)
BifrostListModelsResponse(23-32)
tests/core-providers/scenarios/list_models.go (5)
tests/core-providers/config/account.go (1)
ComprehensiveTestConfig(44-60)core/schemas/provider.go (1)
Provider(197-224)core/schemas/models.go (1)
BifrostListModelsRequest(10-21)core/schemas/bifrost.go (2)
ListModelsRequest(82-82)RequestType(79-79)tests/core-providers/scenarios/utils.go (1)
GetErrorMessage(550-574)
core/providers/sgl.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/providers/gemini.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (3)
BifrostError(322-331)Gemini(47-47)ListModelsRequest(82-82)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsURL(11-27)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1273-1276)core/schemas/provider.go (1)
Provider(197-224)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/providers/cohere.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (3)
BifrostError(322-331)Cohere(39-39)ListModelsRequest(82-82)core/schemas/providers/cohere/models.go (1)
ToCohereListModelsURL(10-34)core/schemas/providers/cohere/types.go (2)
CohereError(521-525)CohereListModelsResponse(542-545)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(254-262)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (9)
KeySelector(16-16)ListModelsRequest(82-82)BifrostError(322-331)ErrorField(340-347)BifrostResponseExtraFields(254-262)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)
core/providers/anthropic.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (3)
BifrostError(322-331)Anthropic(37-37)ListModelsRequest(82-82)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsURL(11-43)core/schemas/providers/anthropic/types.go (2)
AnthropicError(305-311)AnthropicListModelsResponse(283-288)core/schemas/provider.go (1)
Provider(197-224)
core/providers/openai.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(140-143)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/gemini/models.go (3)
core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/gemini/types.go (3)
DefaultPageSize(1254-1254)GeminiListModelsResponse(1273-1276)GeminiModel(1256-1270)core/schemas/bifrost.go (1)
ModelProvider(32-32)
transports/bifrost-http/handlers/inference.go (5)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(23-32)BifrostListModelsRequest(10-21)core/schemas/provider.go (1)
Provider(197-224)
core/providers/vertex.go (6)
core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (6)
BifrostError(322-331)ErrorField(340-347)RequestCancelled(282-282)RequestType(79-79)ListModelsRequest(82-82)Vertex(40-40)core/schemas/providers/vertex/models.go (1)
ToVertexListModelsURL(10-26)core/schemas/provider.go (1)
Provider(197-224)core/schemas/providers/vertex/types.go (2)
VertexListModelsResponse(61-64)VertexEmbeddingRequest(21-24)
core/providers/parasail.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
transports/bifrost-http/integrations/openai.go (5)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (5)
Azure(36-36)BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)OpenAI(35-35)transports/bifrost-http/integrations/utils.go (8)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(74-103)core/schemas/providers/anthropic/chat.go (1)
ToAnthropicChatCompletionError(986-1007)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(55-80)core/schemas/provider.go (1)
Provider(197-224)
core/providers/azure.go (5)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/utils.go (1)
Ptr(12-14)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (2)
BifrostError(322-331)Groq(43-43)
🔇 Additional comments (50)
plugins/telemetry/version (1)
1-1: LGTM - Version bump coordinated with feature release.The version increment from 1.3.15 to 1.3.16 aligns with the ListModels feature addition across the codebase.
plugins/logging/version (1)
1-1: LGTM - Version bump coordinated with feature release.Consistent with other plugin version updates in this PR.
plugins/mocker/changelog.md (1)
4-4: LGTM - Changelog updated for version bump.The changelog entry correctly documents the version updates to core (1.2.14) and framework (1.1.16) that accompany this feature release.
plugins/jsonparser/version (1)
1-1: LGTM - Version bump coordinated with feature release.Consistent with the coordinated version updates across all plugins.
core/providers/sgl.go (1)
71-74: LGTM - Unsupported operation stub follows established pattern.The ListModels stub correctly returns an unsupported operation error, consistent with other unsupported operations in the SGL provider (Speech, Transcription). Method signature matches the Provider interface.
core/schemas/providers/vertex/types.go (2)
3-3: LGTM - Import added for new time field.The
timepackage import is required for thetime.Timefield inVertexModel.VersionCreateTime(line 56).
48-64: LGTM - Well-structured model types with correct JSON tags.The Model Types section is well-organized:
DefaultPageSize = 100provides a reasonable pagination defaultVertexModelcaptures all necessary model metadata with appropriate field typesVertexListModelsResponsecorrectly uses camelCase"nextPageToken"JSON tag, matching Vertex AI's API response format (as confirmed in previous review discussions)core/schemas/providers/openai/types.go (1)
128-143: LGTM - OpenAI model types with documented GROQ compatibility.The MODEL TYPES section correctly defines:
OpenAIModelwith standard OpenAI fields (ID, Object, Created, OwnedBy) and optional GROQ-specific fields (Active, ContextWindow) for cross-provider compatibilityOpenAIListModelsResponsematching the OpenAI API's list models response structure- Appropriate use of pointers for optional fields with
omitemptytagscore/providers/openrouter.go (1)
7-7: LGTM - Imports added for ListModels implementation.The
fmtandsonicimports are correctly added for debug logging (line 90) and JSON unmarshaling (lines 95, 100) in the new ListModels method.Also applies to: 12-12
core/providers/anthropic.go (1)
191-256: LGTM! Well-structured ListModels implementation.The implementation follows Anthropic provider patterns consistently:
- Proper operation allowlist check with ListModelsRequest
- Uses centralized URL builder (ToAnthropicListModelsURL) for clean separation
- Standard error handling flow with handleProviderAPIError
- Complete extra fields population (Provider, RequestType, Latency, RawResponse)
- Consistent with other provider ListModels implementations in the PR
core/schemas/providers/gemini/types.go (1)
1252-1276: LGTM! Clean type definitions for Gemini models.The types are well-structured:
- DefaultPageSize constant aligns with other providers (1000)
- GeminiModel struct captures all relevant model metadata
- BaseModelID follows Go naming conventions correctly
- GeminiListModelsResponse includes proper godoc comment
- JSON tags are appropriate for API serialization
transports/bifrost-http/integrations/utils.go (3)
118-118: LGTM! Correct ListModelsResponseConverter type definition.The signature
func(*schemas.BifrostListModelsResponse) (interface{}, error)follows the same pattern as other response converters in this file.
263-267: LGTM! Proper validation for GET route configuration.The startup validation ensures ListModelsResponseConverter is non-nil for GET routes, preventing runtime nil dereference errors. This follows the pattern established for other route validations.
388-407: LGTM! Consistent ListModels request handling.The non-streaming handler for ListModels follows the established pattern:
- Calls client.ListModelsRequest with proper context
- Executes post-callback when configured
- Validates response is non-nil after callback
- Converts response using ListModelsResponseConverter
Fully consistent with other request type handlers (TextCompletion, ChatCompletion, etc.).
core/providers/azure.go (2)
73-141: LGTM! Well-refactored completeRequest to support multiple HTTP methods.The signature update and implementation changes properly handle both deployment-based and resource-level endpoints:
- Method parameter enables GET, POST, etc.
- Conditional body marshaling (only when requestBody != nil)
- URL construction correctly branches on model parameter:
model != ""→ deployment URL:/openai/deployments/{deployment}/{path}model == ""→ resource URL:/openai/{path}(for list endpoints)- Body only set for non-GET requests with non-nil requestBody
- Maintains backward compatibility for existing calls
168-201: LGTM! Correct ListModels implementation for Azure.The implementation properly leverages the refactored completeRequest:
- Uses
http.MethodGetwith nil requestBody- Path
"models"and model""produce correct URL:{endpoint}/openai/models?api-version=...- Response parsing and conversion follow established Azure patterns
- Extra fields (Provider, Latency, RequestType) properly populated
- RawResponse conditionally set based on sendBackRawResponse flag
This addresses the past review feedback about URL construction.
plugins/jsonparser/changelog.md (1)
4-4: LGTM! Coordinated version bump.Version update aligns with the broader PR changes adding ListModels support.
tests/core-providers/openai_test.go (1)
58-58: LGTM! Enables ListModels testing for OpenAI.The flag addition follows the established pattern for test scenarios and properly enables the new ListModels functionality in comprehensive tests.
core/schemas/providers/gemini/chat.go (1)
442-444: LGTM! Correct error handling for function arguments.The logic now properly:
- Resets argsMap only when unmarshaling fails
- Preserves parsed arguments when unmarshaling succeeds
This addresses the critical bug identified in past review comments.
plugins/governance/changelog.md (1)
4-4: LGTM! Coordinated version bump.Version update matches other plugins and aligns with the core ListModels feature addition.
plugins/telemetry/changelog.md (1)
4-4: LGTM - Version update is appropriate.The changelog entry accurately documents the version bump that aligns with the ListModels feature additions across the core and framework.
core/changelog.md (1)
4-5: LGTM - Changelog accurately documents the ListModels feature.The changelog entries clearly describe the public API surface changes introduced in this PR.
tests/core-providers/sgl_test.go (1)
43-43: LGTM - Correctly disables ListModels for SGL.The test configuration appropriately sets
ListModels: falsefor the SGL provider, which aligns with SGL's current capabilities.tests/core-providers/cohere_test.go (1)
44-44: LGTM - Enables ListModels testing for Cohere.The test configuration correctly enables
ListModels: truefor Cohere, ensuring the new functionality is properly tested.ui/lib/types/config.ts (1)
123-123: LGTM - Type definition correctly extends AllowedRequests.The addition of
list_models: booleanproperly extends theAllowedRequestsinterface to match the backend schema changes.docs/features/unified-interface.mdx (2)
88-104: LGTM - Documentation accurately reflects ListModels support.The Provider Support Matrix has been properly updated with the new "Models" column, accurately showing which providers support the list models operation. The Yes/No values align with the provider implementations and test configurations throughout the PR.
106-106: LGTM - Clear explanation of the Models column.The notes section appropriately explains that "Models" refers to the list models operation (
/v1/models), providing helpful context for readers.core/providers/groq.go (1)
69-82: LGTM - ListModels implementation is correct and consistent.The
ListModelsmethod properly delegates tohandleOpenAIListModelsRequestwith all necessary parameters. The implementation follows the established pattern for OpenAI-compatible providers and uses the correct endpoint.tests/core-providers/config/account.go (3)
40-41: New TestScenarios.ListModels flag — LGTMField addition is clear and documented. No concerns.
183-186: Azure embeddings API version default: please verify and document rationaleDefaulting Azure embeddings APIVersion to "2024-10-21" while chat uses "2024-08-01-preview". If intentional (stable vs preview), add a brief comment linking to docs; otherwise align both or drive via env only.
503-504: ListModels enabled across many providers in testsEnabling these is fine; ensure CI/env has keys per provider to prevent skips/flakes when aggregating. If any provider is intermittently unavailable, consider gating by env or marking flaky suites.
Also applies to: 531-532, 559-560, 587-588, 615-616, 643-644, 671-671, 699-699, 727-727, 755-755, 785-786, 813-814
core/schemas/providers/cohere/types.go (1)
529-545: Cohere list-models types and default page size — LGTMStructs and tags look correct; DefaultPageSize=1000 matches cross-provider defaults used elsewhere.
transports/bifrost-http/integrations/genai.go (1)
67-90: Gemini list-models route wiring — LGTMRoute, request/response converters, error mapping, and PreCallback are consistent with other integrations.
Also applies to: 156-177
transports/bifrost-http/handlers/inference.go (1)
275-279: Route registration for GET /v1/models — LGTMEndpoint exposure matches the new handler and middleware chain.
transports/bifrost-http/integrations/openai.go (2)
58-60: Azure pre-hook handles list-models — LGTMSetting r.Provider = schemas.Azure for list-models on deployment paths is correct.
327-362: OpenAI list-models route configs — LGTMRoutes, converters, and combined pre-callback are wired correctly.
transports/bifrost-http/integrations/anthropic.go (2)
77-103: Anthropic list-models route wiring — LGTMGood parity with other routes; Anthropic-shaped errors maintained.
105-137: Param extraction for Anthropic — LGTMProvider set, limit/before_id/after_id mapped to request/ExtraParams as expected.
core/providers/gemini.go (1)
74-127: Gemini ListModels implementation — LGTMUses centralized URL builder, proper headers, latency/extra fields. Looks good.
core/providers/parasail.go (1)
69-72: Aggregator behavior confirmed—no action neededVerified that
ListAllModels()correctly handles unsupported errors: when a provider returns unsupported (like Parasail's new ListModels), the aggregator logs the error, skips that provider, and continues collecting models from other configured providers. The call only fails if all providers fail and no models are accumulated. The implementation is correct.core/providers/bedrock.go (1)
395-405: Remove the nil-deref concern; all callers guarantee non-nil request.All code paths calling
ListModelsconstruct a non-nilBifrostListModelsRequestbefore passing it. TheListAllModelsfunction explicitly guards nil at its entry (line 281–283), andListModelsRequestconstructs a non-nilproviderRequestat line 237 before invokingprovider.ListModels. The implicit contract—enforced by convention across all callers—is thatrequestis non-nil. Adding a nil guard would mask this contract and introduce defensive code without addressing a real risk.Likely an incorrect or invalid review comment.
core/providers/cohere.go (1)
98-158: Cohere ListModels: solid implementation and consistent metadata.URL builder, auth, error handling, conversion, and ExtraFields look correct.
core/providers/openai.go (2)
80-154: OpenAI ListModels: correct orchestration and translation.Permission check, GET setup, error parsing, conversion, pagination, and ExtraFields are implemented cleanly.
205-206: Good consistency refactor: use http.Method constants.*Replacing string literals with stdlib constants improves readability and avoids typos.
Also applies to: 561-562, 314-314, 673-673, 1030-1030, 1321-1321, 1390-1390, 1576-1576, 1656-1656
tests/core-providers/scenarios/list_models.go (2)
14-91: ListModels test: clear assertions and good diagnostics.Covers happy path, validates critical fields, and logs sample models.
93-161: Pagination test: sensible checks and token hop.Verifies page-size cap and next-page flow; simple distinctness check is fine here.
core/providers/vertex.go (2)
135-283: Vertex ListModels: well-structured with auth pooling and 401/403 eviction.URL build, auth, error parsing, conversion, and ExtraFields are solid.
671-799: Centralized embedding handler is clean and reusable.Typed marshal, native endpoint, pooled auth, error mapping, and ExtraFields are correct.
core/bifrost.go (2)
1213-1226: CAS append for providers: nice concurrency‑safe update.Prevents lost updates when appending under races.
2159-2227: ListModels key selection branch is appropriate.Skips model/deployment checks but still filters unusable keys unless provider allows empty values.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
6675c82 to
743c6ff
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/bedrock.go (1)
67-71: Compile error: invalidfor rangeover int.This won’t compile. Use a counted loop.
Apply this diff:
- for range config.ConcurrencyAndBufferSize.Concurrency { - bedrockChatResponsePool.Put(&bedrock.BedrockConverseResponse{}) - } + for i := 0; i < config.ConcurrencyAndBufferSize.Concurrency; i++ { + bedrockChatResponsePool.Put(&bedrock.BedrockConverseResponse{}) + }transports/bifrost-http/integrations/openai.go (1)
34-93: Azure non-deployment list-models misclassified as OpenAI.When hitting /openai/v1/models with Azure endpoint/key headers (no deployment-id), Provider stays empty and later defaults to OpenAI. Detect Azure via headers and set Provider=Azure; optionally inject direct key.
Apply near the function tail:
@@ func AzureEndpointPreHook(handlerStore lib.HandlerStore) func(ctx *fasthttp.RequestCtx, req interface{}) error { - return nil + // Support Azure list-models without deployment-id using endpoint + key headers + if r, ok := req.(*schemas.BifrostListModelsRequest); ok && len(deploymentEndpoint) > 0 { + r.Provider = schemas.Azure + if handlerStore.ShouldAllowDirectKeys() && azureKey != nil { + key := schemas.Key{ + ID: uuid.New().String(), + Value: strings.TrimPrefix(string(azureKey), "Bearer "), + AzureKeyConfig: &schemas.AzureKeyConfig{Endpoint: string(deploymentEndpoint)}, + } + if len(apiVersion) > 0 { + v := string(apiVersion) + key.AzureKeyConfig.APIVersion = &v + } + ctx.SetUserValue(string(schemas.BifrostContextKeyDirectKey), key) + } + } + return nilThis keeps existing deployment behavior and correctly routes Azure non-deployment model listings.
♻️ Duplicate comments (4)
core/providers/openrouter.go (1)
89-92: Attach provider context to errors from parseOpenAIError.Currently returns an error without Provider/RequestType set, causing inconsistent error payloads.
Apply this diff:
- return nil, parseOpenAIError(resp) + err := parseOpenAIError(resp) + if err != nil { + err.ExtraFields.Provider = provider.GetProviderKey() + err.ExtraFields.RequestType = schemas.ListModelsRequest + } + return nil, errtransports/bifrost-http/integrations/anthropic.go (1)
111-118: Validate limit > 0 and return 400 on bad input.Current code accepts 0/negative limits and on parse errors returns a generic error that becomes HTTP 500. Enforce limit > 0 and surface HTTP 400.
Apply in this function:
- if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { - if limit, err := strconv.Atoi(limitStr); err == nil { - listModelsReq.PageSize = limit - } else { - return fmt.Errorf("invalid limit parameter: %w", err) - } - } + if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { + limit, err := strconv.Atoi(limitStr) + if err != nil || limit <= 0 { + return badRequest("invalid limit parameter: must be > 0") + } + listModelsReq.PageSize = limit + }Support function and error wiring (outside this file) in integrations/utils.go:
@@ -import ( +import ( + "errors" "context" "encoding/json" "fmt" @@ type PostRequestCallback func(ctx *fasthttp.RequestCtx, req interface{}, resp interface{}) error @@ +// httpStatusError allows PreCallback/Parser to signal specific HTTP codes. +type httpStatusError struct { + status int + msg string +} +func (e *httpStatusError) Error() string { return e.msg } +func badRequest(msg string) error { return &httpStatusError{status: fasthttp.StatusBadRequest, msg: msg} } @@ func newBifrostError(err error, message string) *schemas.BifrostError { if err == nil { @@ } + var se *httpStatusError + if errors.As(err, &se) { + code := se.status + return &schemas.BifrostError{ + IsBifrostError: false, + StatusCode: &code, + Error: &schemas.ErrorField{ + Message: se.msg, + Error: err, + }, + } + } + return &schemas.BifrostError{This preserves existing behavior while enabling correct 400s from PreCallback. As per coding guidelines.
core/schemas/providers/gemini/models.go (1)
55-80: Preserve NextPageToken in reverse conversion.Gemini clients won’t receive pagination token; set it from resp.NextPageToken.
- geminiResponse := &GeminiListModelsResponse{ - Models: make([]GeminiModel, 0, len(resp.Data)), - } + geminiResponse := &GeminiListModelsResponse{ + Models: make([]GeminiModel, 0, len(resp.Data)), + NextPageToken: resp.NextPageToken, + }core/schemas/providers/gemini/types.go (1)
1256-1270: Add godoc and use omitempty for optional fields; consider pointer/int32 alignment.GeminiModel lacks a doc comment and most fields should be optional; align with surrounding types (many use pointers/int32 and omitempty). Also keep Name required.
Apply this minimal patch:
+// GeminiModel represents Gemini model metadata returned by the list models API. type GeminiModel struct { - Name string `json:"name"` - BaseModelID string `json:"baseModelId"` - Version string `json:"version"` - DisplayName string `json:"displayName"` - Description string `json:"description"` - InputTokenLimit int `json:"inputTokenLimit"` - OutputTokenLimit int `json:"outputTokenLimit"` - SupportedGenerationMethods []string `json:"supportedGenerationMethods"` - Thinking bool `json:"thinking"` - Temperature float64 `json:"temperature"` - MaxTemperature float64 `json:"maxTemperature"` - TopP float64 `json:"topP"` - TopK int `json:"topK"` + Name string `json:"name"` + BaseModelID string `json:"baseModelId,omitempty"` + Version string `json:"version,omitempty"` + DisplayName string `json:"displayName,omitempty"` + Description string `json:"description,omitempty"` + InputTokenLimit int `json:"inputTokenLimit,omitempty"` + OutputTokenLimit int `json:"outputTokenLimit,omitempty"` + SupportedGenerationMethods []string `json:"supportedGenerationMethods,omitempty"` + Thinking bool `json:"thinking,omitempty"` + Temperature float64 `json:"temperature,omitempty"` + MaxTemperature float64 `json:"maxTemperature,omitempty"` + TopP float64 `json:"topP,omitempty"` + TopK int `json:"topK,omitempty"` }Optionally: consider using pointers for Temperature/TopP/TopK and int32 for token limits to match nearby config types.
🧹 Nitpick comments (15)
core/schemas/providers/openai/types.go (1)
129-138: Clarify the comment about GROQ-specific fields.The comment indicates that
ActiveandContextWindoware "GROQ specific fields," but these types are defined in theopenaipackage. This creates potential confusion about the intended scope and usage of these fields.Consider one of the following:
- If these fields are truly GROQ-specific and not used by OpenAI, update the comment to explain why they're in the OpenAI types package (e.g., "Optional fields for OpenAI-compatible providers like GROQ").
- If OpenAI also uses these fields, update the comment to reflect that.
- If the design intent is to share types across OpenAI-compatible providers, add a comment explaining the shared usage pattern.
The current implementation is functionally correct since the fields are optional (
omitempty), but clearer documentation would improve maintainability.docs/features/custom-providers.mdx (1)
64-73: Also listlist_modelsunder “Available operations.”The examples include
list_models, but the operations list below omits it. Add a bullet forlist_modelsto keep docs consistent.Also applies to: 96-105
core/providers/openrouter.go (1)
99-103: Only parse rawResponse when configured.Avoid extra allocations when raw response is disabled.
Apply this diff:
- var rawResponse interface{} - if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { - return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) - } + var rawResponse interface{} + if provider.sendBackRawResponse { + if err := sonic.Unmarshal(resp.Body(), &rawResponse); err != nil { + return nil, newBifrostOperationError(schemas.ErrProviderResponseUnmarshal, err, schemas.OpenRouter) + } + }Also applies to: 115-118
core/providers/bedrock.go (1)
120-136: Avoid shadowingurlidentifier for readability.Rename local
urltofullURLto prevent confusion with thenet/urlimport used elsewhere in this file.Apply this diff:
- var url string + var fullURL string if useRuntimeEndpoint { - url = fmt.Sprintf("https://bedrock-runtime.%s.amazonaws.com/model/%s", region, path) + fullURL = fmt.Sprintf("https://bedrock-runtime.%s.amazonaws.com/model/%s", region, path) } else { - url = fmt.Sprintf("https://bedrock.%s.amazonaws.com%s", region, path) + fullURL = fmt.Sprintf("https://bedrock.%s.amazonaws.com%s", region, path) } @@ - if requestBody != nil { - req, err = http.NewRequestWithContext(ctx, method, url, bytes.NewBuffer(jsonBody)) + if requestBody != nil { + req, err = http.NewRequestWithContext(ctx, method, fullURL, bytes.NewBuffer(jsonBody)) } else { - req, err = http.NewRequestWithContext(ctx, method, url, nil) + req, err = http.NewRequestWithContext(ctx, method, fullURL, nil) }Also applies to: 158-161
core/schemas/providers/bedrock/types.go (1)
434-451: Add GoDoc comments for exported types.Keep comments consistent with the rest of this file.
Apply this diff:
-// ==================== MODELS TYPES ==================== +// ==================== MODELS TYPES ==================== + +// BedrockModelLifecycle represents the lifecycle status of a Bedrock model. type BedrockModelLifecycle struct { Status string `json:"status"` } +// BedrockModel describes a foundation model returned by AWS Bedrock, +// including identifiers, modalities, provider, lifecycle, and streaming support. type BedrockModel struct { CustomizationsSupported []string `json:"customizationsSupported,omitempty"` InferenceTypesSupported []string `json:"inferenceTypesSupported,omitempty"` InputModalities []string `json:"inputModalities,omitempty"` ModelArn string `json:"modelArn"` ModelID string `json:"modelId"` ModelLifecycle BedrockModelLifecycle `json:"modelLifecycle,omitempty"` ModelName string `json:"modelName"` OutputModalities []string `json:"outputModalities,omitempty"` ProviderName string `json:"providerName"` ResponseStreamingSupported bool `json:"responseStreamingSupported"` } // BedrockListModelsResponse represents the response from AWS Bedrock's ListFoundationModels API type BedrockListModelsResponse struct { ModelSummaries []BedrockModel `json:"modelSummaries"` }Also applies to: 452-455
transports/bifrost-http/integrations/anthropic.go (1)
77-103: Route wiring LGTM; optional: remove unused param.CreateAnthropicListModelsRouteConfigs doesn’t use handlerStore. Consider removing it or documenting parity intent with other providers.
transports/bifrost-http/integrations/openai.go (1)
364-366: Docstring mismatch: no query param extraction here.The helper only chains Azure pre-hook and sets provider default; update the comment to reflect behavior.
-// setQueryParamsAndAzureEndpointPreHook creates a combined pre-callback for OpenAI list models -// that handles both Azure endpoint preprocessing and query parameter extraction +// setQueryParamsAndAzureEndpointPreHook combines Azure endpoint preprocessing +// with provider defaulting for list-models requests.Also applies to: 367-389
core/schemas/providers/gemini/types.go (2)
1272-1276: Make NextPageToken omitempty.Avoid emitting an empty token when not provided.
type GeminiListModelsResponse struct { Models []GeminiModel `json:"models"` - NextPageToken string `json:"nextPageToken"` + NextPageToken string `json:"nextPageToken,omitempty"` }
1254-1255: Optional: clarify constant scope.
DefaultPageSizeis very generic in the gemini package; considerGeminiDefaultPageSizeor colocating it with URL builders to avoid cross‑package ambiguity.core/providers/vertex.go (3)
193-201: Evict cached OAuth client on non-context Do() errors (parity with other paths).On HTTP Do errors here we don’t evict the cached client; Chat/Embedding do. Add eviction to avoid reusing a bad client.
resp, err := client.Do(req) latency := time.Since(startTime) if err != nil { + // Evict cached client on network/auth related errors (parity with other methods) + removeVertexClient(key.VertexKeyConfig.AuthCredentials) if errors.Is(err, context.Canceled) { return nil, &schemas.BifrostError{
181-186: Minor: avoid Content-Type on GET; consider Accept header.
Content-Typeisn’t needed for GET; optionally setAccept: application/jsoninstead.
135-152: Consider adding operation-allowed check for consistency.Other providers guard ListModels with
checkOperationAllowed. If Vertex should follow the same policy, add it; otherwise, add a brief comment explaining the exemption.core/providers/openai.go (1)
91-154: Defensive nil-guard for pagination; optional header tweak.Guard against a nil
requestbeforeApplyPagination. Also,Content-Typeon GET is unnecessary; you may setAccept: application/jsoninstead.- response := openaiResponse.ToBifrostListModelsResponse(providerName) - response = response.ApplyPagination(request.PageSize, request.PageToken) + response := openaiResponse.ToBifrostListModelsResponse(providerName) + if request != nil { + response = response.ApplyPagination(request.PageSize, request.PageToken) + }tests/core-providers/scenarios/list_models.go (1)
100-160: Solid pagination coverage.Looks good. Optional: assert no overlap between page 1 and 2 IDs when NextPageToken is present (set-based check) to strengthen the test.
core/bifrost.go (1)
211-274: Consider adding retry logic for network resilience.The method calls
provider.ListModelsdirectly without the retry wrapper used by inference requests. While ListModels is a metadata operation, it still makes network calls that can fail transiently. Adding basic retry logic (similar torequestWorkerlines 1828-1862) would improve resilience against transient network errors.If you'd like to maintain consistency with other requests, you could route this through a similar retry mechanism, though for a read-only metadata operation the current approach may be acceptable.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (88)
core/bifrost.go(8 hunks)core/changelog.md(1 hunks)core/providers/anthropic.go(4 hunks)core/providers/azure.go(10 hunks)core/providers/bedrock.go(7 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(6 hunks)core/providers/gemini.go(5 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(2 hunks)core/providers/openai.go(12 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(4 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(1 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(1 hunks)core/schemas/providers/gemini/chat.go(1 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/version(1 hunks)docs/apis/openapi.json(5 hunks)docs/features/custom-providers.mdx(2 hunks)docs/features/unified-interface.mdx(1 hunks)framework/changelog.md(1 hunks)framework/version(1 hunks)plugins/governance/changelog.md(1 hunks)plugins/governance/version(1 hunks)plugins/jsonparser/changelog.md(1 hunks)plugins/jsonparser/version(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/version(1 hunks)plugins/maxim/changelog.md(1 hunks)plugins/maxim/version(1 hunks)plugins/mocker/changelog.md(1 hunks)plugins/mocker/version(1 hunks)plugins/otel/changelog.md(1 hunks)plugins/otel/version(1 hunks)plugins/semanticcache/changelog.md(1 hunks)plugins/semanticcache/version(1 hunks)plugins/telemetry/changelog.md(1 hunks)plugins/telemetry/version(1 hunks)tests/core-providers/anthropic_test.go(1 hunks)tests/core-providers/azure_test.go(1 hunks)tests/core-providers/bedrock_test.go(1 hunks)tests/core-providers/cohere_test.go(1 hunks)tests/core-providers/config/account.go(14 hunks)tests/core-providers/gemini_test.go(1 hunks)tests/core-providers/groq_test.go(1 hunks)tests/core-providers/mistral_test.go(1 hunks)tests/core-providers/ollama_test.go(1 hunks)tests/core-providers/openai_test.go(1 hunks)tests/core-providers/openrouter_test.go(1 hunks)tests/core-providers/parasail_test.go(1 hunks)tests/core-providers/scenarios/list_models.go(1 hunks)tests/core-providers/sgl_test.go(1 hunks)tests/core-providers/tests.go(2 hunks)tests/core-providers/vertex_test.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(8 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)transports/changelog.md(1 hunks)transports/version(1 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (4)
- plugins/semanticcache/changelog.md
- core/version
- plugins/maxim/version
- plugins/otel/changelog.md
🚧 Files skipped from review as they are similar to previous changes (50)
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- transports/version
- core/schemas/providers/gemini/chat.go
- core/schemas/providers/bedrock/models.go
- tests/core-providers/bedrock_test.go
- transports/bifrost-http/handlers/inference.go
- core/providers/parasail.go
- plugins/maxim/changelog.md
- tests/core-providers/parasail_test.go
- core/schemas/providers/mistral/models.go
- tests/core-providers/anthropic_test.go
- ui/lib/types/config.ts
- plugins/otel/version
- core/schemas/providers/mistral/types.go
- core/schemas/account.go
- core/schemas/providers/openai/models.go
- framework/changelog.md
- tests/core-providers/openai_test.go
- framework/version
- core/schemas/providers/cohere/models.go
- core/providers/mistral.go
- core/schemas/providers/azure/models.go
- plugins/telemetry/changelog.md
- tests/core-providers/openrouter_test.go
- core/providers/gemini.go
- ui/lib/constants/config.ts
- tests/core-providers/ollama_test.go
- tests/core-providers/groq_test.go
- core/schemas/providers/anthropic/models.go
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- core/schemas/bifrost.go
- plugins/jsonparser/version
- plugins/governance/changelog.md
- plugins/logging/version
- ui/app/providers/fragments/allowedRequestsFields.tsx
- tests/core-providers/gemini_test.go
- core/schemas/providers/azure/types.go
- core/schemas/providers/cohere/types.go
- tests/core-providers/cohere_test.go
- transports/changelog.md
- plugins/jsonparser/changelog.md
- docs/apis/openapi.json
- plugins/telemetry/version
- core/schemas/providers/anthropic/types.go
- tests/core-providers/mistral_test.go
- tests/core-providers/azure_test.go
- tests/core-providers/vertex_test.go
- plugins/mocker/version
- plugins/mocker/changelog.md
- tests/core-providers/sgl_test.go
🧰 Additional context used
🧬 Code graph analysis (23)
tests/core-providers/tests.go (1)
tests/core-providers/scenarios/list_models.go (2)
RunListModelsTest(15-91)RunListModelsPaginationTest(94-161)
core/providers/anthropic.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (3)
BifrostError(322-331)Anthropic(37-37)ListModelsRequest(82-82)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsURL(11-43)core/schemas/providers/anthropic/types.go (2)
AnthropicError(305-311)AnthropicListModelsResponse(283-288)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/config/account.go (1)
core/utils.go (1)
Ptr(12-14)
core/schemas/providers/gemini/models.go (6)
core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/anthropic/types.go (1)
DefaultPageSize(274-274)core/schemas/providers/cohere/types.go (1)
DefaultPageSize(529-529)core/schemas/providers/gemini/types.go (3)
DefaultPageSize(1254-1254)GeminiListModelsResponse(1273-1276)GeminiModel(1256-1270)core/schemas/providers/vertex/types.go (1)
DefaultPageSize(50-50)core/schemas/bifrost.go (1)
ModelProvider(32-32)
core/providers/sgl.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/providers/cerebras.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/models.go (1)
BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
ListModelsRequest(82-82)
core/schemas/providers/vertex/models.go (3)
core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/vertex/types.go (2)
DefaultPageSize(50-50)VertexListModelsResponse(61-64)core/schemas/bifrost.go (1)
Vertex(40-40)
core/providers/ollama.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (1)
BifrostError(322-331)
tests/core-providers/scenarios/list_models.go (5)
tests/core-providers/config/account.go (1)
ComprehensiveTestConfig(44-60)core/schemas/provider.go (1)
Provider(197-224)core/schemas/models.go (1)
BifrostListModelsRequest(10-21)core/schemas/bifrost.go (2)
ListModelsRequest(82-82)RequestType(79-79)tests/core-providers/scenarios/utils.go (1)
GetErrorMessage(550-574)
core/schemas/provider.go (3)
core/schemas/bifrost.go (2)
ListModelsRequest(82-82)BifrostError(322-331)core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)
core/providers/bedrock.go (5)
core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/bifrost.go (4)
BifrostError(322-331)ErrorField(340-347)Bedrock(38-38)ListModelsRequest(82-82)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(453-455)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(74-103)core/schemas/providers/anthropic/chat.go (1)
ToAnthropicChatCompletionError(986-1007)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(55-80)core/schemas/provider.go (1)
Provider(197-224)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (8)
KeySelector(16-16)ListModelsRequest(82-82)BifrostError(322-331)BifrostResponseExtraFields(254-262)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(254-262)
core/providers/vertex.go (6)
core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (4)
BifrostError(322-331)ErrorField(340-347)ListModelsRequest(82-82)Vertex(40-40)core/schemas/providers/vertex/models.go (1)
ToVertexListModelsURL(10-26)core/schemas/provider.go (5)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)ErrProviderJSONMarshaling(26-26)core/schemas/providers/vertex/types.go (2)
VertexListModelsResponse(61-64)VertexEmbeddingRequest(21-24)
transports/bifrost-http/integrations/openai.go (5)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (5)
Azure(36-36)BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)OpenAI(35-35)transports/bifrost-http/integrations/utils.go (6)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/providers/azure.go (5)
core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/provider.go (2)
ErrProviderJSONMarshaling(26-26)Provider(197-224)core/utils.go (1)
Ptr(12-14)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)
core/providers/openrouter.go (3)
core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(82-82)core/schemas/provider.go (2)
ErrProviderResponseUnmarshal(25-25)Provider(197-224)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (5)
BifrostError(322-331)OpenAI(35-35)ListModelsRequest(82-82)ModelProvider(32-32)RequestType(79-79)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(140-143)core/schemas/provider.go (1)
Provider(197-224)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)core/schemas/bifrost.go (2)
BifrostError(322-331)Groq(43-43)
core/providers/cohere.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(10-21)BifrostListModelsResponse(23-32)Model(34-51)core/schemas/bifrost.go (3)
BifrostError(322-331)Cohere(39-39)ListModelsRequest(82-82)core/schemas/providers/cohere/models.go (1)
ToCohereListModelsURL(10-34)core/schemas/providers/cohere/types.go (2)
CohereError(521-525)CohereListModelsResponse(542-545)core/schemas/provider.go (1)
Provider(197-224)
🔇 Additional comments (42)
plugins/governance/version (1)
1-1: LGTM!Standard version bump as part of the ListModels feature release.
plugins/logging/changelog.md (1)
4-4: LGTM!Changelog entry appropriately documents the version updates for core and framework.
plugins/semanticcache/version (1)
1-1: LGTM!Standard version bump coordinated with the ListModels feature additions.
core/changelog.md (1)
4-5: LGTM!Changelog appropriately documents the public API additions for the ListModels feature, including the new Provider interface method and core tracking enhancements.
docs/features/unified-interface.mdx (2)
88-103: LGTM!The provider support matrix accurately reflects the ListModels implementation status across providers. The "Models" column correctly shows "Yes" for providers with ListModels support (like Groq) and "No" for providers returning unsupported errors (like SGL and Parasail).
106-106: LGTM!Clear explanation of the Models column and its relationship to the
/v1/modelsendpoint.core/providers/sgl.go (1)
71-74: LGTM!The ListModels implementation correctly returns an unsupported operation error for SGL, which aligns with the documentation showing "No" for Models support. The method signature matches the Provider interface requirements.
core/providers/groq.go (1)
69-82: LGTM!The ListModels implementation correctly delegates to the shared
handleOpenAIListModelsRequesthelper, following the established pattern for OpenAI-compatible providers. The URL construction and parameter passing are appropriate.core/schemas/providers/openai/types.go (1)
140-143: LGTM!The
OpenAIListModelsResponsestructure follows standard OpenAI API conventions with theobjectfield anddataarray.core/schemas/provider.go (3)
90-100: ListModels allowlist added — LGTM.Field and JSON tag look correct; ordering matches doc guidance.
109-111: IsOperationAllowed supports ListModels — LGTM.Switch branch returns the correct flag.
200-201: Provider interface extended with ListModels — LGTM.Signature matches schemas and downstream usage.
core/providers/anthropic.go (1)
191-256: ListModels implementation looks solid.Good use of typed models, helpers, and consistent ExtraFields population.
core/schemas/models.go (1)
161-221: Pagination helper — LGTM.Opaque token, bounds checks, and LastID validation are implemented correctly; ExtraFields preserved.
core/providers/azure.go (1)
168-201: ListModels endpoint wiring — LGTM.Correct non-deployment path and consistent response handling.
ui/lib/types/schemas.ts (1)
284-284: LGTM! Consistent schema extension.The
list_modelsfield follows the established pattern for allowed request types and integrates cleanly with the broader ListModels feature.tests/core-providers/tests.go (2)
52-53: LGTM! Test scenarios properly integrated.The new ListModels test scenarios are correctly added to the execution flow, ensuring comprehensive coverage for model listing and pagination.
89-89: LGTM! Test summary updated.The test summary correctly reflects the new ListModels scenario, maintaining consistency with the test reporting structure.
core/schemas/providers/vertex/types.go (1)
48-64: LGTM! Vertex model types properly defined.The model listing types are correctly implemented. The
nextPageTokenJSON tag matches Vertex AI's API format (as confirmed in previous review discussion), and theDefaultPageSizeof 100 is reasonable for pagination.core/providers/ollama.go (1)
71-74: LGTM! OpenAI-compatible delegation.The ListModels implementation correctly delegates to the shared OpenAI handler, consistent with Ollama's OpenAI-compatible API surface.
core/providers/cerebras.go (1)
64-67: LGTM! Consistent OpenAI-compatible implementation.The ListModels method follows the same delegation pattern as other OpenAI-compatible providers, ensuring consistency across the codebase.
tests/core-providers/config/account.go (3)
40-40: LGTM! TestScenarios extended for ListModels.The new
ListModelsfield enables systematic testing of model listing functionality across providers.
503-504: LGTM! ListModels enabled across providers.The test configurations correctly enable ListModels for all providers that support the functionality, ensuring comprehensive test coverage.
Also applies to: 531-532, 559-560, 587-588, 615-616, 643-644, 670-671, 698-699, 726-727, 754-755, 785-786, 813-814
183-185: Azure API version 2024-10-21 is confirmed as valid.The API version 2024-10-21 is officially documented by Microsoft in the Azure OpenAI REST reference for the embeddings endpoint, and the upgrade from the preview version to this stable release is appropriate.
transports/bifrost-http/integrations/genai.go (3)
6-6: LGTM! Import added for pagination parsing.The
strconvimport is correctly added to support pageSize parsing in the ListModels parameter extraction.
67-89: LGTM! Gemini ListModels route properly configured.The route configuration correctly:
- Maps
/v1beta/modelsGET endpoint to ListModels functionality- Converts requests/responses using Gemini-specific converters
- Uses consistent error mapping with
gemini.ToGeminiError- Delegates parameter extraction to the PreCallback
156-177: LGTM! Parameter extraction with downstream validation.The parameter extraction correctly:
- Sets the provider to Gemini
- Parses pagination parameters from query string
- Silently ignores parsing errors (validation handled downstream per developer comment)
As confirmed in the previous review, pageSize validation (checking for positive values) is handled downstream in the request processing pipeline.
core/schemas/providers/vertex/models.go (2)
10-26: LGTM! URL builder with proper pagination defaults.The function correctly:
- Applies a sensible default pageSize (100) when not specified or invalid
- Builds URL with properly encoded query parameters
- Includes optional pageToken for pagination continuation
28-48: LGTM! Response mapping with provider-namespaced IDs.The conversion function correctly:
- Handles nil receiver gracefully
- Preallocates the Data slice for efficiency
- Preserves NextPageToken for pagination
- Namespaces model IDs with "Vertex/" prefix for cross-provider uniqueness
- Maps Vertex-specific fields (DisplayName, Description, VersionCreateTime) to the common Bifrost model schema
transports/bifrost-http/integrations/anthropic.go (1)
97-99: Anthropic error shape: good fix.ErrorConverter now returns Anthropic-formatted errors; consistent with other Anthropic endpoints.
transports/bifrost-http/integrations/openai.go (1)
327-362: OpenAI list-models routes: LGTM.Endpoints registered for /v1/models, /models, and Azure deployment path with converters and pre-hook chaining look correct.
core/schemas/providers/gemini/models.go (1)
29-53: ToBifrost conversion: LGTM.Preallocation, “models/” trim, and pointer handling are correct.
transports/bifrost-http/integrations/utils.go (4)
118-119: New ListModelsResponseConverter type: LGTM.Clear contract; aligns with non-streaming list-models flow.
206-206: RouteConfig additions: LGTM.Type field kept; ListModelsResponseConverter required for GET routes.
Also applies to: 212-212
250-267: Startup validation: LGTM.Validates factory and list-models converter for GET; prevents nil panics.
388-408: Non-streaming list-models handling: LGTM.Calls client.ListModelsRequest and uses integration converter; post-callback hook respected.
core/providers/cohere.go (1)
98-158: Cohere ListModels: solid implementation.Auth check, URL builder with encoding, error mapping, and ExtraFields wiring look correct.
core/providers/vertex.go (2)
681-686: Embedding endpoint choice and request shaping look correct.Using Vertex publishers/google predict endpoint for embeddings is appropriate.
371-373: Method constant refactor LGTM.Using
http.MethodPostimproves consistency.core/providers/openai.go (2)
80-89: OpenAI ListModels wiring looks good.Checks permission, delegates to handler, and returns enriched response.
111-114: HTTP method constants refactor LGTM.Consistent use of
http.MethodGet/http.MethodPostacross handlers improves clarity and avoids string literals.Also applies to: 204-206, 561-562, 923-924, 1321-1322, 1389-1390, 314-315, 673-674, 1030-1031, 1576-1577, 1656-1657
tests/core-providers/scenarios/list_models.go (1)
21-90: Baseline list-models scenario reads well.Validations and logging are sufficient and provider-agnostic.
Pratham-Mishra04
left a comment
Pratham-Mishra04
left a comment
There was a problem hiding this comment.
added comments
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
2 times, most recently
from
6e6b6cb to
1a64780
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
transports/bifrost-http/handlers/inference.go (1)
319-329: Critical compilation error: fasthttp Args has no All() method.Line 321 uses
ctx.QueryArgs().All()which does not exist in fasthttp's API. This will cause a compilation failure. The correct approach is to useVisitAllas documented in fasthttp.Apply this diff:
- // Pass-through unknown query params for provider-specific features - extraParams := map[string]interface{}{} - for k, v := range ctx.QueryArgs().All() { - s := string(k) - if s != "provider" && s != "page_size" && s != "page_token" { - extraParams[s] = string(v) - } - } + // Pass-through unknown query params for provider-specific features + extraParams := make(map[string]interface{}) + ctx.QueryArgs().VisitAll(func(k, v []byte) { + s := string(k) + if s != "provider" && s != "page_size" && s != "page_token" { + extraParams[s] = string(v) + } + })core/providers/bedrock.go (1)
97-116: Remove unreachable context deadline check on JSON marshal.Marshaling errors won’t be context deadline errors; simplify error handling.
- jsonBody, err := sonic.Marshal(requestBody) - if err != nil { - if errors.Is(err, context.DeadlineExceeded) { - return nil, 0, &schemas.BifrostError{ - IsBifrostError: false, - Error: &schemas.ErrorField{ - Type: schemas.Ptr(schemas.RequestCancelled), - Message: fmt.Sprintf("Request cancelled or timed out by context: %v", ctx.Err()), - Error: err, - }, - } - } - return nil, 0, &schemas.BifrostError{ + jsonBody, err := sonic.Marshal(requestBody) + if err != nil { + return nil, 0, &schemas.BifrostError{ IsBifrostError: true, Error: &schemas.ErrorField{ Message: schemas.ErrProviderJSONMarshaling, Error: err, }, } }transports/bifrost-http/integrations/anthropic.go (1)
111-118: Validate limit > 0; reject zero/negative.Allowing non‑positive values is misleading; return a 400‑style error instead.
- if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { - if limit, err := strconv.Atoi(limitStr); err == nil { - listModelsReq.PageSize = limit - } else { - return fmt.Errorf("invalid limit parameter: %w", err) - } - } + if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { + limit, err := strconv.Atoi(limitStr) + if err != nil || limit <= 0 { + return fmt.Errorf("invalid limit parameter: must be > 0") + } + listModelsReq.PageSize = limit + }
🧹 Nitpick comments (12)
core/schemas/providers/vertex/types.go (3)
50-50: Consider adding documentation for the exported constant.The exported
DefaultPageSizeconstant would benefit from a godoc comment explaining its purpose and usage context (e.g., "DefaultPageSize is the default number of models returned per page when listing Vertex AI models").
52-59: Struct looks good; consider adding godoc for the exported type.The
VertexModelstruct correctly uses camelCase JSON tags consistent with Google API conventions. The field types are appropriate.As an optional improvement, add a godoc comment (e.g., "VertexModel represents a model resource in the Vertex AI API") to document the exported type.
61-64: LGTM: NextPageToken JSON tag correctly uses camelCase.The struct correctly implements the Vertex AI list models response format. The
NextPageTokenfield now uses the proper"nextPageToken"JSON tag (camelCase) as required by the Vertex AI API, addressing the previous review concern.As an optional improvement, consider adding a godoc comment for this exported type (e.g., "VertexListModelsResponse represents the response from the Vertex AI list models API").
core/schemas/providers/bedrock/types.go (1)
438-454: Add documentation for BedrockModelLifecycle and BedrockModel.While BedrockListModelsResponse has a doc comment (line 455), BedrockModelLifecycle and BedrockModel lack documentation. All existing types in this file have comprehensive documentation comments, and maintaining this consistency improves code maintainability.
Apply this diff to add documentation:
// ==================== MODELS TYPES ==================== + +// BedrockModelLifecycle represents the lifecycle status of a Bedrock model type BedrockModelLifecycle struct { Status string `json:"status"` } +// BedrockModel represents a foundation model available in AWS Bedrock +// It includes metadata about the model's capabilities, modalities, and configuration type BedrockModel struct { CustomizationsSupported []string `json:"customizationsSupported,omitempty"` InferenceTypesSupported []string `json:"inferenceTypesSupported,omitempty"` InputModalities []string `json:"inputModalities,omitempty"` ModelArn string `json:"modelArn"` ModelID string `json:"modelId"` ModelLifecycle BedrockModelLifecycle `json:"modelLifecycle,omitempty"` ModelName string `json:"modelName"` OutputModalities []string `json:"outputModalities,omitempty"` ProviderName string `json:"providerName"` ResponseStreamingSupported bool `json:"responseStreamingSupported"` }core/schemas/providers/openai/models.go (1)
3-3: OpenAI reverse mapping: strip provider prefix and set object="list".OpenAI model IDs are raw (e.g., "gpt-4o"); returning "openai/gpt-4o" may confuse clients. Also, OpenAI list responses usually include object="list".
- package openai - -import "github.com/maximhq/bifrost/core/schemas" +package openai + +import ( + "strings" + "github.com/maximhq/bifrost/core/schemas" +) @@ - openaiResponse := &OpenAIListModelsResponse{ - Data: make([]OpenAIModel, 0, len(response.Data)), - } + openaiResponse := &OpenAIListModelsResponse{ + Object: "list", + Data: make([]OpenAIModel, 0, len(response.Data)), + } @@ - for _, model := range response.Data { - openaiModel := OpenAIModel{ - ID: model.ID, + for _, model := range response.Data { + id := model.ID + if parts := strings.SplitN(id, "/", 2); len(parts) == 2 { + id = parts[1] + } + openaiModel := OpenAIModel{ + ID: id, Object: "model", }Also applies to: 33-41
core/providers/bedrock.go (1)
118-121: Use http.MethodPost constant.Minor consistency/readability improvement.
- req, err := http.NewRequestWithContext(ctx, "POST", fmt.Sprintf("https://bedrock-runtime.%s.amazonaws.com/model/%s", region, path), bytes.NewBuffer(jsonBody)) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, fmt.Sprintf("https://bedrock-runtime.%s.amazonaws.com/model/%s", region, path), bytes.NewBuffer(jsonBody))core/schemas/providers/gemini/models.go (1)
60-62: Propagate NextPageToken in reverse mapping.Include pagination token so Gemini clients can request next page.
- geminiResponse := &GeminiListModelsResponse{ - Models: make([]GeminiModel, 0, len(resp.Data)), - } + geminiResponse := &GeminiListModelsResponse{ + Models: make([]GeminiModel, 0, len(resp.Data)), + NextPageToken: resp.NextPageToken, + }core/providers/vertex.go (1)
184-186: Minor: Prefer Accept over Content-Type on GET.Setting Content-Type on GET is unnecessary; optionally set Accept: application/json instead.
- req.Header.Set("Content-Type", "application/json") + req.Header.Set("Accept", "application/json")core/providers/azure.go (2)
177-180: Avoid Content-Type on GET.Setting Content-Type on a GET is unnecessary and can confuse intermediaries. Remove it for ListModels.
req.SetRequestURI(url) req.Header.SetMethod(http.MethodGet) -req.Header.SetContentType("application/json")
64-67: Make helper intent explicit (POST to deployments only).completeRequest always uses POST and deployment paths. Add a note to avoid future misuse for resource-level GETs like /models.
-// completeRequest sends a request to Azure's API and handles the response. -// It constructs the API URL, sets up authentication, and processes the response. +// completeRequest sends a POST request to Azure deployment-scoped endpoints and handles the response. +// Note: Not for resource-level operations (e.g., GET /openai/models). Use a dedicated request path for those. // Returns the response body, request latency, or an error if the request fails.Also applies to: 110-112
docs/apis/openapi.json (2)
312-320: Constrain page_size with sane bounds.Add minimum/maximum to aid clients and prevent pathological requests.
"schema": { - "type": "integer" + "type": "integer", + "minimum": 1, + "maximum": 1000 },
6578-6597: Deduplicate extra_fields by referencing existing schema.Reuse BifrostResponseExtraFields to keep one source of truth for latency/raw_response/provider/request_type.
- "extra_fields": { - "type": "object", - "properties": { - "provider": { "$ref": "#/components/schemas/ModelProvider" }, - "request_type": { "$ref": "#/components/schemas/RequestType" }, - "latency": { - "type": "number", - "description": "Request latency in milliseconds", - "example": 1234 - }, - "raw_response": { - "type": "object", - "description": "Raw provider response" - } - } - } + "extra_fields": { + "$ref": "#/components/schemas/BifrostResponseExtraFields" + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (88)
core/bifrost.go(8 hunks)core/changelog.md(1 hunks)core/providers/anthropic.go(4 hunks)core/providers/azure.go(6 hunks)core/providers/bedrock.go(4 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(6 hunks)core/providers/gemini.go(5 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(2 hunks)core/providers/openai.go(12 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(4 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(2 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(1 hunks)core/schemas/providers/gemini/chat.go(1 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/version(1 hunks)docs/apis/openapi.json(5 hunks)docs/features/custom-providers.mdx(2 hunks)docs/features/unified-interface.mdx(1 hunks)framework/changelog.md(1 hunks)framework/version(1 hunks)plugins/governance/changelog.md(1 hunks)plugins/governance/version(1 hunks)plugins/jsonparser/changelog.md(1 hunks)plugins/jsonparser/version(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/version(1 hunks)plugins/maxim/changelog.md(1 hunks)plugins/maxim/version(1 hunks)plugins/mocker/changelog.md(1 hunks)plugins/mocker/version(1 hunks)plugins/otel/changelog.md(1 hunks)plugins/otel/version(1 hunks)plugins/semanticcache/changelog.md(1 hunks)plugins/semanticcache/version(1 hunks)plugins/telemetry/changelog.md(1 hunks)plugins/telemetry/version(1 hunks)tests/core-providers/anthropic_test.go(1 hunks)tests/core-providers/azure_test.go(1 hunks)tests/core-providers/bedrock_test.go(1 hunks)tests/core-providers/cohere_test.go(1 hunks)tests/core-providers/config/account.go(14 hunks)tests/core-providers/gemini_test.go(1 hunks)tests/core-providers/groq_test.go(1 hunks)tests/core-providers/mistral_test.go(1 hunks)tests/core-providers/ollama_test.go(1 hunks)tests/core-providers/openai_test.go(1 hunks)tests/core-providers/openrouter_test.go(1 hunks)tests/core-providers/parasail_test.go(1 hunks)tests/core-providers/scenarios/list_models.go(1 hunks)tests/core-providers/sgl_test.go(1 hunks)tests/core-providers/tests.go(2 hunks)tests/core-providers/vertex_test.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(8 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)transports/changelog.md(1 hunks)transports/version(1 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (3)
- plugins/telemetry/version
- plugins/governance/version
- plugins/jsonparser/version
🚧 Files skipped from review as they are similar to previous changes (48)
- tests/core-providers/sgl_test.go
- plugins/logging/version
- ui/lib/constants/config.ts
- tests/core-providers/bedrock_test.go
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- plugins/governance/changelog.md
- tests/core-providers/cohere_test.go
- core/schemas/providers/azure/models.go
- core/version
- core/schemas/providers/mistral/models.go
- core/schemas/account.go
- core/schemas/providers/gemini/chat.go
- core/schemas/providers/gemini/types.go
- plugins/jsonparser/changelog.md
- tests/core-providers/openai_test.go
- transports/changelog.md
- tests/core-providers/parasail_test.go
- framework/changelog.md
- core/schemas/providers/bedrock/models.go
- framework/version
- core/providers/anthropic.go
- core/providers/groq.go
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- plugins/semanticcache/changelog.md
- core/schemas/providers/azure/types.go
- tests/core-providers/scenarios/list_models.go
- plugins/otel/version
- core/schemas/providers/vertex/models.go
- tests/core-providers/openrouter_test.go
- ui/lib/types/config.ts
- core/schemas/providers/anthropic/models.go
- ui/app/providers/fragments/allowedRequestsFields.tsx
- core/providers/openai.go
- plugins/mocker/changelog.md
- core/schemas/providers/mistral/types.go
- plugins/mocker/version
- core/schemas/providers/cohere/models.go
- plugins/maxim/version
- tests/core-providers/groq_test.go
- plugins/otel/changelog.md
- plugins/maxim/changelog.md
- core/providers/cerebras.go
- core/schemas/bifrost.go
- core/schemas/providers/cohere/types.go
- transports/version
- core/changelog.md
- plugins/logging/changelog.md
- plugins/telemetry/changelog.md
🧰 Additional context used
🧬 Code graph analysis (23)
core/schemas/providers/anthropic/types.go (1)
core/schemas/providers/gemini/types.go (1)
Type(769-769)
core/providers/cohere.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (4)
BifrostError(322-331)Cohere(39-39)ListModelsRequest(82-82)RequestType(79-79)core/schemas/providers/cohere/models.go (1)
ToCohereListModelsURL(10-34)core/schemas/providers/cohere/types.go (2)
CohereError(521-525)CohereListModelsResponse(540-543)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/handlers/inference.go (5)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(28-37)BifrostListModelsRequest(15-26)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/tests.go (1)
tests/core-providers/scenarios/list_models.go (2)
RunListModelsTest(15-91)RunListModelsPaginationTest(94-161)
core/providers/sgl.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/schemas/providers/openai/models.go (3)
core/schemas/providers/openai/types.go (2)
OpenAIListModelsResponse(140-143)OpenAIModel(129-138)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (2)
BifrostListModelsResponse(28-37)Model(39-56)
core/providers/vertex.go (8)
core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (4)
BifrostError(322-331)ErrorField(340-347)ListModelsRequest(82-82)Vertex(40-40)core/schemas/providers/vertex/models.go (1)
ToVertexListModelsURL(10-26)core/utils.go (1)
Ptr(12-14)core/schemas/provider.go (5)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)ErrProviderJSONMarshaling(26-26)core/schemas/providers/vertex/types.go (2)
VertexListModelsResponse(61-64)VertexEmbeddingRequest(21-24)core/schemas/embedding.go (1)
BifrostEmbeddingResponse(17-23)
core/providers/parasail.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (1)
BifrostError(322-331)
transports/bifrost-http/integrations/utils.go (1)
core/schemas/models.go (1)
BifrostListModelsResponse(28-37)
core/providers/ollama.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/schemas/provider.go (3)
core/schemas/bifrost.go (1)
ListModelsRequest(82-82)core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)
transports/bifrost-http/integrations/openai.go (6)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (5)
Azure(36-36)BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)OpenAI(35-35)transports/bifrost-http/integrations/utils.go (6)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (9)
KeySelector(16-16)ListModelsRequest(82-82)BifrostError(322-331)ErrorField(340-347)BifrostResponseExtraFields(254-262)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (5)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)DefaultPageSize(11-11)MaxPaginationRequests(13-13)
core/providers/bedrock.go (5)
core/schemas/providers/bedrock/types.go (3)
DefaultBedrockRegion(4-4)BedrockError(349-353)BedrockListModelsResponse(456-458)core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (5)
BifrostError(322-331)Bedrock(38-38)ListModelsRequest(82-82)ErrorField(340-347)RequestType(79-79)core/schemas/provider.go (4)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(254-262)
transports/bifrost-http/integrations/anthropic.go (5)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(74-103)core/schemas/providers/anthropic/chat.go (1)
ToAnthropicChatCompletionError(986-1007)
core/providers/gemini.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (3)
BifrostError(322-331)Gemini(47-47)ListModelsRequest(82-82)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsURL(11-27)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1272-1275)core/schemas/provider.go (1)
Provider(197-224)
core/providers/mistral.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/provider.go (1)
Provider(197-224)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(82-82)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/gemini/models.go (2)
core/schemas/models.go (4)
BifrostListModelsRequest(15-26)DefaultPageSize(11-11)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/providers/gemini/types.go (2)
GeminiListModelsResponse(1272-1275)GeminiModel(1255-1269)
tests/core-providers/config/account.go (1)
core/utils.go (1)
Ptr(12-14)
core/providers/azure.go (5)
core/utils.go (1)
Ptr(12-14)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(55-80)core/schemas/provider.go (1)
Provider(197-224)
🔇 Additional comments (60)
plugins/semanticcache/version (1)
1-1: Verify version bump aligns with PR scope.This patch-level bump (1.3.15 → 1.3.16) for the semantic cache plugin seems modest given that the PR introduces a significant new feature (list models endpoint across multiple providers). Please confirm whether this version bump is appropriate or if a minor version bump would better reflect the scope of changes.
core/schemas/providers/vertex/types.go (1)
3-4: LGTM: Time import necessary for timestamp field.The
timeimport is correctly added to support theVersionCreateTimefield inVertexModel.docs/features/custom-providers.mdx (1)
64-73: LGTM!The
list_modelsfield has been correctly added to the allowed_requests examples and positioned first in the field order, which aligns with the codebase conventions.Also applies to: 96-105
tests/core-providers/vertex_test.go (1)
43-43: LGTM!Enabling ListModels in the test scenarios is appropriate and aligns with the broader PR changes introducing ListModels support across providers.
core/schemas/providers/anthropic/types.go (1)
272-286: LGTM!The model types are well-structured and correctly handle nullable pagination fields with pointer types, which properly addresses Anthropic's API where these fields can be
null.core/providers/ollama.go (1)
71-74: LGTM!The ListModels implementation correctly delegates to the shared OpenAI-compatible handler, which is appropriate since Ollama exposes an OpenAI-compatible API. The pattern is consistent with other methods in this provider.
core/schemas/providers/openai/types.go (1)
128-143: LGTM!The model types are well-structured and appropriately handle OpenAI's standard fields while also accommodating GROQ-specific extensions with optional pointer fields.
core/providers/cohere.go (1)
98-158: LGTM!The ListModels implementation is comprehensive and follows best practices:
- Includes proper authorization checks
- Uses centralized URL construction to ensure proper encoding
- Implements robust error handling
- Tracks latency and populates response metadata
- Follows the established pattern used by other operations in this provider
core/schemas/providers/bedrock/types.go (1)
3-4: LGTM!The DefaultBedrockRegion constant is a good practice for centralizing configuration and enabling per-key overrides.
tests/core-providers/ollama_test.go (1)
42-42: LGTM!Enabling ListModels in the test scenarios is appropriate and aligns with the broader PR changes introducing ListModels support across providers.
tests/core-providers/azure_test.go (1)
47-47: LGTM!Enabling ListModels testing for Azure is consistent with the PR's objectives and test infrastructure additions.
tests/core-providers/mistral_test.go (1)
45-45: LGTM!Enabling ListModels testing for Mistral follows the same pattern as other providers in this PR.
tests/core-providers/anthropic_test.go (1)
46-46: LGTM!Enabling ListModels testing for Anthropic is consistent with the broader test coverage additions.
tests/core-providers/gemini_test.go (1)
53-53: LGTM!Enabling ListModels testing for Gemini completes the test coverage pattern across providers.
ui/lib/types/schemas.ts (1)
284-284: LGTM!Adding the
list_modelsboolean field to the allowed requests schema is consistent with the existing pattern and enables UI configuration of the new capability.docs/features/unified-interface.mdx (1)
88-110: LGTM!The provider support matrix has been comprehensively updated to include the new Models column, and the notes section clearly explains the feature. The past feedback regarding Cerebras support has been addressed.
tests/core-providers/tests.go (2)
52-53: LGTM!The new ListModels test scenarios are properly integrated into the comprehensive test runner, covering both basic functionality and pagination.
89-89: LGTM!The test summary correctly includes the ListModels scenario for reporting.
transports/bifrost-http/handlers/inference.go (2)
277-278: LGTM!The route registration for the list models endpoint is correct and properly chains the middlewares.
289-318: LGTM!The handler structure is well-organized: context conversion, parameter parsing, conditional routing to ListAllModels or ListModelsRequest, and proper error handling.
Also applies to: 330-345
tests/core-providers/config/account.go (3)
40-40: LGTM! ListModels field addition is consistent.The new
ListModelsbool field follows the same pattern as other test scenario flags in the struct and aligns with the PR's objective to add ListModels support across providers.
183-185: Clarify the API version change for Azure embedding config.The comment and API version were updated from a preview version to
2024-10-21. Ensure this is the correct stable version for Azure embedding endpoints and that it's intentional to have different versions for the main Azure config (line 171:2024-08-01-preview) versus the embedding config.
503-503: LGTM! ListModels enabled consistently across all providers.The
ListModels: truesettings are applied uniformly across all provider configurations, which aligns with the PR objectives. The generic routing architecture (as clarified in past comments) ensures these work without provider-specific route configs.Also applies to: 531-531, 559-559, 587-587, 615-615, 643-643, 670-670, 698-698, 726-726, 754-754, 785-785, 813-813
transports/bifrost-http/integrations/genai.go (3)
6-6: LGTM! Import addition is necessary.The
strconvimport is required forstrconv.Atoiusage in theextractGeminiListModelsParamsfunction.
67-89: LGTM! ListModels route config follows established pattern.The new route configuration:
- Uses the correct HTTP method (GET) and path (
/v1beta/models)- Properly wires up request/response converters and error handling
- Uses Gemini-specific error mapping via
gemini.ToGeminiError- Follows the same structure as the existing chat route above
156-177: LGTM! Parameter extraction is clean and consistent.The
extractGeminiListModelsParamsfunction:
- Correctly sets the provider to Gemini
- Safely parses query parameters (pageSize and pageToken)
- Silently ignores parsing errors for pageSize (falling back to zero), which is acceptable since validation is handled downstream
Based on learnings
core/providers/mistral.go (2)
12-12: LGTM! Import addition is necessary.The mistral schemas import is required for accessing
MistralListModelsResponseand related types.
70-120: LGTM! ListModels implementation follows established pattern.The implementation:
- Uses standard fasthttp request/response pattern with proper resource cleanup
- Applies extra headers from network config
- Makes GET request to
/v1/modelsendpoint with Bearer token authentication- Handles errors using
parseOpenAIError(appropriate for OpenAI-compatible APIs)- Converts Mistral response to Bifrost format via
ToBifrostListModelsResponse- Applies client-side pagination (correct approach since Mistral doesn't support server-side pagination)
- Properly sets all ExtraFields metadata
Based on learnings
core/providers/sgl.go (1)
71-74: LGTM! Correctly returns unsupported operation error.The ListModels implementation appropriately returns an unsupported operation error for SGL, following the same pattern used for other unsupported operations in this provider.
core/providers/parasail.go (1)
69-72: LGTM! Correctly returns unsupported operation error.The ListModels implementation appropriately returns an unsupported operation error for Parasail, consistent with the pattern for other unsupported operations in this provider.
core/schemas/provider.go (3)
90-90: LGTM! ListModels field positioned correctly.The
ListModelsbool field is appropriately placed first in theAllowedRequestsstruct, following the reviewer's guidance to keeplist_modelsfirst in schema definitions.As per coding guidelines
109-110: LGTM! Operation check correctly implemented.The
ListModelsRequestcase correctly returnsar.ListModels, maintaining consistency with the existing operation permission logic.
200-201: LGTM! Interface method is well-defined.The
ListModelsmethod signature is properly documented and consistent with other Provider interface methods. Past reviews confirm all concrete provider implementations have been added.core/providers/openrouter.go (2)
7-7: LGTM! Import addition is necessary.The
fmtimport is required for debug logging in the error handling path (line 89).
65-116: LGTM! ListModels implementation is solid.The implementation:
- Uses correct HTTP method (GET) and Authorization Bearer header
- Properly handles errors and measures latency
- Directly unmarshals into
BifrostListModelsResponse(acceptable since OpenRouter's schema matches Bifrost's, per developer clarification)- Prefixes model IDs with provider namespace for disambiguation
- Applies client-side pagination correctly
- Sets all required ExtraFields metadata
Note: Past reviewer suggested using
handleRequestutil for response parsing (line 93 comment), but the current approach usinghandleProviderResponseis consistent with other providers in this file.core/providers/gemini.go (2)
74-127: LGTM! ListModels implementation follows best practices.The implementation:
- Properly gates operations via
checkOperationAllowed- Uses centralized URL construction via
gemini.ToGeminiListModelsURL(good separation of concerns)- Follows standard request/response pattern with proper resource cleanup
- Handles errors using Gemini-specific error parsing
- Converts response to Bifrost format via
ToBifrostListModelsResponse- Sets all required ExtraFields metadata
171-171: LGTM! HTTP method constant normalization improves code quality.Replacing hard-coded
"POST"strings withhttp.MethodPostconstant improves type safety and follows Go best practices.Also applies to: 377-377, 655-655, 914-914
transports/bifrost-http/integrations/openai.go (3)
58-60: Good addition: Azure list‑models request is recognized.Setting Provider = schemas.Azure for deployment routes unblocks correct downstream handling.
327-362: Routes and converters look correct.GET routes register with proper request factory, request converter, response converter, and pre‑callback. LGTM.
364-389: Confirm upstream parsing of provider/page params.This pre‑callback only defaults Provider to OpenAI; it doesn’t parse provider/page params. If those are parsed earlier (as stated), great—otherwise consider adding here.
transports/bifrost-http/integrations/anthropic.go (1)
97-101: Anthropic errors returned in provider format.Good consistency with other endpoints.
core/schemas/providers/gemini/models.go (1)
34-50: Bifrost mapping looks solid.Preallocation, ID normalization, and pointers are correct.
transports/bifrost-http/integrations/utils.go (2)
262-267: Startup validation for GET routes is correct.Requiring ListModelsResponseConverter avoids nil panics.
387-407: ListModels execution path wired correctly.Calls client, optional post‑callback, converts via ListModelsResponseConverter. LGTM.
core/providers/vertex.go (3)
233-260: Auth client eviction on 401/403 is good.Prevents replaying stale tokens; matches other Vertex paths.
667-669: Embedding helper extraction looks clean.Centralizes request/response handling; improves consistency.
781-786: ModelRequested set from actual model.Good for observability/telemetry.
core/schemas/models.go (2)
77-81: Token counts as integers.Correct type choice for JSON and clients.
166-226: Pagination helper is robust.Opaque cursor, lastID validation, safe fallbacks. LGTM.
core/providers/azure.go (1)
165-167: Correct Azure models URL.Good fix: targets /openai/models with api-version correctly.
docs/apis/openapi.json (2)
55-58: Models tag added correctly.Tag is declared and used by the new endpoint.
291-350: /v1/models endpoint looks solid.Clear summary/description, tags, params, and responses wired to new schemas.
core/bifrost.go (8)
36-51: LGTM! Providers tracking field properly declared.The
providersatomic pointer field is appropriately used for concurrent access and integrates well with the initialization and lookup logic.
95-97: LGTM! Providers slice properly initialized.The empty slice initialization ensures the atomic pointer is never nil and ready for provider registration.
211-274: Consider provider initialization edge case.The method uses
getProviderByKey(line 237) which looks up the cached provider list, unlike other request methods that usegetProviderQueuewith auto-initialization. If a provider configuration is added dynamically afterInitand hasn't been initialized by another request yet,ListModelsRequestwill fail with "provider not found."While this is likely a rare edge case (providers are initialized in
Initfor all configured providers), consider documenting this behavior or adding fallback initialization if needed.Note:
ListAllModelshandles this gracefully by logging and continuing with other providers (lines 320-327).
276-372: LGTM! ListAllModels implementation is robust and well-designed.The method properly handles:
- Nil request guard (lines 281-283)
- Pagination safety limits using constants (lines 311-317)
- Nil response checks (lines 329-331)
- Per-provider error handling with graceful degradation (lines 320-327)
- Consistent output via sorting (lines 353-355)
- Final pagination application (line 369)
All concerns from past reviews have been addressed.
1211-1224: LGTM! CAS loop correctly prevents concurrent update issues.The compare-and-swap loop ensures that concurrent
prepareProvidercalls don't lose updates when appending to the providers slice. This addresses the concern raised in past reviews and provides robust concurrency safety.
1278-1293: LGTM! getProviderByKey helper is correctly implemented.The method safely loads the atomic providers pointer, handles nil cases, and performs a straightforward lookup. Callers appropriately handle the nil return value.
1792-1792: LGTM! selectKeyFromProviderForModel call updated correctly.The call now includes
req.RequestTypeas the second parameter, matching the updated method signature and enabling request-type-specific key selection logic.
2157-2225: LGTM! Key selection logic correctly handles ListModelsRequest.The updated signature and logic properly differentiate between request types:
ListModelsRequest (lines 2177-2183): Skips model/deployment checks since listing models doesn't require a model-specific key; validates only that the key value exists or the provider allows empty keys.
Other request types (lines 2184-2205): Performs full validation including model support checks and Azure/Bedrock deployment mapping validation.
This design is sound and addresses all concerns from past reviews.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
1a64780 to
87d97a2
Compare
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
87d97a2 to
9bafa85
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
transports/bifrost-http/integrations/utils.go (1)
969-1001: Fix fallbacks reflection: FieldByName("fallbacks") misses exported/json‑tagged fields.Current lookup won’t find exported Fallbacks or fields tagged
json:"fallbacks", so fallbacks are silently ignored.- // Look for the "fallbacks" field - fallbacksField := reqValue.FieldByName("fallbacks") - if !fallbacksField.IsValid() { - return nil, nil // No fallbacks field found - } - - // Handle different types of fallbacks field - switch fallbacksField.Kind() { + // Find exported field named Fallbacks (case-insensitive) or with json tag "fallbacks" + t := reqValue.Type() + var fallbacksField reflect.Value + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if f.PkgPath != "" { // unexported + continue + } + nameMatch := strings.EqualFold(f.Name, "fallbacks") + tagName := strings.Split(f.Tag.Get("json"), ",")[0] + tagMatch := tagName == "fallbacks" + if nameMatch || tagMatch { + fallbacksField = reqValue.Field(i) + break + } + } + if !fallbacksField.IsValid() { + return nil, nil // No fallbacks field found + } + if fallbacksField.Kind() == reflect.Ptr && !fallbacksField.IsNil() { + fallbacksField = fallbacksField.Elem() + } + + // Handle different types of fallbacks field + switch fallbacksField.Kind() { case reflect.Slice: if fallbacksField.Type().Elem().Kind() == reflect.String { // []string case fallbacks := make([]string, fallbacksField.Len()) for i := 0; i < fallbacksField.Len(); i++ { fallbacks[i] = fallbacksField.Index(i).String() } return fallbacks, nil } case reflect.String: // Single string case - treat as one fallback return []string{fallbacksField.String()}, nil }
♻️ Duplicate comments (4)
transports/bifrost-http/handlers/inference.go (1)
321-329: Compile-time bug: fasthttp QueryArgs has no All(); use VisitAllReplace the iteration with VisitAll to build ExtraParams.
- // Pass-through unknown query params for provider-specific features - extraParams := map[string]interface{}{} - for k, v := range ctx.QueryArgs().All() { - s := string(k) - if s != "provider" && s != "page_size" && s != "page_token" { - extraParams[s] = string(v) - } - } + // Pass-through unknown query params for provider-specific features + extraParams := make(map[string]interface{}) + ctx.QueryArgs().VisitAll(func(k, v []byte) { + s := string(k) + if s != "provider" && s != "page_size" && s != "page_token" { + extraParams[s] = string(v) + } + })transports/bifrost-http/integrations/anthropic.go (1)
112-118: Missing validation: enforce positive limit.The code parses the
limitparameter but doesn't validate it's positive. Negative or zero values will pass through, potentially causing unexpected behavior.Apply this fix:
if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { if limit, err := strconv.Atoi(limitStr); err == nil { + if limit <= 0 { + return fmt.Errorf("invalid limit parameter: must be positive") + } listModelsReq.PageSize = limit } else { return fmt.Errorf("invalid limit parameter: %w", err) } }core/schemas/providers/bedrock/types.go (1)
438-453: Add missing documentation comments.
BedrockModelLifecycleandBedrockModellack doc comments. All other types in this file have comprehensive documentation. Add doc comments to maintain consistency:+// BedrockModelLifecycle represents the lifecycle status of a Bedrock foundation model type BedrockModelLifecycle struct { Status string `json:"status"` } +// BedrockModel represents a foundation model available in AWS Bedrock +// It includes metadata about the model's capabilities, modalities, ARN, and provider information type BedrockModel struct { CustomizationsSupported []string `json:"customizationsSupported,omitempty"` InferenceTypesSupported []string `json:"inferenceTypesSupported,omitempty"` InputModalities []string `json:"inputModalities,omitempty"` ModelArn string `json:"modelArn"` ModelID string `json:"modelId"` ModelLifecycle BedrockModelLifecycle `json:"modelLifecycle,omitempty"` ModelName string `json:"modelName"` OutputModalities []string `json:"outputModalities,omitempty"` ProviderName string `json:"providerName"` ResponseStreamingSupported bool `json:"responseStreamingSupported"` }docs/apis/openapi.json (1)
6736-6776: Prefer numeric types for pricing fields (repeat).Using string for monetary amounts complicates client math and validation; consider number/double.
- "prompt": { "type": "string", "description": "Cost per input token in USD", "example": "0.0000025" }, + "prompt": { "type": "number", "format": "double", "description": "Cost per input token in USD", "example": 0.0000025 }, - "completion": { "type": "string", "description": "Cost per output token in USD", "example": "0.00001" }, + "completion": { "type": "number", "format": "double", "description": "Cost per output token in USD", "example": 0.00001 }, - "request": { "type": "string", "description": "Fixed cost per API request in USD", "example": "0" }, + "request": { "type": "number", "format": "double", "description": "Fixed cost per API request in USD", "example": 0 }, - "image": { "type": "string", "description": "Cost per image input in USD", "example": "0.003613" }, + "image": { "type": "number", "format": "double", "description": "Cost per image input in USD", "example": 0.003613 }, - "web_search": { "type": "string", "description": "Cost per web search operation in USD", "example": "0" }, + "web_search": { "type": "number", "format": "double", "description": "Cost per web search operation in USD", "example": 0 }, - "internal_reasoning": { "type": "string", "description": "Cost for internal reasoning tokens in USD", "example": "0" }, + "internal_reasoning": { "type": "number", "format": "double", "description": "Cost for internal reasoning tokens in USD", "example": 0 }, - "input_cache_read": { "type": "string", "description": "Cost per cached input token read in USD", "example": "0.00000125" }, + "input_cache_read": { "type": "number", "format": "double", "description": "Cost per cached input token read in USD", "example": 0.00000125 }, - "input_cache_write": { "type": "string", "description": "Cost per cached input token write in USD", "example": "0" } + "input_cache_write": { "type": "number", "format": "double", "description": "Cost per cached input token write in USD", "example": 0 }
🧹 Nitpick comments (7)
core/schemas/models.go (2)
39-56: Consider type consistency for Created field.Line 43 uses
*int64for the Created timestamp field, but similar numeric fields (ContextLength, MaxCompletionTokens) in TopProvider were changed from*float64to*intper the past review comment to align with OpenAPI integer types. For consistency, consider whether Created should also be*int.If OpenAPI defines this as an integer, apply this diff:
type Model struct { ID string `json:"id"` CanonicalSlug *string `json:"canonical_slug,omitempty"` Name *string `json:"name,omitempty"` - Created *int64 `json:"created,omitempty"` + Created *int `json:"created,omitempty"` ContextLength *int `json:"context_length,omitempty"`
169-196: Silent cursor reset could hide errors.Lines 184-187 silently reset an invalid cursor to offset 0 without logging or returning an error. This behavior could mask cursor tampering, data corruption, or client bugs.
Consider either:
- Logging when a cursor is reset:
logger.Warn("Invalid pagination cursor, resetting to start")- Returning an error to the caller so they're aware the cursor was rejected
- Including validation failure details in ExtraFields for debugging
This would improve observability and help diagnose pagination issues in production.
tests/core-providers/scenarios/list_models.go (2)
15-25: Mark helpers with t.Helper() for accurate failure locations.Small ergonomics win; keeps failure lines pointing to calling tests.
func RunListModelsTest(t *testing.T, client *bifrost.Bifrost, ctx context.Context, testConfig config.ComprehensiveTestConfig) { + t.Helper() @@ func RunListModelsPaginationTest(t *testing.T, client *bifrost.Bifrost, ctx context.Context, testConfig config.ComprehensiveTestConfig) { + t.Helper()Also applies to: 100-111
128-157: Also assert ExtraFields in pagination test (parity with basic test).Keeps both scenarios consistent and catches mis‑wiring early.
response, bifrostErr := client.ListModelsRequest(ctx, request) @@ if response == nil { t.Fatal("❌ List models pagination response is nil") } + + // Sanity-check extra fields + if response.ExtraFields.RequestType != schemas.ListModelsRequest { + t.Errorf("❌ Request type mismatch: expected %s, got %s", schemas.ListModelsRequest, response.ExtraFields.RequestType) + } + if response.ExtraFields.Provider != testConfig.Provider { + t.Errorf("❌ Provider mismatch: expected %s, got %s", testConfig.Provider, response.ExtraFields.Provider) + }docs/apis/openapi.json (2)
312-319: Constrain page_size.Add a minimum (and optional sensible maximum) to aid client validation.
"schema": { - "type": "integer" + "type": "integer", + "minimum": 1, + "maximum": 1000 },
6581-6597: Allow provider to be null/omitted when listing across all providers.Matches the endpoint behavior when the query param is not supplied.
"extra_fields": { "type": "object", "properties": { "provider": { - "$ref": "#/components/schemas/ModelProvider" + "$ref": "#/components/schemas/ModelProvider", + "nullable": true, + "description": "Provider for this request; null/omitted when listing across all providers." },core/bifrost.go (1)
230-235: Consider applying default PageSize for consistency.The
PageSizefield is copied as-is from the request (line 232). If the caller passes 0 or omits it, the behavior depends on downstream handling. For consistency, consider applyingschemas.DefaultPageSizewhenreq.PageSizeis 0, similar to howListAllModelsexplicitly usesschemas.DefaultPageSizeat line 308.Apply this diff if you want to enforce a consistent default:
request := &schemas.BifrostListModelsRequest{ Provider: req.Provider, - PageSize: req.PageSize, + PageSize: req.PageSize, PageToken: req.PageToken, ExtraParams: req.ExtraParams, } + +// Apply default page size if not specified +if request.PageSize == 0 { + request.PageSize = schemas.DefaultPageSize +}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (89)
core/bifrost.go(8 hunks)core/changelog.md(1 hunks)core/providers/anthropic.go(4 hunks)core/providers/azure.go(6 hunks)core/providers/bedrock.go(4 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(6 hunks)core/providers/gemini.go(5 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(2 hunks)core/providers/openai.go(12 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(4 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(2 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(1 hunks)core/schemas/providers/gemini/chat.go(1 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/version(1 hunks)docs/apis/openapi.json(6 hunks)docs/features/custom-providers.mdx(2 hunks)docs/features/unified-interface.mdx(1 hunks)framework/changelog.md(1 hunks)framework/version(1 hunks)plugins/governance/changelog.md(1 hunks)plugins/governance/version(1 hunks)plugins/jsonparser/changelog.md(1 hunks)plugins/jsonparser/version(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/version(1 hunks)plugins/maxim/changelog.md(1 hunks)plugins/maxim/version(1 hunks)plugins/mocker/changelog.md(1 hunks)plugins/mocker/version(1 hunks)plugins/otel/changelog.md(1 hunks)plugins/otel/version(1 hunks)plugins/semanticcache/changelog.md(1 hunks)plugins/semanticcache/version(1 hunks)plugins/telemetry/changelog.md(1 hunks)plugins/telemetry/version(1 hunks)tests/core-providers/anthropic_test.go(1 hunks)tests/core-providers/azure_test.go(1 hunks)tests/core-providers/bedrock_test.go(1 hunks)tests/core-providers/cerebras_test.go(1 hunks)tests/core-providers/cohere_test.go(1 hunks)tests/core-providers/config/account.go(14 hunks)tests/core-providers/gemini_test.go(1 hunks)tests/core-providers/groq_test.go(1 hunks)tests/core-providers/mistral_test.go(1 hunks)tests/core-providers/ollama_test.go(1 hunks)tests/core-providers/openai_test.go(1 hunks)tests/core-providers/openrouter_test.go(1 hunks)tests/core-providers/parasail_test.go(1 hunks)tests/core-providers/scenarios/list_models.go(1 hunks)tests/core-providers/sgl_test.go(1 hunks)tests/core-providers/tests.go(2 hunks)tests/core-providers/vertex_test.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(8 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)transports/changelog.md(1 hunks)transports/version(1 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- framework/changelog.md
🚧 Files skipped from review as they are similar to previous changes (53)
- plugins/otel/version
- tests/core-providers/gemini_test.go
- tests/core-providers/sgl_test.go
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- core/schemas/provider.go
- tests/core-providers/groq_test.go
- core/schemas/providers/vertex/models.go
- tests/core-providers/mistral_test.go
- core/schemas/providers/openai/models.go
- core/schemas/providers/azure/models.go
- ui/lib/types/config.ts
- tests/core-providers/bedrock_test.go
- core/schemas/providers/cohere/types.go
- core/schemas/providers/anthropic/models.go
- tests/core-providers/parasail_test.go
- core/providers/gemini.go
- plugins/governance/changelog.md
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- tests/core-providers/openai_test.go
- docs/features/custom-providers.mdx
- core/schemas/providers/gemini/types.go
- transports/changelog.md
- core/providers/cerebras.go
- core/schemas/providers/mistral/types.go
- framework/version
- plugins/semanticcache/version
- tests/core-providers/anthropic_test.go
- plugins/maxim/changelog.md
- core/providers/ollama.go
- ui/lib/constants/config.ts
- core/providers/sgl.go
- core/schemas/account.go
- tests/core-providers/azure_test.go
- core/schemas/providers/gemini/models.go
- plugins/semanticcache/changelog.md
- core/version
- core/providers/openrouter.go
- plugins/otel/changelog.md
- tests/core-providers/cohere_test.go
- core/providers/anthropic.go
- plugins/governance/version
- core/providers/parasail.go
- plugins/logging/version
- ui/app/providers/fragments/allowedRequestsFields.tsx
- plugins/logging/changelog.md
- plugins/mocker/changelog.md
- plugins/mocker/version
- core/schemas/providers/openai/types.go
- tests/core-providers/ollama_test.go
- tests/core-providers/openrouter_test.go
- transports/version
- docs/features/unified-interface.mdx
- core/schemas/providers/vertex/types.go
🧰 Additional context used
🧬 Code graph analysis (22)
core/schemas/bifrost.go (2)
core/schemas/models.go (1)
BifrostListModelsRequest(15-26)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/config/account.go (1)
core/utils.go (1)
Ptr(12-14)
transports/bifrost-http/integrations/genai.go (5)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(55-80)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/models.go (1)
BifrostListModelsResponse(28-37)core/schemas/bifrost.go (1)
ListModelsRequest(82-82)
core/providers/azure.go (6)
core/utils.go (1)
Ptr(12-14)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostError(322-331)Azure(36-36)RequestType(79-79)ListModelsRequest(82-82)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(39-56)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
transports/bifrost-http/integrations/openai.go (6)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (5)
Azure(36-36)BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)OpenAI(35-35)transports/bifrost-http/integrations/utils.go (8)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/schemas/providers/bedrock/models.go (3)
core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(456-458)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (3)
BifrostListModelsResponse(28-37)Model(39-56)Architecture(58-64)
core/schemas/providers/cohere/models.go (3)
core/schemas/models.go (4)
BifrostListModelsRequest(15-26)DefaultPageSize(11-11)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/providers/cohere/types.go (1)
CohereListModelsResponse(540-543)core/schemas/bifrost.go (1)
ModelProvider(32-32)
core/providers/groq.go (2)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (2)
BifrostError(322-331)Groq(43-43)
transports/bifrost-http/integrations/anthropic.go (7)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(74-103)core/schemas/providers/anthropic/chat.go (1)
ToAnthropicChatCompletionError(986-1007)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/handlers/inference.go (5)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(28-37)BifrostListModelsRequest(15-26)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(254-262)
core/schemas/providers/mistral/models.go (3)
core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (1)
Mistral(41-41)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenAI(35-35)ListModelsRequest(82-82)ModelProvider(32-32)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(140-143)core/schemas/provider.go (1)
Provider(197-224)
core/providers/vertex.go (6)
core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (5)
BifrostError(322-331)ErrorField(340-347)RequestCancelled(282-282)ListModelsRequest(82-82)Vertex(40-40)core/schemas/providers/vertex/models.go (1)
ToVertexListModelsURL(10-26)core/schemas/provider.go (5)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)ErrProviderJSONMarshaling(26-26)core/schemas/providers/vertex/types.go (2)
VertexListModelsResponse(61-64)VertexEmbeddingRequest(21-24)
tests/core-providers/tests.go (1)
tests/core-providers/scenarios/list_models.go (2)
RunListModelsTest(15-91)RunListModelsPaginationTest(94-161)
core/providers/mistral.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (2)
BifrostError(322-331)ListModelsRequest(82-82)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/provider.go (1)
Provider(197-224)
core/providers/bedrock.go (5)
core/schemas/providers/bedrock/types.go (3)
DefaultBedrockRegion(4-4)BedrockError(349-353)BedrockListModelsResponse(456-458)core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (6)
BifrostError(322-331)Bedrock(38-38)ListModelsRequest(82-82)ErrorField(340-347)RequestCancelled(282-282)RequestType(79-79)core/schemas/provider.go (5)
ErrRequestCancelled(23-23)ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (9)
KeySelector(16-16)ListModelsRequest(82-82)BifrostError(322-331)ErrorField(340-347)BifrostResponseExtraFields(254-262)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (5)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)DefaultPageSize(11-11)MaxPaginationRequests(13-13)
core/providers/cohere.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (3)
BifrostError(322-331)Cohere(39-39)ListModelsRequest(82-82)core/schemas/providers/cohere/models.go (1)
ToCohereListModelsURL(10-34)core/schemas/providers/cohere/types.go (2)
CohereError(521-525)CohereListModelsResponse(540-543)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/scenarios/list_models.go (6)
core/bifrost.go (1)
Bifrost(34-52)tests/core-providers/config/account.go (1)
ComprehensiveTestConfig(44-60)core/schemas/provider.go (1)
Provider(197-224)core/schemas/models.go (1)
BifrostListModelsRequest(15-26)core/schemas/bifrost.go (2)
ListModelsRequest(82-82)RequestType(79-79)tests/core-providers/scenarios/utils.go (1)
GetErrorMessage(550-574)
🔇 Additional comments (40)
plugins/telemetry/changelog.md (1)
4-4: Changelog entry looks good.The version bump (core 1.2.13 → 1.2.14, framework 1.1.15 → 1.1.16) is appropriate for the ListModels feature addition, and the entry format is consistent with the file pattern.
plugins/jsonparser/version (1)
1-1: Verify whether a version bump is appropriate for dependency-only changes.While the changelog was updated, it documents only dependency version updates (core 1.2.14, framework 1.1.16) marked as "chore"—not functional changes to the jsonparser plugin itself. There are no code changes to the plugin.
Clarify:
- Is bumping the patch version (1.3.15 → 1.3.16) the intended policy when only dependency versions change, not plugin functionality?
- Or should metadata-only updates not trigger version increments?
plugins/maxim/version (1)
1-1: LGTM: Version bump.Standard version increment as part of the coordinated release for ListModels functionality.
plugins/jsonparser/changelog.md (1)
4-4: LGTM: Changelog update.Version dependency documentation is accurate and consistent with the broader PR version bumps.
plugins/telemetry/version (1)
1-1: LGTM: Version bump.Standard version increment aligned with the release.
ui/lib/types/schemas.ts (1)
284-284: LGTM: Schema addition for list_models permission.The new
list_modelsfield follows the established pattern for request type permissions and is consistent with other boolean flags in the schema.core/providers/groq.go (1)
69-82: LGTM: ListModels implementation for Groq.The implementation correctly delegates to the shared OpenAI-compatible handler with proper URL construction and parameter passing. Consistent with Groq's other method implementations.
core/schemas/providers/mistral/models.go (1)
5-27: LGTM: Converter implementation with pointer capture fix.The implementation correctly uses
schemas.Ptr()for all pointer fields, avoiding the pointer capture bug that was identified in earlier reviews. The pre-allocated slice and ID prefixing follow established patterns.core/providers/mistral.go (1)
70-120: LGTM: ListModels implementation with client-side pagination.The implementation correctly handles Mistral's list models endpoint. Client-side pagination via
ApplyPaginationis appropriate since Mistral's API doesn't support pagination query parameters. Based on learnings.core/providers/vertex.go (2)
135-283: LGTM: ListModels implementation with server-side pagination.The implementation correctly:
- Validates Vertex-specific configuration requirements
- Delegates URL construction (with pagination params) to
vertex.ToVertexListModelsURL- Handles authentication client pooling and eviction on 401/403 errors
- Uses server-side pagination (no client-side re-pagination needed)
- Implements comprehensive error handling for cancellations, timeouts, and provider errors
667-669: LGTM: Embedding flow refactored for clarity.The refactoring consolidates embedding logic into
handleVertexEmbedding, improving code organization. Line 785 correctly setsModelRequestedfrom themodelparameter, ensuring accurate model tracking in the response.Also applies to: 671-799
core/schemas/models.go (2)
10-37: LGTM!The pagination constants and request/response structures are well-designed. The separation of provider-specific fields (Anthropic) using
json:"-"is appropriate.
100-120: Verify offset 0 handling in pagination cursor encoding.Lines 103-105 return an empty string when
offset <= 0. While this prevents encoding a cursor for the first page, it means offset 0 (a valid starting position) is treated the same as negative offsets (invalid). Confirm this behavior is intentional—if a cursor should never encode offset 0, this is correct; otherwise, the check should beoffset < 0.Consider whether this distinction matters for your pagination semantics:
- Current: first page has no encoded cursor (offset=0 → empty string)
- Alternative: only reject negative offsets (offset < 0 → empty string)
core/providers/openai.go (2)
80-154: LGTM! Well-structured ListModels implementation.The ListModels implementation follows the established patterns in this provider:
- Proper permission checking via
checkOperationAllowed- Clean delegation to handler function
- Correct HTTP method usage (
http.MethodGet)- Appropriate error handling and response conversion
- Pagination applied with
ApplyPagination- Metadata properly populated in
ExtraFields
205-205: LGTM! Good use of HTTP method constants.Replacing string literals with
http.MethodPostandhttp.MethodGetconstants throughout the file improves maintainability and follows Go best practices.Also applies to: 314-314, 561-561, 673-673, 923-923, 1030-1030, 1246-1246, 1321-1321, 1389-1389, 1576-1576, 1656-1656
tests/core-providers/cerebras_test.go (1)
47-47: LGTM! Test configuration updated for ListModels.Enabling
ListModels: truein the Cerebras test scenarios appropriately exercises the new ListModels functionality for this provider.core/schemas/providers/gemini/chat.go (1)
442-444: LGTM! Correct error handling for tool call arguments.The error handling now correctly preserves successfully parsed arguments by only resetting
argsMapwhenjson.Unmarshalfails. The previously problematic else branch that discarded valid data has been removed.core/changelog.md (1)
4-5: LGTM! Clear changelog entries.The changelog appropriately documents the ListModels feature addition with concise descriptions of both the Provider interface method and core tracking functionality.
core/schemas/bifrost.go (2)
82-82: LGTM! Consistent ListModels integration.The ListModelsRequest constant and field additions follow the established pattern for other request types in the BifrostRequest struct.
Also applies to: 122-122, 133-133
256-257: Verify omitempty behavior for Provider and ModelRequested.Adding
omitemptyto the Provider and ModelRequested JSON tags means these fields will be omitted from JSON output when they have zero values (empty string for strings). If downstream clients or APIs expect these fields to always be present (even if empty), this change could cause compatibility issues.Confirm that:
- It's acceptable for these fields to be absent from JSON when empty
- No existing clients rely on these fields always being present in the response
- The API contract doesn't require these fields in all responses
tests/core-providers/tests.go (1)
52-53: LGTM! Clean test integration.The ListModels test scenarios are properly integrated into the comprehensive test suite, following the established pattern for test registration and summary reporting.
Also applies to: 89-89
tests/core-providers/vertex_test.go (1)
43-43: LGTM! Test configuration updated for ListModels.Enabling
ListModels: truein the Vertex test scenarios appropriately exercises the new ListModels functionality for this provider.core/schemas/providers/anthropic/types.go (1)
281-286: Nullable pagination IDs look goodSwitch to *string with omitempty aligns with Anthropic’s nullable first_id/last_id. No further changes needed here.
tests/core-providers/config/account.go (3)
40-41: Good addition: explicit ListModels scenario flagClear test gating for model listing. Looks good.
183-186: I'll verify the Azure API version and check how list-models is implemented in the codebase.Now I'll search the codebase for list-models implementation and how it uses the APIVersion:
No issues found with the Azure API version default
Azure OpenAI API version 2024-10-21 is currently the latest GA API release. The models list endpoint is documented as
GET {endpoint}/openai/models?api-version=2024-10-21and gets a list of all models accessible by the Azure OpenAI resource.The code pattern
getEnvWithDefault("AZURE_API_VERSION", "2024-10-21")is sound: it allows environment configuration while defaulting to the latest stable production version. This is a better approach than hardcoding outdated API versions. No compatibility issues exist with list-models behavior using this version.
503-504: Review comment is incorrect — existing env var checks already protect against CI flakes.Each provider's test file (anthropic_test.go, vertex_test.go, bedrock_test.go, etc.) has upfront environment variable validation that skips the entire test suite if credentials are missing. For example, TestAnthropic checks
ANTHROPIC_API_KEYand callst.Skip()before any scenarios execute. The individual test guards already prevent ListModels from triggering API calls when required credentials are absent. The config file declarations are capability metadata, not runtime triggers. No additional gating is needed.Likely an incorrect or invalid review comment.
transports/bifrost-http/handlers/inference.go (1)
277-279: Route wiring LGTMGET /v1/models registered with middleware chain correctly.
transports/bifrost-http/integrations/genai.go (1)
67-90: Gemini list-models route is correctly integratedRequest/response/error converters and PreCallback are wired consistently with other GenAI routes.
core/schemas/providers/bedrock/models.go (1)
10-26: Solid conversion and pointer handlingPreallocation + schemas.Ptr avoids range-variable aliasing; ID prefixing and modality mapping look correct.
core/schemas/providers/cohere/models.go (2)
10-34: LGTM! Proper URL encoding and parameter handling.The function correctly uses
url.Valuesandparams.Encode()for URL-safe query parameter encoding, handles default page size for non-positive values, and safely extracts optional parameters fromExtraParamswith type assertions.
36-56: LGTM! Proper pointer handling.The conversion correctly uses
schemas.Ptr()helper for creating pointers to loop variables, avoiding the pointer capture bug. The ID construction with provider prefix and proper slice preallocation are appropriate.core/providers/azure.go (1)
147-236: LGTM! Correct Azure ListModels implementation.The method properly:
- Uses
DefaultAzureAPIVersionconstant for API version defaulting- Constructs resource-level URL (
/openai/models) without deployment path- Uses
http.MethodGetconstant- Handles both Bearer token and api-key authentication
- Sets latency in milliseconds per coding guidelines
- Applies local pagination via
ApplyPaginationThe implementation aligns with Azure's resource-level models endpoint and is consistent with other provider ListModels patterns.
transports/bifrost-http/integrations/openai.go (3)
58-59: LGTM! Azure provider correctly set for list models.The addition properly handles
BifrostListModelsRequestin the Azure endpoint pre-hook, ensuring the provider is set to Azure for deployment-based list models requests.
327-362: LGTM! Correct route configuration for list models.The function properly:
- Registers GET endpoints for standard and Azure deployment paths
- Uses appropriate request/response converters for OpenAI format
- Applies combined pre-callback for Azure endpoint handling and query parameter extraction
364-389: LGTM! Correct pre-callback composition.The function properly composes the Azure endpoint pre-hook with provider defaulting logic. It correctly:
- Runs Azure hook first (which sets provider for deployment endpoints)
- Defaults to OpenAI provider when not set by Azure hook
- Omits pagination parameter extraction (handled elsewhere per past review discussion)
core/providers/cohere.go (1)
98-158: LGTM! All past review issues addressed.The ListModels implementation correctly:
- Includes
checkOperationAllowedauthorization check- Uses
ToCohereListModelsURLhelper with proper URL encoding- Uses
http.MethodGetconstant- Sets latency in milliseconds
- Has accurate comment ("list models response")
The implementation is consistent with other Cohere operations and provider patterns.
transports/bifrost-http/integrations/anthropic.go (1)
77-103: LGTM! Correct Anthropic list models route configuration.The function properly:
- Registers GET endpoint for
/v1/models- Uses Anthropic-specific response converter
- Uses Anthropic error converter per past review
- Applies parameter extraction pre-callback
core/providers/bedrock.go (1)
348-482: LGTM! Correct Bedrock ListModels implementation.The method properly:
- Includes
checkOperationAllowedauthorization check- Uses
DefaultBedrockRegionwith per-key override support- Targets the correct
bedrockservice endpoint (notbedrock-runtime)- Uses
http.MethodGetconstant- Handles both API Key and AWS IAM authentication
- Applies local pagination via
ApplyPagination(correct per learnings that Bedrock ListFoundationModels doesn't support pagination)- Sets latency in milliseconds per coding guidelines
Based on learnings
core/schemas/providers/azure/types.go (1)
1-34: LGTM! Correct Azure type definitions.The types properly represent Azure OpenAI model metadata:
DefaultAzureAPIVersionset to latest stable version ("2024-10-21")- Capabilities structure with appropriate boolean fields
- Deprecation timestamps with
omitemptyfor optional fields- Complete model metadata structure with proper JSON tags
The types align with Azure OpenAI API schema and are consistent with other provider patterns.
transports/bifrost-http/integrations/utils.go (1)
263-267: Validation is currently correct; concern is about future extensibility rather than existing bugs.All GET routes in the codebase are list_models endpoints with ListModelsResponseConverter properly set (OpenAI, GenAI, Anthropic). The validation logic at lines 263–267 is accurate for the present architecture. However, the review comment raises a valid design concern: if a future GET endpoint serves a different purpose (e.g., metadata, status), this check would incorrectly block it since there's no mechanism to distinguish route intent beyond path patterns.
No immediate changes required since current GET routes are all models endpoints and properly configured. If extensibility for non-list_models GET routes becomes a planned feature, consider adding a route-intent flag or path-based disambiguation to the validation logic.
TejasGhatte
force-pushed
the
10-17-feat_added_list_models_request
branch
from
9bafa85 to
85e2245
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (5)
core/schemas/providers/anthropic/types.go (1)
274-279: Make CreatedAt a pointer to tolerate null/omitted valuesAnthropic may omit or null created_at; time.Time will fail on null and emit zero-time. Prefer *time.Time with omitempty and guard in converters.
type AnthropicModel struct { ID string `json:"id"` DisplayName string `json:"display_name"` - CreatedAt time.Time `json:"created_at"` + CreatedAt *time.Time `json:"created_at,omitempty"` Type string `json:"type"` }Also adjust conversions (outside this hunk):
- FromAnthropicListModelsResponse: check nil before .Unix()
- ToAnthropicListModelsResponse: set pointer value when model.Created != nil
core/schemas/providers/bedrock/types.go (1)
437-458: Add doc comments for new exported types.Keep this file consistent with existing documented types.
- type BedrockModelLifecycle struct { + // BedrockModelLifecycle represents the lifecycle status of a Bedrock model. + type BedrockModelLifecycle struct { Status string `json:"status"` } - type BedrockModel struct { + // BedrockModel describes a foundation model returned by Bedrock's listing APIs. + // It captures capabilities, modalities, identifiers, and provider metadata. + type BedrockModel struct { CustomizationsSupported []string `json:"customizationsSupported,omitempty"` InferenceTypesSupported []string `json:"inferenceTypesSupported,omitempty"` InputModalities []string `json:"inputModalities,omitempty"` ModelArn string `json:"modelArn"` ModelID string `json:"modelId"` ModelLifecycle BedrockModelLifecycle `json:"modelLifecycle,omitempty"` ModelName string `json:"modelName"` OutputModalities []string `json:"outputModalities,omitempty"` ProviderName string `json:"providerName"` ResponseStreamingSupported bool `json:"responseStreamingSupported"` }core/providers/openrouter.go (1)
88-91: Normalize provider on error returned by parseOpenAIError.Attach OpenRouter as the provider before returning the parsed error.
- return nil, parseOpenAIError(resp) + err := parseOpenAIError(resp) + if err != nil { + err.ExtraFields.Provider = provider.GetProviderKey() + } + return nil, errcore/providers/gemini.go (1)
377-377: Sanitize and URL-escape model path segment.Direct concatenation can break paths (e.g., prefixed IDs, special chars). Normalize and url.PathEscape the model part.
-req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+request.Model+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody)) +m := sanitizeGeminiModelID(request.Model) +req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+url.PathEscape(m)+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody))-req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+request.Model+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody)) +m := sanitizeGeminiModelID(request.Model) +req, err := http.NewRequestWithContext(ctx, http.MethodPost, provider.networkConfig.BaseURL+"/models/"+url.PathEscape(m)+":streamGenerateContent?alt=sse", bytes.NewReader(jsonBody))-req.SetRequestURI(provider.networkConfig.BaseURL + "/models/" + model + endpoint) +m := sanitizeGeminiModelID(model) +req.SetRequestURI(provider.networkConfig.BaseURL + "/models/" + url.PathEscape(m) + endpoint)Add import and helper:
import ( "bufio" "bytes" "context" "errors" "fmt" "io" "net/http" + "net/url" "strconv" "strings" "time" )// place near other helpers in this file func sanitizeGeminiModelID(model string) string { m := strings.TrimPrefix(model, "gemini/") return strings.TrimPrefix(m, "models/") }Also applies to: 655-655, 913-915
transports/bifrost-http/handlers/inference.go (1)
319-329: CRITICAL: fasthttp.QueryArgs has no All() method - code will not compile.The loop at line 321 calls
ctx.QueryArgs().All()which does not exist in the fasthttp API. Despite your earlier comment stating "All is recommended," the fasthttp documentation confirms thatVisitAllis the standard method for iterating query parameters.This is a blocking compilation error that must be fixed before merge.
Apply this fix:
// Pass-through unknown query params for provider-specific features -extraParams := map[string]interface{}{} -for k, v := range ctx.QueryArgs().All() { - s := string(k) - if s != "provider" && s != "page_size" && s != "page_token" { - extraParams[s] = string(v) - } -} +extraParams := make(map[string]interface{}) +ctx.QueryArgs().VisitAll(func(k, v []byte) { + s := string(k) + if s != "provider" && s != "page_size" && s != "page_token" { + extraParams[s] = string(v) + } +})
🧹 Nitpick comments (11)
core/schemas/providers/gemini/types.go (1)
1255-1269: Consider adding a godoc comment for GeminiModel.The struct is well-formed with proper Go naming conventions (e.g.,
BaseModelID), but it lacks a godoc comment. Adding one would improve consistency withGeminiListModelsResponsebelow and follow Go documentation conventions.Apply this diff:
+// GeminiModel represents metadata for a Google Gemini model including capabilities and limits. type GeminiModel struct { Name string `json:"name"` BaseModelID string `json:"baseModelId"`ui/app/providers/fragments/allowedRequestsFields.tsx (1)
48-49: Coerce to boolean to avoid uncontrolled/controlled warningsfield.value can be undefined; Switch.checked expects boolean. Coerce to Boolean for stability.
- <Switch checked={field.value} onCheckedChange={field.onChange} size="md" /> + <Switch checked={Boolean(field.value)} onCheckedChange={field.onChange} size="md" />Also applies to: 67-68
core/providers/groq.go (1)
49-54: Optional: normalize BaseURL to include “/openai” when user overrides itIf a custom BaseURL is provided without “/openai”, requests will hit /v1/* at the root. Consider auto-appending or documenting the requirement.
if config.NetworkConfig.BaseURL == "" { config.NetworkConfig.BaseURL = "https://api.groq.com/openai" } -config.NetworkConfig.BaseURL = strings.TrimRight(config.NetworkConfig.BaseURL, "/") +config.NetworkConfig.BaseURL = strings.TrimRight(config.NetworkConfig.BaseURL, "/") +if !strings.Contains(config.NetworkConfig.BaseURL, "/openai") { + config.NetworkConfig.BaseURL += "/openai" +}core/schemas/providers/cohere/types.go (2)
540-543: Omit empty next_page_tokenAvoid emitting empty string on final page.
type CohereListModelsResponse struct { Models []CohereModel `json:"models"` - NextPageToken string `json:"next_page_token"` + NextPageToken string `json:"next_page_token,omitempty"` }
529-538: Optional: mark rarely-present fields as omitemptySeveral fields (e.g., tokenizer_url, default_endpoints, features) may be absent; adding omitempty keeps payloads lean and mirrors API variance.
core/providers/openrouter.go (1)
77-80: Minor: avoid Content-Type on GET; set Accept instead.Not harmful, but cleaner to advertise what you accept rather than what you send.
req.Header.SetMethod(http.MethodGet) -req.Header.SetContentType("application/json") +req.Header.Set("Accept", "application/json") req.Header.Set("Authorization", "Bearer "+key.Value)transports/bifrost-http/integrations/anthropic.go (1)
112-118: Consider adding validation for the limit parameter.The current implementation accepts any integer value for
limit, including zero or negative numbers. While the provider API might reject invalid values, adding a validation check here would provide earlier feedback and prevent unnecessary API calls.Apply this diff to add validation:
// Extract limit from query parameters if limitStr := string(ctx.QueryArgs().Peek("limit")); limitStr != "" { if limit, err := strconv.Atoi(limitStr); err == nil { + if limit <= 0 { + return fmt.Errorf("invalid limit parameter: must be positive") + } listModelsReq.PageSize = limit } else { return fmt.Errorf("invalid limit parameter: %w", err) } }core/schemas/providers/anthropic/models.go (1)
63-69: Consider adding Object field for consistency.While the current implementation correctly uses
schemas.Ptr()to avoid pointer capture issues, theObjectfield is not set in theModelstruct. Other providers (e.g., OpenAI) set this to"model"for consistency with standard model list responses.Apply this diff to add the Object field:
for _, model := range response.Data { bifrostResponse.Data = append(bifrostResponse.Data, schemas.Model{ ID: string(providerKey) + "/" + model.ID, + Object: "model", Name: schemas.Ptr(model.DisplayName), Created: schemas.Ptr(model.CreatedAt.Unix()), }) }core/schemas/models.go (1)
166-178: Apply default page size when pageSize is invalid.Lines 176-178 return the entire response (no pagination) when
pageSize <= 0. This allows clients to bypass pagination by sending zero or negative page sizes, which could return thousands of models across all providers and cause memory/bandwidth issues.Apply this diff to default to
DefaultPageSizewhen the page size is invalid:func (response *BifrostListModelsResponse) ApplyPagination(pageSize int, pageToken string) *BifrostListModelsResponse { if response == nil { return nil } totalItems := len(response.Data) if pageSize <= 0 { - return response + pageSize = DefaultPageSize } cursor := decodePaginationCursor(pageToken)docs/apis/openapi.json (2)
55-58: Minor: Improve Models tag description.The tag description is minimal. Consider making it more descriptive about the scope of model discovery endpoints.
{ "name": "Models", - "description": "Endpoint for listing available models." + "description": "Endpoints for discovering and listing available AI models from configured providers." }
6796-6810: ModelPerRequestLimits and ModelDefaultParameters schemas present.Both schemas are defined and referenced:
- ModelPerRequestLimits (lines 6796–6807): integer token limits with clear descriptions.
- ModelDefaultParameters (lines 6809–6810): defined as an empty
type: "object".Consider clarifying ModelDefaultParameters intent: should it allow
additionalProperties: trueto accommodate provider-specific defaults? Currently, it's defined as a strict empty object, which may be intended to signal "no defaults" or could benefit from explicitadditionalProperties: falsefor clarity.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (89)
core/bifrost.go(8 hunks)core/changelog.md(1 hunks)core/providers/anthropic.go(4 hunks)core/providers/azure.go(6 hunks)core/providers/bedrock.go(4 hunks)core/providers/cerebras.go(1 hunks)core/providers/cohere.go(6 hunks)core/providers/gemini.go(5 hunks)core/providers/groq.go(1 hunks)core/providers/mistral.go(2 hunks)core/providers/ollama.go(2 hunks)core/providers/openai.go(12 hunks)core/providers/openrouter.go(2 hunks)core/providers/parasail.go(1 hunks)core/providers/sgl.go(1 hunks)core/providers/vertex.go(4 hunks)core/schemas/account.go(1 hunks)core/schemas/bifrost.go(4 hunks)core/schemas/models.go(1 hunks)core/schemas/provider.go(3 hunks)core/schemas/providers/anthropic/models.go(1 hunks)core/schemas/providers/anthropic/types.go(2 hunks)core/schemas/providers/azure/models.go(1 hunks)core/schemas/providers/azure/types.go(1 hunks)core/schemas/providers/bedrock/models.go(1 hunks)core/schemas/providers/bedrock/types.go(2 hunks)core/schemas/providers/cohere/models.go(1 hunks)core/schemas/providers/cohere/types.go(1 hunks)core/schemas/providers/gemini/chat.go(1 hunks)core/schemas/providers/gemini/models.go(1 hunks)core/schemas/providers/gemini/types.go(1 hunks)core/schemas/providers/mistral/models.go(1 hunks)core/schemas/providers/mistral/types.go(1 hunks)core/schemas/providers/openai/models.go(1 hunks)core/schemas/providers/openai/types.go(1 hunks)core/schemas/providers/vertex/models.go(1 hunks)core/schemas/providers/vertex/types.go(2 hunks)core/version(1 hunks)docs/apis/openapi.json(6 hunks)docs/features/custom-providers.mdx(2 hunks)docs/features/unified-interface.mdx(1 hunks)framework/changelog.md(1 hunks)framework/version(1 hunks)plugins/governance/changelog.md(1 hunks)plugins/governance/version(1 hunks)plugins/jsonparser/changelog.md(1 hunks)plugins/jsonparser/version(1 hunks)plugins/logging/changelog.md(1 hunks)plugins/logging/version(1 hunks)plugins/maxim/changelog.md(1 hunks)plugins/maxim/version(1 hunks)plugins/mocker/changelog.md(1 hunks)plugins/mocker/version(1 hunks)plugins/otel/changelog.md(1 hunks)plugins/otel/version(1 hunks)plugins/semanticcache/changelog.md(1 hunks)plugins/semanticcache/version(1 hunks)plugins/telemetry/changelog.md(1 hunks)plugins/telemetry/version(1 hunks)tests/core-providers/anthropic_test.go(1 hunks)tests/core-providers/azure_test.go(1 hunks)tests/core-providers/bedrock_test.go(1 hunks)tests/core-providers/cerebras_test.go(1 hunks)tests/core-providers/cohere_test.go(1 hunks)tests/core-providers/config/account.go(14 hunks)tests/core-providers/gemini_test.go(1 hunks)tests/core-providers/groq_test.go(1 hunks)tests/core-providers/mistral_test.go(1 hunks)tests/core-providers/ollama_test.go(1 hunks)tests/core-providers/openai_test.go(1 hunks)tests/core-providers/openrouter_test.go(1 hunks)tests/core-providers/parasail_test.go(1 hunks)tests/core-providers/scenarios/list_models.go(1 hunks)tests/core-providers/sgl_test.go(1 hunks)tests/core-providers/tests.go(2 hunks)tests/core-providers/vertex_test.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(3 hunks)transports/bifrost-http/integrations/genai.go(4 hunks)transports/bifrost-http/integrations/openai.go(8 hunks)transports/bifrost-http/integrations/utils.go(5 hunks)transports/changelog.md(1 hunks)transports/version(1 hunks)ui/app/providers/dialogs/addNewCustomProviderDialog.tsx(2 hunks)ui/app/providers/fragments/allowedRequestsFields.tsx(1 hunks)ui/app/providers/fragments/apiStructureFormFragment.tsx(1 hunks)ui/lib/constants/config.ts(1 hunks)ui/lib/types/config.ts(1 hunks)ui/lib/types/schemas.ts(1 hunks)
✅ Files skipped from review due to trivial changes (5)
- plugins/governance/version
- plugins/maxim/version
- plugins/otel/changelog.md
- plugins/governance/changelog.md
- transports/changelog.md
🚧 Files skipped from review as they are similar to previous changes (37)
- docs/features/custom-providers.mdx
- plugins/semanticcache/changelog.md
- transports/version
- tests/core-providers/groq_test.go
- plugins/mocker/version
- plugins/semanticcache/version
- core/schemas/providers/azure/models.go
- ui/app/providers/fragments/apiStructureFormFragment.tsx
- tests/core-providers/mistral_test.go
- core/providers/sgl.go
- tests/core-providers/vertex_test.go
- core/providers/cerebras.go
- tests/core-providers/anthropic_test.go
- ui/app/providers/dialogs/addNewCustomProviderDialog.tsx
- core/schemas/providers/vertex/types.go
- plugins/logging/version
- core/providers/anthropic.go
- tests/core-providers/cohere_test.go
- tests/core-providers/sgl_test.go
- core/schemas/providers/gemini/chat.go
- core/schemas/providers/openai/models.go
- tests/core-providers/gemini_test.go
- ui/lib/types/schemas.ts
- tests/core-providers/tests.go
- core/schemas/providers/cohere/models.go
- ui/lib/constants/config.ts
- tests/core-providers/openrouter_test.go
- core/schemas/account.go
- core/providers/parasail.go
- core/schemas/providers/mistral/types.go
- core/version
- core/schemas/providers/gemini/models.go
- plugins/otel/version
- tests/core-providers/parasail_test.go
- tests/core-providers/cerebras_test.go
- plugins/telemetry/version
- plugins/logging/changelog.md
🧰 Additional context used
🧬 Code graph analysis (26)
core/providers/mistral.go (5)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (2)
BifrostError(322-331)ListModelsRequest(82-82)core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/provider.go (1)
Provider(197-224)
tests/core-providers/config/account.go (1)
core/utils.go (1)
Ptr(12-14)
tests/core-providers/scenarios/list_models.go (5)
tests/core-providers/config/account.go (1)
ComprehensiveTestConfig(44-60)core/schemas/provider.go (1)
Provider(197-224)core/schemas/models.go (1)
BifrostListModelsRequest(15-26)core/schemas/bifrost.go (2)
ListModelsRequest(82-82)RequestType(79-79)tests/core-providers/scenarios/utils.go (1)
GetErrorMessage(550-574)
core/providers/groq.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (2)
BifrostError(322-331)Groq(43-43)
core/providers/gemini.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (3)
BifrostError(322-331)Gemini(47-47)ListModelsRequest(82-82)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsURL(11-27)core/schemas/providers/gemini/types.go (1)
GeminiListModelsResponse(1272-1275)core/schemas/provider.go (1)
Provider(197-224)
core/providers/ollama.go (3)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (1)
BifrostError(322-331)
core/schemas/providers/azure/types.go (2)
core/schemas/models.go (1)
Model(39-56)core/schemas/providers/mistral/types.go (1)
Capabilities(21-28)
core/schemas/bifrost.go (2)
core/schemas/models.go (1)
BifrostListModelsRequest(15-26)core/schemas/provider.go (1)
Provider(197-224)
core/providers/openrouter.go (4)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostError(322-331)OpenRouter(48-48)RequestType(79-79)ListModelsRequest(82-82)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/models.go (2)
core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (2)
ModelProvider(32-32)BifrostResponseExtraFields(254-262)
core/schemas/providers/bedrock/models.go (3)
core/schemas/providers/bedrock/types.go (1)
BedrockListModelsResponse(456-458)core/schemas/bifrost.go (1)
ModelProvider(32-32)core/schemas/models.go (3)
BifrostListModelsResponse(28-37)Model(39-56)Architecture(58-64)
transports/bifrost-http/integrations/genai.go (4)
transports/bifrost-http/integrations/utils.go (5)
RouteConfigTypeGenAI(200-200)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Gemini(47-47)core/schemas/providers/gemini/models.go (1)
ToGeminiListModelsResponse(55-81)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/models.go (1)
BifrostListModelsResponse(28-37)core/schemas/bifrost.go (1)
ListModelsRequest(82-82)
core/providers/openai.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (5)
BifrostError(322-331)OpenAI(35-35)ListModelsRequest(82-82)ModelProvider(32-32)RequestType(79-79)core/schemas/logger.go (1)
Logger(28-55)core/schemas/providers/openai/types.go (1)
OpenAIListModelsResponse(140-143)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/mistral/models.go (3)
core/schemas/providers/mistral/types.go (1)
MistralListModelsResponse(31-34)core/schemas/models.go (2)
BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (1)
Mistral(41-41)
core/schemas/providers/vertex/models.go (3)
core/schemas/models.go (4)
BifrostListModelsRequest(15-26)DefaultPageSize(11-11)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/providers/vertex/types.go (2)
DefaultPageSize(50-50)VertexListModelsResponse(61-64)core/schemas/bifrost.go (1)
Vertex(40-40)
core/providers/azure.go (5)
core/utils.go (1)
Ptr(12-14)core/schemas/providers/azure/types.go (2)
DefaultAzureAPIVersion(4-4)AzureListModelsResponse(31-34)core/schemas/account.go (2)
Key(8-17)AzureKeyConfig(21-25)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/provider.go (1)
Provider(197-224)
core/providers/bedrock.go (5)
core/schemas/providers/bedrock/types.go (3)
DefaultBedrockRegion(4-4)BedrockError(349-353)BedrockListModelsResponse(456-458)core/schemas/account.go (2)
Key(8-17)BedrockKeyConfig(39-46)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostError(322-331)Bedrock(38-38)ListModelsRequest(82-82)ErrorField(340-347)core/schemas/provider.go (4)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)
core/providers/cohere.go (6)
core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (3)
BifrostError(322-331)Cohere(39-39)ListModelsRequest(82-82)core/schemas/providers/cohere/models.go (1)
ToCohereListModelsURL(10-34)core/schemas/providers/cohere/types.go (2)
CohereError(521-525)CohereListModelsResponse(540-543)core/schemas/provider.go (1)
Provider(197-224)
core/schemas/providers/anthropic/models.go (3)
core/schemas/models.go (4)
BifrostListModelsRequest(15-26)DefaultPageSize(11-11)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/providers/anthropic/types.go (2)
AnthropicListModelsResponse(281-286)AnthropicModel(274-279)core/schemas/bifrost.go (1)
ModelProvider(32-32)
core/bifrost.go (4)
core/schemas/account.go (4)
Account(53-69)Key(8-17)AzureKeyConfig(21-25)BedrockKeyConfig(39-46)core/schemas/provider.go (1)
Provider(197-224)core/schemas/bifrost.go (9)
KeySelector(16-16)ListModelsRequest(82-82)BifrostError(322-331)ErrorField(340-347)BifrostResponseExtraFields(254-262)RequestType(79-79)ModelProvider(32-32)Azure(36-36)Bedrock(38-38)core/schemas/models.go (5)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)DefaultPageSize(11-11)MaxPaginationRequests(13-13)
transports/bifrost-http/integrations/openai.go (5)
core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (5)
Azure(36-36)BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)OpenAI(35-35)transports/bifrost-http/integrations/utils.go (6)
RouteConfigTypeOpenAI(198-198)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)PreRequestCallback(159-159)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/providers/openai/models.go (1)
ToOpenAIListModelsResponse(27-54)
core/schemas/provider.go (3)
core/schemas/bifrost.go (2)
ListModelsRequest(82-82)BifrostError(322-331)core/schemas/account.go (1)
Key(8-17)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)
transports/bifrost-http/handlers/inference.go (6)
transports/bifrost-http/lib/middleware.go (1)
ChainMiddlewares(12-24)transports/bifrost-http/lib/ctx.go (1)
ConvertToBifrostContext(66-244)transports/bifrost-http/handlers/utils.go (3)
SendError(35-44)SendBifrostError(47-62)SendJSON(16-22)core/schemas/models.go (2)
BifrostListModelsResponse(28-37)BifrostListModelsRequest(15-26)core/schemas/bifrost.go (3)
BifrostError(322-331)ModelProvider(32-32)ListModelsRequest(82-82)core/schemas/provider.go (1)
Provider(197-224)
transports/bifrost-http/integrations/anthropic.go (6)
transports/bifrost-http/integrations/utils.go (7)
RouteConfigTypeAnthropic(199-199)RouteConfig(205-222)RequestConverter(114-114)ListModelsResponseConverter(118-118)ErrorConverter(144-144)GenericRouter(227-232)NewGenericRouter(236-243)transports/bifrost-http/lib/config.go (1)
HandlerStore(34-37)core/schemas/models.go (2)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)core/schemas/bifrost.go (4)
BifrostRequest(130-140)ListModelsRequest(82-82)BifrostError(322-331)Anthropic(37-37)core/schemas/providers/anthropic/models.go (1)
ToAnthropicListModelsResponse(74-103)core/schemas/providers/anthropic/chat.go (1)
ToAnthropicChatCompletionError(986-1007)
core/providers/vertex.go (7)
core/schemas/account.go (2)
Key(8-17)VertexKeyConfig(29-33)core/schemas/models.go (3)
BifrostListModelsRequest(15-26)BifrostListModelsResponse(28-37)Model(39-56)core/schemas/bifrost.go (4)
BifrostError(322-331)ErrorField(340-347)ListModelsRequest(82-82)Vertex(40-40)core/schemas/providers/vertex/models.go (1)
ToVertexListModelsURL(10-26)core/schemas/provider.go (5)
ErrProviderRequestTimedOut(22-22)ErrProviderRequest(24-24)ErrProviderResponseUnmarshal(25-25)Provider(197-224)ErrProviderJSONMarshaling(26-26)core/schemas/providers/vertex/types.go (2)
VertexListModelsResponse(61-64)VertexEmbeddingRequest(21-24)core/schemas/embedding.go (1)
BifrostEmbeddingResponse(17-23)
🔇 Additional comments (82)
plugins/jsonparser/version (1)
1-1: The version bump is justified—dependency updates warrant this patch increment.The jsonparser plugin has legitimate changes: its changelog was updated to reflect dependency version updates (core 1.2.13→1.2.14, framework 1.1.15→1.1.16). A patch version bump to 1.3.16 is appropriate for maintenance updates like this. The original review comment was based on incomplete context.
Likely an incorrect or invalid review comment.
framework/version (1)
1-1: LGTM! Version bump aligns with PR changes.The framework version update from 1.1.15 to 1.1.16 is consistent with the ListModels feature addition across the codebase.
plugins/jsonparser/changelog.md (1)
4-4: LGTM! Changelog entry properly reflects version updates.The version references are correctly updated to match the core and framework version bumps in this PR.
plugins/maxim/changelog.md (1)
4-4: LGTM! Changelog properly updated.Version references correctly reflect the core 1.2.14 and framework 1.1.16 updates.
plugins/mocker/changelog.md (1)
4-4: LGTM! Version update is consistent.Changelog entry aligns with the version bumps across the repository.
framework/changelog.md (1)
4-4: LGTM! Framework changelog properly updated.Version reference correctly updated to core 1.2.14.
plugins/telemetry/changelog.md (1)
4-4: LGTM! Telemetry changelog updated correctly.Version references properly updated to core 1.2.14 and framework 1.1.16.
core/schemas/providers/gemini/types.go (1)
1271-1275: LGTM! GeminiListModelsResponse is well-documented.The struct properly represents the Gemini list models API response with clear godoc and appropriate fields for pagination support.
ui/lib/types/config.ts (1)
123-123: LGTM! New field properly integrated.The
list_modelsfield is correctly added to theAllowedRequestsinterface, following the established naming convention and boolean type pattern. This enables UI support for the new ListModels API feature.ui/app/providers/fragments/allowedRequestsFields.tsx (1)
12-22: List Models surfaced first — good consistencyPlacing list_models first aligns UI with new capability and prior guidance.
core/schemas/provider.go (3)
90-100: AllowedRequests gains list_models — placement and naming LGTMField is first and matches JSON schema across UI/config.
109-111: IsOperationAllowed updated for list_models — LGTMMaps the new request type to the new flag.
200-202: Provider interface extended with ListModels — LGTMSurface makes sense and matches provider implementations.
core/schemas/providers/anthropic/types.go (1)
281-286: Nullable pagination IDs handled — LGTMfirst_id/last_id as pointers with omitempty is correct.
core/providers/groq.go (1)
69-82: No issues found; Authorization header is correctly injected by handleOpenAIListModelsRequestThe helper function properly sets the Authorization header at lines 115–117 when key.Value is non-empty. Groq's ListModels delegation is wired correctly.
core/providers/vertex.go (2)
153-283: ListModels flow looks solid.Proper URL construction, OAuth client handling, 401/403 eviction, typed parsing, and ExtraFields set. No local re-pagination—good.
671-799: Embedding helper LGTM.Uses native predict, consistent error handling, usage of pooled auth client, and ExtraFields populated correctly.
transports/bifrost-http/integrations/utils.go (3)
118-118: LGTM!The
ListModelsResponseConvertertype definition is clean and follows the same pattern as other response converters in the codebase.
250-267: LGTM!The validation logic correctly:
- Normalizes the HTTP method upfront
- Validates
GetRequestTypeInstanceand its return value for all routes- Ensures
ListModelsResponseConverteris present for GET routes- Validates
RequestConverterpresenceThis fail-fast approach at startup prevents runtime errors and aligns with past review feedback.
388-407: LGTM!The
ListModelsRequesthandling follows the same pattern as other request types: invoke the client method, apply the optionalPostCallback, validate the response, and convert using the configured converter. The nil-guard forListModelsResponseConverteris handled upstream during route registration.tests/core-providers/ollama_test.go (1)
42-42: LGTM!Enables ListModels testing for the Ollama provider, aligning with the broader ListModels feature rollout across all providers.
tests/core-providers/openai_test.go (1)
58-58: LGTM!Enables ListModels testing for the OpenAI provider, consistent with the ListModels feature addition across the codebase.
core/schemas/bifrost.go (3)
82-82: LGTM!The
ListModelsRequestconstant is correctly positioned first in theRequestTypeconstants list, consistent with the past review feedback to keeplist_modelsfirst in schema definitions.
133-133: LGTM!The
ListModelsRequestfield is correctly positioned first in theBifrostRequeststruct, maintaining consistency with the constant ordering.
143-211: No changes needed. ListModels intentionally uses a separate code path.ListModels is handled through dedicated handlers (
ListModelsRequest()andListAllModels()in core/bifrost.go) that take*BifrostListModelsRequestdirectly, bypassing the generic request processing that usesGetRequestFields(). Additionally,BifrostListModelsRequesthas noModelorFallbacksfields—onlyProvider,PageSize,PageToken, andExtraParams—making these fields semantically inapplicable. The omission is intentional and correct.core/changelog.md (1)
4-5: LGTM!The changelog entries clearly document the addition of the
ListModelsmethod to the Provider interface and provider tracking in Bifrost core.tests/core-providers/bedrock_test.go (1)
48-48: LGTM!Enables ListModels testing for the Bedrock provider, consistent with the feature rollout.
tests/core-providers/azure_test.go (1)
47-47: LGTM!Enables ListModels testing for the Azure provider, aligning with the broader ListModels feature addition.
docs/features/unified-interface.mdx (1)
88-107: LGTM!The provider support matrix has been updated to include the new "Models" column, accurately reflecting ListModels support across all providers. The explanatory note clearly defines what the "Models" operation refers to.
transports/bifrost-http/handlers/inference.go (4)
277-279: LGTM! Route registration looks correct.The GET /v1/models endpoint is properly registered with middleware chaining.
289-304: Handler setup and context conversion are correct.The listModels handler properly extracts the provider parameter and converts the context. The optional provider pattern (empty provider lists all) is well-designed.
305-317: Pagination parameter extraction is well-implemented.The pageSize parsing correctly validates non-negative values and silently ignores invalid input, which is appropriate for optional parameters.
331-345: Request routing and response handling look correct.The conditional logic for ListAllModels vs ListModelsRequest is appropriate, and error handling follows the established patterns.
core/schemas/providers/mistral/models.go (1)
5-27: LGTM! Conversion method is correctly implemented.The method properly:
- Handles nil input
- Preallocates capacity for Data
- Uses
schemas.Ptr()to avoid pointer capture bugs (previously flagged and now fixed)- Prefixes model IDs with the provider key for uniqueness
- Maps all relevant fields from Mistral's response to Bifrost's unified schema
core/providers/mistral.go (1)
70-120: LGTM! ListModels implementation is appropriate for Mistral.The implementation correctly:
- Makes a GET request to /v1/models with Bearer auth
- Parses the response using Mistral-specific schemas
- Converts to Bifrost's unified response format
- Applies client-side pagination (appropriate since Mistral's API doesn't support pagination query parameters, as discussed in past reviews)
- Populates standard ExtraFields metadata
The client-side pagination approach is reasonable for providers without native pagination support.
Based on learnings.
tests/core-providers/config/account.go (3)
40-40: LGTM! ListModels field addition is appropriate.Adding the ListModels boolean to TestScenarios enables comprehensive testing of the new model listing functionality across providers.
183-186: Good: Azure embeddings APIVersion updated to stable release.Changing from
2024-08-01-previewto2024-10-21moves Azure embeddings to a stable API version, which improves reliability for ListModels and other operations.
503-503: LGTM! ListModels enabled across all providers.The test configurations correctly enable ListModels for all providers. As clarified in previous reviews, the generic
/v1/modelsHTTP endpoint dynamically routes to provider implementations based on theproviderquery parameter, so provider-specific route configs are not required for core functionality.Also applies to: 531-531, 559-559, 587-587, 615-615, 643-643, 670-670, 698-698, 726-726, 754-754, 785-785, 813-813
core/providers/ollama.go (1)
71-74: LGTM! Ollama ListModels implementation correctly reuses OpenAI handler.The implementation appropriately delegates to
handleOpenAIListModelsRequestsince Ollama provides an OpenAI-compatible API at/v1/models. This is consistent with other Ollama provider methods that also reuse OpenAI-compatible handlers.transports/bifrost-http/integrations/genai.go (3)
6-6: LGTM! Import addition is necessary.The
strconvimport is needed for the pageSize parsing inextractGeminiListModelsParams.
67-89: LGTM! Gemini ListModels route config is well-structured.The route configuration correctly:
- Maps GET requests to
/genai/v1beta/models- Uses
BifrostListModelsRequestas the request type- Converts responses using
gemini.ToGeminiListModelsResponse- Converts errors using
gemini.ToGeminiError(previously flagged and now fixed)- Delegates parameter extraction to
extractGeminiListModelsParamsThis follows the established pattern for other Gemini routes.
156-177: LGTM! Parameter extraction follows established patterns.The
extractGeminiListModelsParamsfunction correctly:
- Sets the provider to Gemini
- Parses
pageSizefrom query params (using Gemini's parameter name)- Extracts
pageTokenfor pagination- Silently ignores invalid pageSize values (consistent with optional parameter handling)
The error handling is appropriate since invalid values are treated as absent parameters.
core/schemas/providers/bedrock/models.go (1)
5-27: LGTM! Bedrock conversion method is correctly implemented.The method properly:
- Handles nil input
- Preallocates capacity for Data
- Uses
schemas.Ptr()to avoid pointer capture bugs (previously flagged and now fixed)- Prefixes model IDs with the provider key
- Maps Bedrock-specific fields (InputModalities, OutputModalities) into the Architecture field
The design of accepting
providerKeyas a parameter provides flexibility for Bedrock's multiple deployment scenarios.core/schemas/providers/openai/types.go (1)
128-143: LGTM! Model types are well-defined.The new types correctly:
- Define
OpenAIModelwith core fields (ID, Object, OwnedBy, Created)- Use pointer type with
omitemptyfor Created (allows it to be absent in responses)- Include optional GROQ-specific fields (Active, ContextWindow) with clear comments
- Define
OpenAIListModelsResponsewith standard OpenAI list response structureThese types follow OpenAI API conventions and support provider-specific extensions.
transports/bifrost-http/integrations/openai.go (4)
58-59: LGTM! Azure deployment handling correctly integrated.The addition of
BifrostListModelsRequestcase to the Azure endpoint pre-hook correctly sets the provider to Azure for deployment-based list models requests, maintaining consistency with other request types.
327-362: Well-structured route configuration for list models endpoints.The function correctly:
- Exposes three OpenAI-compatible list models endpoints
- Uses GET method for read-only operations
- Applies proper request/response conversion
- Integrates Azure-specific preprocessing via the combined pre-callback
364-389: Combined pre-callback correctly sequences Azure and OpenAI handling.The function properly chains Azure deployment preprocessing (which sets
Provider = Azurefor deployment routes) before defaulting to OpenAI for non-deployment routes. The conditional check at line 380 ensures the default only applies when the provider hasn't already been set by the Azure hook.
394-394: Router integration looks good.List models routes are correctly appended to the existing OpenAI route configurations.
transports/bifrost-http/integrations/anthropic.go (2)
77-103: Route configuration correctly structured.The list models route is properly configured with:
- Appropriate GET method for read-only operation
- Anthropic-specific response and error converters
- Parameter extraction pre-callback for pagination
142-142: Router integration is correct.List models routes are properly appended to the Anthropic router configuration.
core/providers/azure.go (4)
13-13: Good centralization of Azure API version constant.The import of Azure provider schemas enables use of the centralized
DefaultAzureAPIVersionconstant throughout the file, eliminating the previous hardcoded duplicates and preventing future drift.
110-110: Good use of http.MethodPost constant.Using the standard library constant instead of a string literal improves code clarity and maintainability.
147-236: ListModels implementation is well-structured and consistent.The method correctly:
- Validates Azure configuration before making requests
- Uses the resource-level
/openai/modelsendpoint (non-deployment)- Applies centralized API version default
- Handles both Bearer token and api-key authentication
- Measures latency and populates all required ExtraFields
- Applies local pagination for consistent user experience
92-92: Centralized API version usage is consistent throughout.All Azure API version defaults now reference the single constant, ensuring consistency across text completion, streaming, and other operations.
core/providers/bedrock.go (3)
92-92: Good centralization of default region.Using
bedrock.DefaultBedrockRegioninstead of hardcoded values provides a single source of truth and makes region configuration more maintainable.
233-233: Good use of http method constant.Replacing the string literal with
http.MethodPostimproves code clarity and follows Go best practices.
348-482: ListModels implementation is correct and well-documented.The method properly:
- Validates authorization and configuration
- Uses the correct Bedrock service endpoint (
/foundation-models)- Handles both API key and IAM authentication patterns
- Applies local pagination (necessary since AWS Bedrock doesn't support server-side pagination for this endpoint)
- Measures latency and populates all required metadata
Based on learnings
core/schemas/providers/azure/types.go (2)
3-4: Centralized API version constant is well-placed.Defining
DefaultAzureAPIVersionin the Azure types package provides a logical home for the constant and enables its use across Azure provider implementations.
6-34: Azure model types are well-defined.The type definitions correctly model Azure's list models response structure:
AzureModelCapabilitiesuses booleans for feature flagsAzureModelDeprecationuses optional int64 timestampsAzureModelcaptures all relevant metadata fields- JSON tags properly map to Azure's response format
core/providers/cohere.go (3)
236-236: Good use of HTTP method constants throughout.Replacing string literals with
http.MethodPostandhttp.MethodGetconstants improves code clarity and consistency across the file.
98-158: ListModels implementation is well-structured.The method correctly:
- Validates authorization before proceeding
- Delegates URL construction to a centralized helper (good separation of concerns)
- Uses GET method for read-only operation
- Handles errors with provider-specific formatting
- Converts Cohere's response to Bifrost's standardized format
- Populates all required metadata fields
196-209: Response handling is consistent and complete.The ChatCompletion method correctly populates all response fields including Model, Provider, ModelRequested, RequestType, Latency, and optional RawResponse.
core/schemas/providers/anthropic/models.go (2)
11-43: URL construction is robust and safe.The function correctly:
- Applies sensible defaults for page size
- Uses
url.Valuesfor proper query parameter encoding- Supports both forward and backward pagination via
after_id/before_id- Handles
PageTokenappropriately when explicit pagination params aren't provided
74-103: Reverse conversion is well-implemented.The function correctly converts Bifrost's standardized response back to Anthropic's format, properly handling all optional fields and maintaining data integrity.
tests/core-providers/scenarios/list_models.go (2)
15-91: Test implementation is comprehensive and well-structured.The test helper correctly:
- Checks provider support before running
- Creates minimal required request
- Validates response structure and content
- Verifies model integrity (non-empty IDs)
- Validates all ExtraFields (provider, request type, latency)
- Includes sensible latency thresholds with warnings
- Supports parallel execution control via environment variable
93-161: Pagination test is thorough and well-designed.The test helper effectively validates:
- Page size limits are respected
- Next page tokens are generated when appropriate
- Multiple pages can be fetched successfully
- Different pages return distinct models
- Single-page scenarios are handled gracefully
core/providers/openai.go (2)
80-89: LGTM!The ListModels implementation follows the established pattern for provider operations, with proper permission checks and clean delegation to the handler function.
91-154: LGTM!The handler implementation is well-structured with proper error handling, response conversion, pagination support, and metadata population. The use of
http.MethodGetconstant (line 112) addresses the previous review feedback.core/bifrost.go (5)
36-38: LGTM!The providers field is properly initialized as an atomic pointer to a slice, following the same pattern as the plugins field. This enables thread-safe access to the provider list.
Also applies to: 96-97
211-274: LGTM!The ListModelsRequest implementation is thorough with proper validation, correct base provider type handling for custom providers, and appropriate key selection using the requestType parameter. All previous review feedback has been addressed.
276-372: LGTM!The ListAllModels implementation properly aggregates models across providers with comprehensive error handling, pagination safety limits, and sorted results. All previous review feedback (nil checks, pagination limits, error handling) has been addressed.
1211-1224: LGTM!The compare-and-swap loop correctly prevents lost updates when multiple providers are prepared concurrently. This addresses the previous review concern about race conditions during provider registration.
1278-1293: LGTM!The provider lookup and key selection logic correctly handles the ListModelsRequest case by skipping model-specific validation while still ensuring keys have valid values. The requestType parameter addition and the split validation logic address all previous review feedback.
Also applies to: 2157-2157, 2177-2206
docs/apis/openapi.json (9)
291-350: Well-structured /v1/models endpoint definition.The endpoint is properly defined with:
- Clear description clarifying provider optionality
- All three pagination parameters documented (provider, page_size, page_token)
- Proper reference to ModelProvider enum for type safety
- Correct response codes (200, 400, 500)
No issues identified in this segment.
3262-3273: RequestType enum properly includes list_models.The new "list_models" value is correctly positioned in the RequestType enum. Clients using this schema for validation will properly recognize responses from the /v1/models endpoint.
3982-3986: Latency units correctly documented as milliseconds.BifrostResponseExtraFields.latency description correctly specifies milliseconds, matching the implementation across provider ListModels methods.
4625-4629: LogEntry latency description consistent.LogEntry.latency correctly documents milliseconds, matching BifrostResponseExtraFields and the provider implementations.
6560-6599: ListModelsResponse structure well-designed.The response schema properly includes:
- Required "data" array of ModelInfo objects
- Optional pagination via next_page_token
- extra_fields with provider, request_type, latency, and raw_response
- Clear descriptions explaining field purposes
No issues identified.
6600-6692: ModelInfo schema comprehensively designed for provider-agnostic model metadata.All optional fields are correctly marked with
nullable: true, allowing Bifrost to omit fields not returned by specific providers. The note at line 6602 clarifies this behavior:"Only fields returned by the provider API are included; unset fields are omitted from the JSON response."
This aligns with the endpoint description and ensures clean client-side parsing without null-checking boilerplate.
6694-6730: ModelArchitecture modality enums are well-constrained.Input/output modalities at lines 6711 and 6720 properly restrict values to valid types: file, image, text, audio, video. Examples are clear and realistic.
6732-6776: ModelPricing uses string type for decimal precision—design confirmed.Pricing fields remain
stringtype (e.g., lines 6737-6744) to preserve decimal precision for fractional token costs. Example values like "0.0000025" demonstrate the rationale. This design choice matches OpenRouter's pricing representation and avoids floating-point rounding issues.
6778-6794: ModelTopProvider token limits correctly use integer type.context_length and max_completion_tokens are properly typed as
integer, notnumber, reflecting that token counts are never fractional.
Merge activity
|
2 participants
Summary
Add support for listing models from various providers via a new
/v1/modelsendpoint, enabling applications to discover available models and their capabilities. Closes #532Changes
ListModelsmethod to the Provider interface/v1/modelsthat accepts aproviderquery parameterType of change
Affected areas
How to test
Breaking changes
Related issues
Enhances model discovery capabilities for applications using Bifrost.
Security considerations
The endpoint requires the same authentication as other Bifrost endpoints and only exposes information about available models, not any sensitive data.
Checklist