feat: add Cerebras provider support and update provider count to 12+ - #267
Conversation
Summary by CodeRabbit
WalkthroughAdds Cerebras as a new AI provider across core, transports, UI, tests, and docs; implements a CerebrasProvider (text/chat/streaming using OpenAI-compatible endpoints), registers it in factories and integrations, updates UI types/icons/labels, adds tests, and updates provider counts to 12+. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant Bifrost
participant ProviderFactory
participant CerebrasProvider
participant CerebrasAPI
Client->>Bifrost: Request (model=cerebras/..., text|chat)
Bifrost->>ProviderFactory: createProviderFromProviderKey(Cerebras)
ProviderFactory-->>Bifrost: CerebrasProvider
Bifrost->>CerebrasProvider: TextCompletion | ChatCompletion
CerebrasProvider->>CerebrasAPI: POST /v1/completions or /v1/chat/completions
CerebrasAPI-->>CerebrasProvider: JSON response (200) / error
CerebrasProvider-->>Bifrost: BifrostResponse
Bifrost-->>Client: Response
sequenceDiagram
participant Client
participant Bifrost
participant CerebrasProvider
participant CerebrasAPI
Client->>Bifrost: ChatCompletionStream (model=cerebras/...)
Bifrost->>CerebrasProvider: Stream request
CerebrasProvider->>CerebrasAPI: SSE /v1/chat/completions?stream=true
CerebrasAPI-->>CerebrasProvider: SSE chunks
CerebrasProvider-->>Bifrost: BifrostStream events
Bifrost-->>Client: Streamed tokens/events
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
✨ Finishing Touches
🧪 Generate unit tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
This stack of pull requests is managed by Graphite. Learn more about stacking. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 16
🔭 Outside diff range comments (2)
ui/README.md (1)
13-13: Update provider count to 12+ for consistency.The headline still says “Configure 8+ AI providers,” but the repo now advertises 12+. Please bump this to avoid confusion.
Apply this minimal change:
- - **⚙️ Provider Management** - Configure 8+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.) + - **⚙️ Provider Management** - Configure 12+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.)transports/bifrost-http/integrations/utils.go (1)
676-727: Clarify provider detection for unprefixed ‘llama-*’ models (optional).Many providers use ‘llama-*’ names. Since GetProviderFromModel defaults to OpenAI for unknown models, “llama-3.3-70b” without a prefix will not resolve to Cerebras. Consider one of:
- Documenting that “cerebras/{model}” is the recommended form to disambiguate.
- Or (with caution) adding a Cerebras detector and placing it appropriately to avoid collisions with Bedrock/others.
No action required if you intend prefix-only detection.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/core-providers/go.sumis excluded by!**/*.sum
📒 Files selected for processing (22)
README.md(3 hunks)ci/npx/package.json(1 hunks)core/bifrost.go(1 hunks)core/providers/cerebras.go(1 hunks)core/schemas/bifrost.go(1 hunks)docs/quickstart/go-package.md(1 hunks)docs/usage/go-package/README.md(1 hunks)docs/usage/go-package/account.md(1 hunks)docs/usage/go-package/schemas.md(1 hunks)docs/usage/providers.md(3 hunks)tests/core-providers/README.md(1 hunks)tests/core-providers/cerebras_test.go(1 hunks)tests/core-providers/config/account.go(3 hunks)tests/core-providers/go.mod(3 hunks)transports/README.md(2 hunks)transports/bifrost-http/integrations/utils.go(1 hunks)transports/go.mod(1 hunks)ui/README.md(1 hunks)ui/app/layout.tsx(1 hunks)ui/lib/constants/icons.tsx(2 hunks)ui/lib/constants/logs.ts(2 hunks)ui/lib/types/config.ts(1 hunks)
👮 Files not reviewed due to content moderation or server errors (1)
- ui/lib/constants/icons.tsx
🧰 Additional context used
🧠 Learnings (12)
📓 Common learnings
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:358-388
Timestamp: 2025-06-04T05:37:59.699Z
Learning: User Pratham-Mishra04 prefers not to extract small code duplications (around 2 lines) into helper functions, considering the overhead not worth it for such minor repetition.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#102
File: README.md:62-66
Timestamp: 2025-06-19T17:03:03.639Z
Learning: Pratham-Mishra04 prefers using the implicit 'latest' tag for the maximhq/bifrost Docker image rather than pinning to specific versions.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#143
File: core/mcp.go:155-196
Timestamp: 2025-07-08T15:33:47.698Z
Learning: Pratham-Mishra04 prefers not to add explanatory comments for obvious code patterns, such as the unlock/lock strategy around network I/O operations, considering them self-explanatory to experienced developers.
📚 Learning: 2025-06-16T04:55:11.886Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/automatic_function_calling.go:22-22
Timestamp: 2025-06-16T04:55:11.886Z
Learning: In the Bifrost test suite (tests/core-providers), parallel tests using t.Parallel() are not being implemented currently. The team plans to add parallel test execution in future enhancements.
Applied to files:
tests/core-providers/cerebras_test.go
📚 Learning: 2025-06-16T04:29:53.409Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/openai_test.go:1-2
Timestamp: 2025-06-16T04:29:53.409Z
Learning: In the Bifrost project, the user prefers to use `package main` for test files in the tests/core-providers directory rather than more descriptive package names like `coreproviders_test`.
Applied to files:
tests/core-providers/cerebras_test.gotests/core-providers/go.mod
📚 Learning: 2025-06-15T14:34:29.401Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: core/providers/ollama.go:59-61
Timestamp: 2025-06-15T14:34:29.401Z
Learning: The `CheckAndSetDefaults()` method on `*schemas.ProviderConfig` in Go does not return any error - it has a void return type and only sets default values on the configuration struct.
Applied to files:
tests/core-providers/config/account.go
📚 Learning: 2025-06-16T03:54:48.005Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/go.mod:38-38
Timestamp: 2025-06-16T03:54:48.005Z
Learning: The `core-providers-test` module in `tests/core-providers/` is an internal testing module that will never be consumed as a dependency by external projects, so the replace directive pointing to `../../core` is acceptable for local development and testing purposes.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-20T16:21:18.912Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#103
File: .github/workflows/transport-dependency-update.yml:53-75
Timestamp: 2025-06-20T16:21:18.912Z
Learning: In the bifrost repository's transport dependency update workflow, when updating the core dependency to a new version using `go get`, the go.mod and go.sum files will always change in normal operation, making the safety check for changes more of a defensive programming practice rather than handling a common scenario.
Applied to files:
tests/core-providers/go.modtransports/go.mod
📚 Learning: 2025-06-16T04:12:05.427Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/complete_end_to_end.go:39-41
Timestamp: 2025-06-16T04:12:05.427Z
Learning: In the Bifrost system, error returns are of type `BifrostError` rather than the standard Go `error` interface. Therefore, use `require.Nilf(t, err, ...)` instead of `require.NoError(t, err)` when checking for errors in Bifrost function calls.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:13:55.437Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/end_to_end_tool_calling.go:43-45
Timestamp: 2025-06-16T04:13:55.437Z
Learning: In the Bifrost codebase, errors returned from client methods like ChatCompletionRequest are of type BifrostError, not the standard error interface. For testing these errors, use require.Nilf instead of require.NoErrorf since BifrostError doesn't work with the standard error assertion methods.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-27T17:07:39.462Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#135
File: docs/core-package.md:105-116
Timestamp: 2025-06-27T17:07:39.462Z
Learning: In Go, when importing a package, the identifier used to access it is determined by the `package` declaration in the Go source files, not the directory name. For the Bifrost project, the core directory files declare `package bifrost`, so importing `"github.com/maximhq/bifrost/core"` automatically makes it accessible as `bifrost.Init`, `bifrost.Ptr`, etc., without needing an explicit alias.
Applied to files:
tests/core-providers/go.modtransports/go.mod
📚 Learning: 2025-07-17T08:56:59.907Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#169
File: docs/usage/http-transport/openapi.json:1352-1358
Timestamp: 2025-07-17T08:56:59.907Z
Learning: In the Bifrost project, the fallback format has been updated from object structure {"provider": "...", "model": "..."} to a simpler string format "provider/model" (e.g., "anthropic/claude-3-sonnet-20240229"). The current OpenAPI schema correctly reflects this new format.
Applied to files:
README.md
📚 Learning: 2025-06-15T16:05:13.489Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: transports/bifrost-http/main.go:2-2
Timestamp: 2025-06-15T16:05:13.489Z
Learning: For the Bifrost project, HTTP transport integration routers for new providers (like Mistral and Ollama) are implemented in separate PRs from the core provider support, following a focused PR strategy.
Applied to files:
README.md
📚 Learning: 2025-08-05T20:43:59.593Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#214
File: core/providers/azure.go:49-50
Timestamp: 2025-08-05T20:43:59.593Z
Learning: In core/providers/azure.go, the azureTextCompletionResponsePool should use AzureTextResponse type, not schemas.BifrostResponse, to maintain consistency with the acquireAzureTextResponse() and releaseAzureTextResponse() functions that work with *AzureTextResponse objects.
Applied to files:
core/providers/cerebras.go
🧬 Code Graph Analysis (7)
core/schemas/bifrost.go (1)
ui/lib/types/config.ts (1)
ModelProvider(4-16)
tests/core-providers/cerebras_test.go (3)
tests/core-providers/config/setup.go (1)
SetupTest(51-59)tests/core-providers/config/account.go (2)
ComprehensiveTestConfig(38-46)TestScenarios(17-35)core/schemas/bifrost.go (1)
Cerebras(52-52)
tests/core-providers/config/account.go (3)
core/schemas/bifrost.go (1)
Cerebras(52-52)core/schemas/account.go (1)
Key(8-16)core/schemas/provider.go (5)
ProviderConfig(87-94)NetworkConfig(32-40)DefaultNetworkConfig(43-48)ConcurrencyAndBufferSize(51-54)DefaultConcurrencyAndBufferSize(57-60)
core/bifrost.go (2)
core/schemas/bifrost.go (1)
Cerebras(52-52)core/providers/cerebras.go (1)
NewCerebrasProvider(72-107)
transports/bifrost-http/integrations/utils.go (1)
core/schemas/bifrost.go (1)
Cerebras(52-52)
ui/lib/types/config.ts (1)
core/schemas/bifrost.go (1)
ModelProvider(38-38)
core/providers/cerebras.go (5)
core/providers/azure.go (1)
AzureTextResponse(19-32)core/schemas/bifrost.go (12)
BifrostResponse(376-389)ModelProvider(38-38)Cerebras(52-52)ModelParameters(166-183)BifrostError(681-690)BifrostResponseChoice(549-555)BifrostNonStreamResponseChoice(558-562)BifrostMessage(287-295)LogProbs(460-464)BifrostResponseExtraFields(654-662)Azure(42-42)BifrostStream(670-673)ui/lib/types/config.ts (6)
NetworkConfig(54-61)ProviderConfig(81-87)ConcurrencyAndBufferSize(64-67)ProxyConfig(73-78)ModelProvider(4-16)Key(43-51)core/schemas/provider.go (3)
DefaultRequestTimeoutInSeconds(14-14)ErrProviderJSONMarshaling(24-24)PostHookRunner(129-129)ui/lib/constants/logs.ts (1)
Provider(76-76)
🪛 LanguageTool
docs/usage/go-package/README.md
[grammar] ~218-~218: There might be a problem here.
Context: ...n](../providers.md) - All 12+ providers setup ### Development - Logging - Debug visibilit...
(QB_NEW_EN_MERGED_MATCH)
tests/core-providers/README.md
[grammar] ~18-~18: Add a comma
Context: ...** - OSS models - Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Te...
(QB_NEW_EN_OTHER_ERROR_IDS_22)
[grammar] ~18-~18: Use correct spacing
Context: ... Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Tests ### Development wit...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
README.md
[grammar] ~13-~13: Use correct spacing
Context: ...d zero-downtime deployments in under 30 seconds. 🚀 Just launched: Native MCP (Model ...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~78-~78: There might be a mistake here.
Context: ... to build AI applications that never go down](#the-fastest-way-to-build-ai-applications-that-never-go-down) - [⚡ Quickstart (30 seconds)](#-quickstart-...
(QB_NEW_EN_OTHER)
🔇 Additional comments (26)
docs/usage/go-package/account.md (1)
582-582: LGTM — provider count text updated to 12+.Change is consistent with the PR objective and broader updates.
core/schemas/bifrost.go (1)
52-52: Approved: Cerebras provider integration validated across core, transport, UI, and testsAll cross-references correctly include the new
cerebrasprovider:
- core/bifrost.go (lines 690–691): adds
case schemas.Cerebras → NewCerebrasProvider- transports/bifrost-http/integrations/utils.go (line 645):
schemas.Cerebras: true- ui/lib/types/config.ts (line 16): includes
'cerebras'- ui/lib/constants/logs.ts (lines 13, 42): lists and labels
cerebras- ui/lib/constants/icons.tsx (line 128): provides a Cerebras icon
- tests/core-providers/cerebras_test.go: covers provider functionality
No further changes needed.
ci/npx/package.json (1)
4-4: Description update to “12+ providers” looks correctMatches the PR objective and keeps CLI metadata consistent with docs/UI changes.
transports/go.mod (1)
20-21: Remove localreplacein transports/go.modThe relative replace in transports/go.mod
-replace github.com/maximhq/bifrost/core => ../corewill break downstream consumers (including CI) that expect to fetch the published core module. Instead:
• transports/go.mod: remove the
replace github.com/maximhq/bifrost/core => ../coredirective
• Introduce a root-level go.work file for local development, e.g.go 1.20 use ( ./core ./transports )• Keep local
replacedirectives only in internal test modules (e.g. tests/core-providers/go.mod)
• Verify that the required core version (v1.1.17) includes the Cerebras provider; if not, bump the version in transports/go.modtransports/README.md (2)
55-55: Provider list correctly adds CerebrasGood to see Cerebras included alongside OSS backends. No content issues spotted here.
295-295: No stale “10+ providers” references found
Ranrg -n --fixed-strings "10+ providers"across the repo; no matches remain.README.md (2)
78-78: TOC anchor checkThe new TOC entry anchor looks correct for the header added above. If you observe broken in-page navigation, consider re-running the README renderer preview to confirm the exact anchor GitHub generates.
251-251: All “10+ providers” references removed — global consistency confirmed
No occurrences of “10+ providers” found across the codebase. 🤝docs/usage/go-package/schemas.md (1)
804-807: Fix enum name in example: use schemas.SGL (not schemas.SGLang).The codebase appears to define the SGL provider as schemas.SGL (see transports/bifrost-http/integrations/utils.go usage), so this snippet likely won’t compile as written.
Apply this diff if the enum is indeed SGL:
providers := []schemas.ModelProvider{ schemas.OpenAI, // GPT models schemas.Anthropic, // Claude models schemas.Azure, // Azure OpenAI schemas.Bedrock, // AWS Bedrock schemas.Vertex, // Google Vertex AI schemas.Cohere, // Cohere models schemas.Mistral, // Mistral models schemas.Ollama, // Local Ollama schemas.Groq, // Groq models - schemas.Parasail, // Parasail models - schemas.SGLang, // SGLang models + schemas.Parasail, // Parasail models + schemas.SGL, // SGLang models schemas.Cerebras, // Cerebras models }ui/README.md (1)
84-84: Cerebras addition in Supported Providers — LGTM.transports/bifrost-http/integrations/utils.go (1)
645-646: LGTM: adding Cerebras to ValidProviders enables “cerebras/{model}” parsing.This ensures ParseModelString cleanly recognizes “cerebras/...”. No other change needed here.
tests/core-providers/go.mod (2)
10-11: Replace directive for local core — LGTM.Consistent with the internal testing module pattern and past preferences; acceptable for local development.
36-49: No direct usage of new indirect dependencies
A search in tests/core-providers found only go.mod/go.sum entries for zerolog, go-colorable and go-isatty—no code references. These remain purely transitive requirements. No action needed;go mod tidywill prune any truly unused deps.docs/usage/providers.md (3)
18-21: Cerebras added to Supported Providers — LGTM.Listing Llama 3.3 70B here aligns with the PR’s scope.
475-486: Double-check feature matrix entries for OpenAI and Cerebras.
- OpenAI “Text Completion” is marked ❌. Confirm if Bifrost no longer supports non-chat completions for OpenAI; if supported via core, this should be ✅.
- Cerebras is marked ✅ for Function Calling and Vision. Ensure the new CerebrasProvider actually implements tool/function calling and vision to avoid overpromising in docs.
If the intention is “chat-only” for OpenAI and “no tools/vision” for Cerebras, please adjust the matrix accordingly.
502-503: Audio features set to N/A for Cerebras — LGTM.Matches the stated scope (no speech/transcription support).
ui/lib/types/config.ts (2)
4-16: Cerebras added to ModelProvider union — LGTMThe new 'cerebras' entry is correctly added to the union and aligns with the backend constant name per the PR summary. No issues spotted.
4-16: Ignore TS–GoModelProviderfull-sync requirementThe frontend’s
ModelProviderunion is intentionally scoped to user-facing providers, not the full set of Go schema values. We’ve verified:• TS
ModelProviderincludes exactly the interactive providers:
- anthropic, azure, bedrock, cerebras, cohere, groq, mistral, ollama, openai, parasail, sgl, vertex
• All of those appear in the UI constants—icons (ui/lib/constants/icons.tsx) and labels (ui/lib/constants/logs.ts)—including the newly addedcerebras.
• The additional Go-only values (any,auto,function, etc.) are internal and shouldn’t be exposed in the UI union.No further changes are needed here.
Likely an incorrect or invalid review comment.
ui/lib/constants/logs.ts (2)
1-14: Cerebras added to provider list — looks good.Provider union type will auto-include 'cerebras' via typeof inference. No issues spotted.
42-42: Label mapping added — correct and consistent.The new label aligns with other providers.
core/bifrost.go (1)
690-691: Factory wiring for Cerebras — correct signature and error propagation.
NewCerebrasProviderreturns(schemas.Provider, error), so returning it directly matches the pattern used for Bedrock/Azure/etc.tests/core-providers/config/account.go (3)
73-74: Include Cerebras in configured providers — OK.Keeps test account coverage in sync with the new provider.
172-179: API key wiring for Cerebras — OK.Env var name matches docs (
CEREBRAS_API_KEY). Models left empty to mean “all” is consistent with existing pattern.
287-291: Default config for Cerebras — OK.Using
DefaultNetworkConfigandDefaultConcurrencyAndBufferSizeis fine. BaseURL defaults are handled in the provider constructor.tests/core-providers/cerebras_test.go (1)
21-23: No change needed for TextModelThe official Cerebras Inference API lists the Llama 3.1 8B model ID as
llama3.1-8b(no hyphen before the version), matching the existing code. No update to theTextModelstring is required.• Location: tests/core-providers/cerebras_test.go, lines 21–23
Likely an incorrect or invalid review comment.
core/providers/cerebras.go (1)
334-348: Unsupported operations — OK.Embedding, speech, and transcription correctly return unsupported errors for Cerebras.
e8185ed to
3e7b965
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🔭 Outside diff range comments (1)
core/providers/groq.go (1)
188-210: Optional: Add keep-alive header for SSE stabilityHeaders for SSE look correct. To improve robustness with some proxies/servers, consider explicitly setting Connection: keep-alive.
headers := map[string]string{ "Content-Type": "application/json", "Accept": "text/event-stream", "Cache-Control": "no-cache", + "Connection": "keep-alive", } headers["Authorization"] = "Bearer " + key.ValueIf handleOpenAIStreaming or transport layer already forces this, feel free to skip.
♻️ Duplicate comments (9)
docs/quickstart/go-package.md (1)
223-223: Fix broken Providers link path (point to usage/providers.md).This link resolves to a non-existent path from this file. It should point to ../usage/providers.md.
Apply this diff:
-| **Add all 12+ providers** | [🔗 Providers](../providers.md) | 5 min | +| **Add all 12+ providers** | [🔗 Providers](../usage/providers.md) | 5 min |Run to verify the correct target exists and the old path does not:
#!/bin/bash set -euo pipefail echo "Looking for providers.md files:" fd -a "providers.md" echo echo "Check that docs/usage/providers.md exists (expected):" test -f docs/usage/providers.md && echo "OK: docs/usage/providers.md found" echo echo "Check that docs/providers.md does NOT exist (expected missing):" if test -f docs/providers.md; then echo "Unexpected: docs/providers.md exists (re-check link choice)"; exit 1 else echo "OK: docs/providers.md not found" fidocs/usage/go-package/README.md (1)
218-218: Grammar polish: “All 12+ providers setup” → clearer phrasing.Rephrase for readability and correctness.
Apply this diff:
- - [Provider Configuration](../providers.md) - All 12+ providers setup + - [Provider Configuration](../providers.md) - Configuration for all 12+ providersui/app/layout.tsx (1)
24-24: Tighten metadata description and capitalize “MCP”.Crisper copy improves SEO/snippet quality; also fix “mcp” capitalization.
Apply this diff:
- 'Production-ready fastest LLM gateway that connects to 12+ providers through a single API. Get automatic failover, load balancing, mcp support and zero-downtime deployments.', + 'The fastest, production-ready LLM gateway connecting to 12+ providers through a single API. Automatic failover, load balancing, MCP support, and zero-downtime deployments.',tests/core-providers/README.md (2)
18-18: Grammar nit: add serial comma in Cerebras descriptionUse the Oxford comma for consistency with other items.
- - **Cerebras** - Llama, Qwen and GPT-OSS models + - **Cerebras** - Llama, Qwen, and GPT-OSS models
15-18: Add Cerebras env var to Prerequisites sectionYou added Cerebras to Supported Providers; document its env var to avoid tests being skipped.
Add under the “Prerequisites” env block:
# Cerebras export CEREBRAS_API_KEY="your-cerebras-key"I can push a patch if you prefer.
ui/lib/types/config.ts (1)
4-16: Optional: derive ModelProvider from a single source-of-truth arrayTo reduce drift across UI lists (labels/icons/selectors), you can export a const array and derive the union from it. Given your prior preference to avoid small refactors, treat this as optional.
Apply this diff to replace the manual union:
-export type ModelProvider = - | 'openai' - | 'azure' - | 'anthropic' - | 'bedrock' - | 'cohere' - | 'vertex' - | 'mistral' - | 'ollama' - | 'groq' - | 'parasail' - | 'sgl' - | 'cerebras' +export const MODEL_PROVIDERS = [ + 'openai', + 'azure', + 'anthropic', + 'bedrock', + 'cohere', + 'vertex', + 'mistral', + 'ollama', + 'groq', + 'parasail', + 'sgl', + 'cerebras', +] as const +export type ModelProvider = (typeof MODEL_PROVIDERS)[number]core/providers/cerebras.go (3)
25-30: Remove pooling for returned BifrostResponse objects.Pooling
*schemas.BifrostResponseobjects that are returned to callers creates use-after-free risks. The object gets returned to the pool while still being accessed by callers.
86-90: Remove pre-warm code for chat response pool.Since the chat response pool is being removed, this pre-warming code is now unnecessary and should be deleted.
269-291: Fix use-after-release: allocate fresh response for return.You're acquiring a pooled
BifrostResponse, deferring its release, and then returning the same instance. This creates a use-after-free risk where the object is returned to the pool while callers are still accessing it.- // Pre-allocate response structs from pools - response := acquireCerebrasChatResponse() - defer releaseCerebrasChatResponse(response) - - // Use enhanced response handler with pre-allocated response - rawResponse, bifrostErr := handleProviderResponse(responseBody, response, provider.sendBackRawResponse) + // Parse into a fresh response object (avoid pooling for returned objects) + response := &schemas.BifrostResponse{} + rawResponse, bifrostErr := handleProviderResponse(responseBody, response, provider.sendBackRawResponse) if bifrostErr != nil { return nil, bifrostErr }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/core-providers/go.sumis excluded by!**/*.sum
📒 Files selected for processing (23)
README.md(3 hunks)ci/npx/package.json(1 hunks)core/bifrost.go(1 hunks)core/providers/cerebras.go(1 hunks)core/providers/groq.go(1 hunks)core/schemas/bifrost.go(1 hunks)docs/quickstart/go-package.md(1 hunks)docs/usage/go-package/README.md(1 hunks)docs/usage/go-package/account.md(1 hunks)docs/usage/go-package/schemas.md(1 hunks)docs/usage/providers.md(3 hunks)tests/core-providers/README.md(1 hunks)tests/core-providers/cerebras_test.go(1 hunks)tests/core-providers/config/account.go(3 hunks)tests/core-providers/go.mod(3 hunks)transports/README.md(2 hunks)transports/bifrost-http/integrations/utils.go(1 hunks)transports/go.mod(1 hunks)ui/README.md(1 hunks)ui/app/layout.tsx(1 hunks)ui/lib/constants/icons.tsx(2 hunks)ui/lib/constants/logs.ts(2 hunks)ui/lib/types/config.ts(1 hunks)
👮 Files not reviewed due to content moderation or server errors (1)
- ui/lib/constants/icons.tsx
🧰 Additional context used
🧠 Learnings (16)
📓 Common learnings
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:358-388
Timestamp: 2025-06-04T05:37:59.699Z
Learning: User Pratham-Mishra04 prefers not to extract small code duplications (around 2 lines) into helper functions, considering the overhead not worth it for such minor repetition.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#102
File: README.md:62-66
Timestamp: 2025-06-19T17:03:03.639Z
Learning: Pratham-Mishra04 prefers using the implicit 'latest' tag for the maximhq/bifrost Docker image rather than pinning to specific versions.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#143
File: core/mcp.go:155-196
Timestamp: 2025-07-08T15:33:47.698Z
Learning: Pratham-Mishra04 prefers not to add explanatory comments for obvious code patterns, such as the unlock/lock strategy around network I/O operations, considering them self-explanatory to experienced developers.
📚 Learning: 2025-08-12T20:32:24.408Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: tests/core-providers/cerebras_test.go:1-1
Timestamp: 2025-08-12T20:32:24.408Z
Learning: In the Bifrost project, test files in tests/core-providers directory should use `package tests` rather than `package main`, as confirmed by Pratham-Mishra04 in PR #267 for tests/core-providers/cerebras_test.go.
Applied to files:
tests/core-providers/cerebras_test.gotests/core-providers/go.mod
📚 Learning: 2025-06-16T04:55:11.886Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/automatic_function_calling.go:22-22
Timestamp: 2025-06-16T04:55:11.886Z
Learning: In the Bifrost test suite (tests/core-providers), parallel tests using t.Parallel() are not being implemented currently. The team plans to add parallel test execution in future enhancements.
Applied to files:
tests/core-providers/cerebras_test.go
📚 Learning: 2025-08-12T20:27:20.757Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:18-23
Timestamp: 2025-08-12T20:27:20.757Z
Learning: In Bifrost providers, it's intentional to reuse compatible response types across different providers (e.g., using AzureTextResponse in Cerebras provider) when they follow the same struct, rather than creating provider-specific aliases. This avoids unnecessary type duplication for identical API response structures.
Applied to files:
core/bifrost.goREADME.mdcore/providers/cerebras.go
📚 Learning: 2025-06-15T14:34:29.401Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: core/providers/ollama.go:59-61
Timestamp: 2025-06-15T14:34:29.401Z
Learning: The `CheckAndSetDefaults()` method on `*schemas.ProviderConfig` in Go does not return any error - it has a void return type and only sets default values on the configuration struct.
Applied to files:
tests/core-providers/config/account.go
📚 Learning: 2025-06-20T16:21:18.912Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#103
File: .github/workflows/transport-dependency-update.yml:53-75
Timestamp: 2025-06-20T16:21:18.912Z
Learning: In the bifrost repository's transport dependency update workflow, when updating the core dependency to a new version using `go get`, the go.mod and go.sum files will always change in normal operation, making the safety check for changes more of a defensive programming practice rather than handling a common scenario.
Applied to files:
transports/go.modtests/core-providers/go.mod
📚 Learning: 2025-06-27T17:07:39.462Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#135
File: docs/core-package.md:105-116
Timestamp: 2025-06-27T17:07:39.462Z
Learning: In Go, when importing a package, the identifier used to access it is determined by the `package` declaration in the Go source files, not the directory name. For the Bifrost project, the core directory files declare `package bifrost`, so importing `"github.com/maximhq/bifrost/core"` automatically makes it accessible as `bifrost.Init`, `bifrost.Ptr`, etc., without needing an explicit alias.
Applied to files:
transports/go.modtests/core-providers/go.mod
📚 Learning: 2025-06-15T16:05:13.489Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: transports/bifrost-http/main.go:2-2
Timestamp: 2025-06-15T16:05:13.489Z
Learning: For the Bifrost project, HTTP transport integration routers for new providers (like Mistral and Ollama) are implemented in separate PRs from the core provider support, following a focused PR strategy.
Applied to files:
README.md
📚 Learning: 2025-06-16T03:54:48.005Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/go.mod:38-38
Timestamp: 2025-06-16T03:54:48.005Z
Learning: The `core-providers-test` module in `tests/core-providers/` is an internal testing module that will never be consumed as a dependency by external projects, so the replace directive pointing to `../../core` is acceptable for local development and testing purposes.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:12:05.427Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/complete_end_to_end.go:39-41
Timestamp: 2025-06-16T04:12:05.427Z
Learning: In the Bifrost system, error returns are of type `BifrostError` rather than the standard Go `error` interface. Therefore, use `require.Nilf(t, err, ...)` instead of `require.NoError(t, err)` when checking for errors in Bifrost function calls.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:13:55.437Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/end_to_end_tool_calling.go:43-45
Timestamp: 2025-06-16T04:13:55.437Z
Learning: In the Bifrost codebase, errors returned from client methods like ChatCompletionRequest are of type BifrostError, not the standard error interface. For testing these errors, use require.Nilf instead of require.NoErrorf since BifrostError doesn't work with the standard error assertion methods.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-08-05T20:43:59.593Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#214
File: core/providers/azure.go:49-50
Timestamp: 2025-08-05T20:43:59.593Z
Learning: In core/providers/azure.go, the azureTextCompletionResponsePool should use AzureTextResponse type, not schemas.BifrostResponse, to maintain consistency with the acquireAzureTextResponse() and releaseAzureTextResponse() functions that work with *AzureTextResponse objects.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-06-04T09:22:18.123Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#54
File: core/schemas/bifrost.go:46-49
Timestamp: 2025-06-04T09:22:18.123Z
Learning: In core/schemas/bifrost.go, the RequestInput struct uses ChatCompletionInput *[]BifrostMessage (pointer-to-slice) rather than []BifrostMessage to properly represent union type semantics. For text completion requests, ChatCompletionInput should be nil to indicate "no chat payload at all", while for chat completion requests it should be non-nil (even if empty slice). This distinguishes between different request types rather than just empty vs non-empty chat messages.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-06-16T06:56:55.290Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#88
File: core/providers/mistral.go:170-176
Timestamp: 2025-06-16T06:56:55.290Z
Learning: When JSON unmarshaling into pooled structs, slice fields like `Choices []schemas.BifrostResponseChoice` get fresh heap memory allocations from `json.Unmarshal()`. The slice data is not part of the pooled struct's memory, so defensive copying is unnecessary. Resetting pooled structs with `*resp = ResponseType{}` only clears slice headers, not the underlying data.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-06-15T14:24:49.882Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#83
File: core/providers/mistral.go:168-170
Timestamp: 2025-06-15T14:24:49.882Z
Learning: In the Bifrost codebase, performance is prioritized over defensive copying for HTTP service operations. Specifically, shallow slice assignments in provider response handling are acceptable due to object pool reset patterns and JSON unmarshaling behavior that minimize practical data corruption risks.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-06-09T17:33:52.234Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/utils.go:169-173
Timestamp: 2025-06-09T17:33:52.234Z
Learning: The ChatCompletionRequest method in the Bifrost client follows a contract where the result parameter will never be nil if the error parameter is nil. This means when error checking passes (err == nil), the result is guaranteed to be valid and can be safely used without additional nil checks.
Applied to files:
core/providers/cerebras.go
🧬 Code Graph Analysis (7)
core/schemas/bifrost.go (1)
ui/lib/types/config.ts (1)
ModelProvider(4-16)
tests/core-providers/cerebras_test.go (3)
tests/core-providers/config/setup.go (1)
SetupTest(51-59)tests/core-providers/config/account.go (2)
ComprehensiveTestConfig(38-46)TestScenarios(17-35)core/schemas/bifrost.go (1)
Cerebras(52-52)
core/bifrost.go (2)
core/schemas/bifrost.go (1)
Cerebras(52-52)core/providers/cerebras.go (1)
NewCerebrasProvider(72-108)
tests/core-providers/config/account.go (5)
core/schemas/bifrost.go (1)
Cerebras(52-52)ui/lib/types/config.ts (4)
Key(43-51)ProviderConfig(81-87)NetworkConfig(54-61)ConcurrencyAndBufferSize(64-67)core/schemas/account.go (1)
Key(8-16)transports/bifrost-http/lib/config.go (1)
ProviderConfig(33-39)core/schemas/provider.go (5)
ProviderConfig(87-94)NetworkConfig(32-40)DefaultNetworkConfig(43-48)ConcurrencyAndBufferSize(51-54)DefaultConcurrencyAndBufferSize(57-60)
ui/lib/types/config.ts (1)
core/schemas/bifrost.go (1)
ModelProvider(38-38)
transports/bifrost-http/integrations/utils.go (1)
core/schemas/bifrost.go (1)
Cerebras(52-52)
core/providers/cerebras.go (5)
core/providers/azure.go (1)
AzureTextResponse(19-32)core/schemas/bifrost.go (13)
BifrostResponse(376-389)ModelProvider(38-38)Cerebras(52-52)ModelParameters(166-183)BifrostError(681-690)BifrostResponseChoice(549-555)BifrostNonStreamResponseChoice(558-562)BifrostMessage(287-295)ModelChatMessageRoleAssistant(30-30)MessageContent(297-300)LogProbs(460-464)BifrostResponseExtraFields(654-662)BifrostStream(670-673)core/schemas/logger.go (1)
Logger(18-39)ui/lib/types/config.ts (2)
NetworkConfig(54-61)ModelProvider(4-16)core/schemas/provider.go (2)
DefaultRequestTimeoutInSeconds(14-14)PostHookRunner(129-129)
🪛 LanguageTool
docs/usage/go-package/README.md
[grammar] ~218-~218: There might be a problem here.
Context: ...n](../providers.md) - All 12+ providers setup ### Development - Logging - Debug visibilit...
(QB_NEW_EN_MERGED_MATCH)
README.md
[grammar] ~13-~13: Use correct spacing
Context: ...d zero-downtime deployments in under 30 seconds. 🚀 Just launched: Native MCP (Model ...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~78-~78: There might be a mistake here.
Context: ... to build AI applications that never go down](#the-fastest-way-to-build-ai-applications-that-never-go-down) - [⚡ Quickstart (30 seconds)](#-quickstart-...
(QB_NEW_EN_OTHER)
tests/core-providers/README.md
[grammar] ~18-~18: Add a comma
Context: ...** - OSS models - Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Te...
(QB_NEW_EN_OTHER_ERROR_IDS_22)
[grammar] ~18-~18: Use correct spacing
Context: ... Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Tests ### Development wit...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
🔇 Additional comments (28)
docs/usage/go-package/account.md (1)
582-582: Docs count update looks good (12+ providers).Link target remains correct relative to this file (../providers.md → docs/usage/providers.md). No further action.
ci/npx/package.json (1)
4-4: LGTM: description updated to 12+ providers.Consistent with the PR scope. No action needed.
transports/README.md (2)
55-55: Provider list updated correctly; verify consistency with ValidProvidersCerebras addition looks good here. Please verify the list stays in sync with transports/bifrost-http/integrations/utils.go: ValidProviders and the core schemas enums to avoid drift.
You can run a quick grep to compare occurrences of provider labels across the repo if needed.
295-295: Count updated to 12+ — LGTMThe cross-references to 12+ providers align with the broader docs/UI updates.
docs/usage/go-package/schemas.md (1)
804-807: Docs list extended with Parasail, SGLang, and Cerebras — LGTMThe provider list now reflects the new enums/providers added in core.
README.md (3)
78-78: TOC entry/anchor for the new headline — LGTMAnchor matches the section title and renders correctly.
251-251: “Multi-Provider Support” updated to 12+ — LGTMConsistent with the new provider count and Cerebras addition.
13-13: No stale “10+”/“8+” mentions found
Ran a full-regex search across the repo for “10+” and “8+ AI providers” and confirmed there are no lingering occurrences. The README bump to “12+ providers” is consistent—no further updates needed.ui/README.md (1)
84-84: No action needed for provider count in ui/README.mdI verified there is no “Configure 8+ AI providers” text in
ui/README.md, so the suggestion to update it to “12+” is no longer applicable.Likely an incorrect or invalid review comment.
core/schemas/bifrost.go (1)
52-53: Cerebras provider mapping verified — LGTMThe factory case for schemas.Cerebras (lines 690–691 in core/bifrost.go) correctly returns NewCerebrasProvider. No further action required.
tests/core-providers/go.mod (2)
10-11: Local replace to ../../core — LGTM and consistent with project practiceMatches established pattern for the internal test module.
10-11: CI Go versions are aligned with go 1.24.1All workflows already target Go 1.24.x or higher, matching the test module’s directive:
- .github/workflows/core-dependency-update.yml
• setup-go@v4 withgo-version: "1.24.1"(lines 31–33, 71–75)- .github/workflows/transports-release.yml
• setup-go@v4 withgo-version: "1.24.1"(lines 85–89)- .github/workflows/snyk.yml
• setup-go@v5 withgo-version: '1.24'(lines 33–37, 73–77) – this semver spec installs the latest 1.24.x patchNo changes required; CI already uses a compatible Go toolchain.
docs/usage/providers.md (2)
18-21: Cerebras added to Supported Providers table — LGTMModel/label addition is clear and consistent with PR scope.
502-503: Audio features for Cerebras marked Not Available — LGTMMatches expected non-audio scope for the initial integration.
ui/lib/types/config.ts (1)
4-16: Cerebras provider added to the union — LGTMThe string literal union correctly includes 'cerebras' and stays consistent with the backend's ModelProvider constants. No breaking changes here.
ui/lib/constants/logs.ts (2)
1-14: Added 'cerebras' to PROVIDERS — LGTMThe provider list update cleanly extends the union type for Provider across the UI. Cross-file updates (icons/types) appear aligned per PR summary.
42-42: Provider label for Cerebras — LGTMLabel addition is consistent with the new provider key and UI rendering expectations.
core/bifrost.go (1)
690-691: Factory switch extended for Cerebras — LGTMThe constructor signature pattern matches other providers returning (schemas.Provider, error). This wires Cerebras cleanly into provider creation.
transports/bifrost-http/integrations/utils.go (1)
645-646: Cerebras added to ValidProviders — LGTM, but confirm model inference expectationsThis enables prefix-based routing like "cerebras/" via ParseModelString. The AI summary mentions “expands model string parsing to recognize Cerebras models,” but this file only adds Cerebras to ValidProviders. If you intended heuristic-based inference (without prefix) for Cerebras, consider documenting that explicit prefix is required or extend GetProviderFromModel accordingly. Given model name overlap (e.g., llama-* across multiple providers), sticking to explicit prefix is probably safer.
Do you want to rely solely on "cerebras/" to disambiguate models, or should we add inference heuristics for Cerebras models?
tests/core-providers/config/account.go (3)
73-74: Include Cerebras in configured providers — LGTMThis ensures the test harness initializes channels/workers for Cerebras during setup.
172-179: Cerebras key sourcing — verify env presenceKey wiring follows existing patterns. Ensure CEREBRAS_API_KEY is set in CI/local env; otherwise tests will 401 at runtime.
Would you like a small guard to skip Cerebras tests when CEREBRAS_API_KEY is unset (consistent with the current suite’s approach if applicable)?
287-291: Cerebras default ProviderConfig — LGTMReusing DefaultNetworkConfig and DefaultConcurrencyAndBufferSize matches other providers and aligns with the repo’s defaults.
tests/core-providers/cerebras_test.go (1)
1-41: End-to-end Cerebras test scaffold — verify TextModel format & feature supportPlease double-check that the Cerebras text model identifier is correct (it may require a hyphen) and that this model supports TextCompletion before running the full suite. If the hyphenated form is required, update as shown below; otherwise leave as-is. If TextCompletion isn’t supported, disable it in the scenarios.
• File: tests/core-providers/cerebras_test.go
- Around the
TextModelassignment intestConfig:- TextModel: "llama3.1-8b", + TextModel: "llama-3.1-8b",• If the chosen model does not support text completions, set
Scenarios.TextCompletion = falsein that same test configuration.core/providers/cerebras.go (5)
18-23: LGTM! Pooling approach follows established pattern.Based on the retrieved learnings, reusing compatible response types across providers (like
AzureTextResponsefor Cerebras) is intentional in Bifrost to avoid unnecessary type duplication when the API response structures are identical.
92-94: Streaming client needs proxy configuration too.The proxy configuration is only applied to the fasthttp client but not to the
streamClientused for SSE. However, as noted in the past review comments, this is a broader issue affecting all providers with streaming functionality.Based on your previous comment, this should be tracked as a separate issue for all providers rather than fixed individually. The GitHub issue #269 has been created to address proxy configuration across all streaming clients systematically.
167-169: Provider correctly set in ExtraFields.The
ExtraFields.Provideris correctly set toschemas.Cerebras, fixing the issue from the previous review where it was incorrectly set toschemas.Azure.
311-318: Correct Authorization header setup.The code correctly sets the Authorization header with the Bearer token when an API key is provided. This is the proper approach for Cerebras API authentication.
335-349: Comprehensive unsupported operation coverage.All unsupported operations (Speech, SpeechStream, Transcription, TranscriptionStream) properly return appropriate error messages using
newUnsupportedOperationError, maintaining consistency with the provider interface.
1a9c8c7 to
122ad1e
Compare
There was a problem hiding this comment.
Actionable comments posted: 15
🔭 Outside diff range comments (3)
tests/core-providers/go.mod (1)
3-3: Invalid go directive: use 'go 1.24' and optionally add a toolchain line for patch pinningThe 'go' directive must be major.minor only. 'go 1.24.1' will cause 'invalid go version' errors.
Apply:
-go 1.24.1 +go 1.24 + +toolchain go1.24.1This preserves your intent to pin the toolchain patch while keeping the module directive valid.
ui/README.md (1)
11-13: Update provider count in Key Features (8+ → 12+)The UI now supports 12+ providers. Keep the headline in sync.
Apply this diff:
- - **⚙️ Provider Management** - Configure 8+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.) + - **⚙️ Provider Management** - Configure 12+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.)tests/core-providers/config/account.go (1)
297-541: Add Cerebras to AllProviderConfigs for comprehensive test coverageThe
AllProviderConfigsslice intests/core-providers/config/account.gois missing an entry for Cerebras. Since Cerebras is already exposed inGetConfiguredProvidersandGetKeysForProvider, we should include it here so our scenario‐driven tests cover it (text completion only).• File:
tests/core-providers/config/account.go
• Location: just before the closing}of theAllProviderConfigssliceInsert this block:
{ Provider: schemas.Cerebras, ChatModel: "llama-3.3-70b", TextModel: "llama-3.3-70b", // Cerebras exposes a Completions endpoint Scenarios: TestScenarios{ TextCompletion: true, SimpleChat: true, ChatCompletionStream: true, MultiTurnConversation: true, ToolCalls: true, MultipleToolCalls: true, End2EndToolCalling: true, AutomaticFunctionCall: true, ImageURL: false, // Llama 3.3 70B is text-only ImageBase64: false, MultipleImages: false, CompleteEnd2End: true, ProviderSpecific: true, SpeechSynthesis: false, SpeechSynthesisStream: false, Transcription: false, TranscriptionStream: false, }, Fallbacks: []schemas.Fallback{ {Provider: schemas.OpenAI, Model: "gpt-4o-mini"}, }, },
♻️ Duplicate comments (7)
docs/usage/go-package/README.md (1)
218-218: Grammar polish: rephrase “All 12+ providers setup.”Prefer “Configuration for all 12+ providers” for clarity and correctness.
Apply this diff:
- [Provider Configuration](../providers.md) - All 12+ providers setup + [Provider Configuration](../providers.md) - Configuration for all 12+ providerstransports/go.mod (1)
20-21: Guard local replace directive; add an inline note to avoid accidental releaseThis local replace is useful for monorepo development but risky if published. Add an inline comment clarifying it must be removed/overridden before release.
Apply this diff:
+// NOTE: Local monorepo development only. Remove or override this before tagging a release. replace github.com/maximhq/bifrost/core => ../coredocs/quickstart/go-package.md (1)
223-223: Fix Providers link path (same as past review)The Providers link should point to ../usage/providers.md, not ../providers.md.
-| **Add all 12+ providers** | [🔗 Providers](../providers.md) | 5 min | +| **Add all 12+ providers** | [🔗 Providers](../usage/providers.md) | 5 min |tests/core-providers/README.md (2)
50-81: Document Cerebras env var in Prerequisites to prevent skipped tests.Tests will skip if the key isn’t set. Add the export like other providers.
# Mistral AI export MISTRAL_API_KEY="your-mistral-key" # Ollama (local installation) # No API key required - ensure Ollama is running locally # Default endpoint: http://localhost:11434 + +# Cerebras +export CEREBRAS_API_KEY="your-cerebras-key"
18-18: Grammar nit: add serial comma in Cerebras description.Use the Oxford comma for consistency across items.
- - **Cerebras** - Llama, Qwen and GPT-OSS models + - **Cerebras** - Llama, Qwen, and GPT-OSS modelsdocs/usage/providers.md (1)
475-486: Correct Cerebras Vision support (set to ❌)Cerebras’ highlighted model (Llama 3.3 70B) is text-only. The provider doesn’t expose image/vision in Bifrost. Update the Vision row to avoid misleading users.
Apply this diff:
-| **Vision** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | +| **Vision** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |core/providers/cerebras.go (1)
25-45: Remove large commented-out chat pooling block to reduce noiseCommented code quickly becomes stale. Given the plan to re-introduce pooling with ref counting later, prefer deleting this block and tracking the future work via a TODO/issue reference.
-// // cerebrasChatResponsePool provides a pool for Cerebras chat response objects. -// var cerebrasChatResponsePool = sync.Pool{ -// -// New: func() interface{} { -// -// return &schemas.BifrostResponse{} -// -// }, -// -// } - -// // acquireCerebrasChatResponse gets a Cerebras response from the pool and resets it. -// func acquireCerebrasChatResponse() *schemas.BifrostResponse { -// -// resp := cerebrasChatResponsePool.Get().(*schemas.BifrostResponse) -// -// *resp = schemas.BifrostResponse{} // Reset the struct -// -// return resp -// -// } - -// // releaseCerebrasChatResponse returns a Cerebras response to the pool. -// func releaseCerebrasChatResponse(resp *schemas.BifrostResponse) { -// -// if resp != nil { -// -// cerebrasChatResponsePool.Put(resp) -// -// } -// -// }Optionally, add a one-line TODO near ChatCompletion:
// TODO: Evaluate reintroducing chat response pooling with ref counting in a future PR.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/core-providers/go.sumis excluded by!**/*.sum
📒 Files selected for processing (30)
README.md(3 hunks)ci/npx/package.json(1 hunks)core/bifrost.go(1 hunks)core/providers/azure.go(4 hunks)core/providers/cerebras.go(1 hunks)core/providers/groq.go(4 hunks)core/providers/mistral.go(4 hunks)core/providers/ollama.go(3 hunks)core/providers/openai.go(4 hunks)core/providers/parasail.go(3 hunks)core/providers/sgl.go(3 hunks)core/providers/vertex.go(3 hunks)core/schemas/bifrost.go(1 hunks)docs/quickstart/go-package.md(1 hunks)docs/usage/go-package/README.md(1 hunks)docs/usage/go-package/account.md(1 hunks)docs/usage/go-package/schemas.md(1 hunks)docs/usage/providers.md(3 hunks)tests/core-providers/README.md(1 hunks)tests/core-providers/cerebras_test.go(1 hunks)tests/core-providers/config/account.go(3 hunks)tests/core-providers/go.mod(3 hunks)transports/README.md(2 hunks)transports/bifrost-http/integrations/utils.go(1 hunks)transports/go.mod(1 hunks)ui/README.md(1 hunks)ui/app/layout.tsx(1 hunks)ui/lib/constants/icons.tsx(2 hunks)ui/lib/constants/logs.ts(2 hunks)ui/lib/types/config.ts(1 hunks)
👮 Files not reviewed due to content moderation or server errors (1)
- ui/lib/constants/icons.tsx
🧰 Additional context used
🧠 Learnings (21)
📓 Common learnings
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:358-388
Timestamp: 2025-06-04T05:37:59.699Z
Learning: User Pratham-Mishra04 prefers not to extract small code duplications (around 2 lines) into helper functions, considering the overhead not worth it for such minor repetition.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:25-31
Timestamp: 2025-08-13T07:56:05.626Z
Learning: Pratham-Mishra04 prefers temporary solutions that prevent correctness issues (like commenting out problematic defer statements) when planning to implement better long-term solutions (like reference counting) in the future, accepting trade-offs like temporary memory leaks for safety.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#102
File: README.md:62-66
Timestamp: 2025-06-19T17:03:03.639Z
Learning: Pratham-Mishra04 prefers using the implicit 'latest' tag for the maximhq/bifrost Docker image rather than pinning to specific versions.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#143
File: core/mcp.go:155-196
Timestamp: 2025-07-08T15:33:47.698Z
Learning: Pratham-Mishra04 prefers not to add explanatory comments for obvious code patterns, such as the unlock/lock strategy around network I/O operations, considering them self-explanatory to experienced developers.
📚 Learning: 2025-08-12T20:27:20.757Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:18-23
Timestamp: 2025-08-12T20:27:20.757Z
Learning: In Bifrost providers, it's intentional to reuse compatible response types across different providers (e.g., using AzureTextResponse in Cerebras provider) when they follow the same struct, rather than creating provider-specific aliases. This avoids unnecessary type duplication for identical API response structures.
Applied to files:
core/bifrost.gocore/providers/parasail.goREADME.mdcore/providers/sgl.gocore/providers/mistral.gocore/providers/ollama.gocore/providers/groq.gocore/providers/vertex.gocore/providers/openai.gocore/providers/azure.gocore/providers/cerebras.go
📚 Learning: 2025-08-12T20:32:24.408Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: tests/core-providers/cerebras_test.go:1-1
Timestamp: 2025-08-12T20:32:24.408Z
Learning: In the Bifrost project, test files in tests/core-providers directory should use `package tests` rather than `package main`, as confirmed by Pratham-Mishra04 in PR #267 for tests/core-providers/cerebras_test.go.
Applied to files:
tests/core-providers/cerebras_test.gotests/core-providers/go.mod
📚 Learning: 2025-06-16T04:55:11.886Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/automatic_function_calling.go:22-22
Timestamp: 2025-06-16T04:55:11.886Z
Learning: In the Bifrost test suite (tests/core-providers), parallel tests using t.Parallel() are not being implemented currently. The team plans to add parallel test execution in future enhancements.
Applied to files:
tests/core-providers/cerebras_test.go
📚 Learning: 2025-08-05T20:43:59.593Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#214
File: core/providers/azure.go:49-50
Timestamp: 2025-08-05T20:43:59.593Z
Learning: In core/providers/azure.go, the azureTextCompletionResponsePool should use AzureTextResponse type, not schemas.BifrostResponse, to maintain consistency with the acquireAzureTextResponse() and releaseAzureTextResponse() functions that work with *AzureTextResponse objects.
Applied to files:
core/providers/parasail.gocore/providers/sgl.gocore/providers/mistral.gocore/providers/ollama.gocore/providers/groq.gocore/providers/vertex.gocore/providers/openai.gocore/providers/azure.gocore/providers/cerebras.go
📚 Learning: 2025-07-29T16:10:52.088Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#196
File: core/providers/openai.go:180-183
Timestamp: 2025-07-29T16:10:52.088Z
Learning: In the Bifrost provider architecture, `handleProviderResponse` is a utility function that only parses and returns raw response data when the `sendBackRawResponse` flag is true. It's the responsibility of each individual provider (OpenAI, Anthropic, etc.) to conditionally set `response.ExtraFields.RawResponse` using the returned raw response data based on their `sendBackRawResponse` flag. This represents a separation of concerns where the utility handles parsing and the provider handles response object construction.
Applied to files:
core/providers/parasail.gocore/providers/sgl.gocore/providers/mistral.gocore/providers/ollama.gocore/providers/groq.gocore/providers/vertex.gocore/providers/openai.gocore/providers/azure.go
📚 Learning: 2025-06-20T16:21:18.912Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#103
File: .github/workflows/transport-dependency-update.yml:53-75
Timestamp: 2025-06-20T16:21:18.912Z
Learning: In the bifrost repository's transport dependency update workflow, when updating the core dependency to a new version using `go get`, the go.mod and go.sum files will always change in normal operation, making the safety check for changes more of a defensive programming practice rather than handling a common scenario.
Applied to files:
transports/go.modtests/core-providers/go.mod
📚 Learning: 2025-06-27T17:07:39.462Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#135
File: docs/core-package.md:105-116
Timestamp: 2025-06-27T17:07:39.462Z
Learning: In Go, when importing a package, the identifier used to access it is determined by the `package` declaration in the Go source files, not the directory name. For the Bifrost project, the core directory files declare `package bifrost`, so importing `"github.com/maximhq/bifrost/core"` automatically makes it accessible as `bifrost.Init`, `bifrost.Ptr`, etc., without needing an explicit alias.
Applied to files:
transports/go.modtests/core-providers/go.mod
📚 Learning: 2025-06-15T16:05:13.489Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: transports/bifrost-http/main.go:2-2
Timestamp: 2025-06-15T16:05:13.489Z
Learning: For the Bifrost project, HTTP transport integration routers for new providers (like Mistral and Ollama) are implemented in separate PRs from the core provider support, following a focused PR strategy.
Applied to files:
README.md
📚 Learning: 2025-06-09T16:46:32.018Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/anthropic/router.go:19-33
Timestamp: 2025-06-09T16:46:32.018Z
Learning: In the GenericRouter (transports/bifrost-http/integrations), ResponseFunc is not called if the BifrostResponse parameter is nil, providing built-in protection against nil response marshaling.
Applied to files:
core/providers/sgl.gocore/providers/mistral.gocore/providers/groq.go
📚 Learning: 2025-06-15T14:24:49.882Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#83
File: core/providers/mistral.go:168-170
Timestamp: 2025-06-15T14:24:49.882Z
Learning: In the Bifrost codebase, performance is prioritized over defensive copying for HTTP service operations. Specifically, shallow slice assignments in provider response handling are acceptable due to object pool reset patterns and JSON unmarshaling behavior that minimize practical data corruption risks.
Applied to files:
core/providers/sgl.gocore/providers/mistral.gocore/providers/groq.gocore/providers/openai.gocore/providers/cerebras.go
📚 Learning: 2025-06-15T14:34:29.401Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: core/providers/ollama.go:59-61
Timestamp: 2025-06-15T14:34:29.401Z
Learning: The `CheckAndSetDefaults()` method on `*schemas.ProviderConfig` in Go does not return any error - it has a void return type and only sets default values on the configuration struct.
Applied to files:
tests/core-providers/config/account.go
📚 Learning: 2025-06-16T03:54:48.005Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/go.mod:38-38
Timestamp: 2025-06-16T03:54:48.005Z
Learning: The `core-providers-test` module in `tests/core-providers/` is an internal testing module that will never be consumed as a dependency by external projects, so the replace directive pointing to `../../core` is acceptable for local development and testing purposes.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:12:05.427Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/complete_end_to_end.go:39-41
Timestamp: 2025-06-16T04:12:05.427Z
Learning: In the Bifrost system, error returns are of type `BifrostError` rather than the standard Go `error` interface. Therefore, use `require.Nilf(t, err, ...)` instead of `require.NoError(t, err)` when checking for errors in Bifrost function calls.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:13:55.437Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/end_to_end_tool_calling.go:43-45
Timestamp: 2025-06-16T04:13:55.437Z
Learning: In the Bifrost codebase, errors returned from client methods like ChatCompletionRequest are of type BifrostError, not the standard error interface. For testing these errors, use require.Nilf instead of require.NoErrorf since BifrostError doesn't work with the standard error assertion methods.
Applied to files:
tests/core-providers/go.modcore/providers/azure.go
📚 Learning: 2025-08-08T07:07:08.828Z
Learnt from: TejasGhatte
PR: maximhq/bifrost#219
File: core/providers/openai.go:459-459
Timestamp: 2025-08-08T07:07:08.828Z
Learning: In core/providers/utils.go, processAndSendResponse and processAndSendError now take exactly five parameters: (ctx, postHookRunner, response|err, responseChan, logger). All providers (OpenAI, Anthropic, Cohere, Bedrock) must call these with the logger as the fifth argument; a repo-wide check in PR maximhq/bifrost#219 confirmed this is already consistent.
Applied to files:
core/providers/openai.go
📚 Learning: 2025-06-04T09:22:18.123Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#54
File: core/schemas/bifrost.go:46-49
Timestamp: 2025-06-04T09:22:18.123Z
Learning: In core/schemas/bifrost.go, the RequestInput struct uses ChatCompletionInput *[]BifrostMessage (pointer-to-slice) rather than []BifrostMessage to properly represent union type semantics. For text completion requests, ChatCompletionInput should be nil to indicate "no chat payload at all", while for chat completion requests it should be non-nil (even if empty slice). This distinguishes between different request types rather than just empty vs non-empty chat messages.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-06-09T17:33:52.234Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/utils.go:169-173
Timestamp: 2025-06-09T17:33:52.234Z
Learning: The ChatCompletionRequest method in the Bifrost client follows a contract where the result parameter will never be nil if the error parameter is nil. This means when error checking passes (err == nil), the result is guaranteed to be valid and can be safely used without additional nil checks.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-06-16T06:56:55.290Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#88
File: core/providers/mistral.go:170-176
Timestamp: 2025-06-16T06:56:55.290Z
Learning: When JSON unmarshaling into pooled structs, slice fields like `Choices []schemas.BifrostResponseChoice` get fresh heap memory allocations from `json.Unmarshal()`. The slice data is not part of the pooled struct's memory, so defensive copying is unnecessary. Resetting pooled structs with `*resp = ResponseType{}` only clears slice headers, not the underlying data.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-08-13T07:56:05.626Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:25-31
Timestamp: 2025-08-13T07:56:05.626Z
Learning: Pratham-Mishra04 prefers temporary solutions that prevent correctness issues (like commenting out problematic defer statements) when planning to implement better long-term solutions (like reference counting) in the future, accepting trade-offs like temporary memory leaks for safety.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-07-08T18:30:08.258Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#141
File: core/bifrost.go:198-272
Timestamp: 2025-07-08T18:30:08.258Z
Learning: Pratham-Mishra04 follows a pattern of implementing core functionality first and deferring non-critical improvements (like race condition fixes, optimizations) to later PRs. This is a reasonable development approach that prioritizes getting the main feature working before addressing edge cases.
Applied to files:
core/providers/cerebras.go
🧬 Code Graph Analysis (15)
core/bifrost.go (2)
core/schemas/bifrost.go (1)
Cerebras(52-52)core/providers/cerebras.go (1)
NewCerebrasProvider(72-108)
tests/core-providers/cerebras_test.go (4)
tests/core-providers/config/setup.go (1)
SetupTest(51-59)tests/core-providers/config/account.go (2)
ComprehensiveTestConfig(38-46)TestScenarios(17-35)ui/lib/constants/logs.ts (1)
Provider(76-76)core/schemas/bifrost.go (1)
Cerebras(52-52)
ui/lib/types/config.ts (1)
core/schemas/bifrost.go (1)
ModelProvider(38-38)
core/providers/parasail.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/schemas/bifrost.go (1)
ui/lib/types/config.ts (1)
ModelProvider(4-16)
transports/bifrost-http/integrations/utils.go (1)
core/schemas/bifrost.go (1)
Cerebras(52-52)
core/providers/sgl.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/mistral.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
tests/core-providers/config/account.go (5)
core/schemas/bifrost.go (1)
Cerebras(52-52)ui/lib/types/config.ts (4)
Key(43-51)ProviderConfig(81-87)NetworkConfig(54-61)ConcurrencyAndBufferSize(64-67)core/schemas/account.go (1)
Key(8-16)transports/bifrost-http/lib/config.go (1)
ProviderConfig(33-39)core/schemas/provider.go (5)
ProviderConfig(87-94)NetworkConfig(32-40)DefaultNetworkConfig(43-48)ConcurrencyAndBufferSize(51-54)DefaultConcurrencyAndBufferSize(57-60)
core/providers/ollama.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/groq.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/vertex.go (3)
ui/lib/constants/logs.ts (1)
Provider(76-76)core/schemas/provider.go (1)
Provider(132-151)core/schemas/bifrost.go (2)
Vertex(46-46)BifrostResponse(376-389)
core/providers/openai.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/azure.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/cerebras.go (4)
core/providers/azure.go (1)
AzureTextResponse(19-32)ui/lib/types/config.ts (6)
NetworkConfig(54-61)ProviderConfig(81-87)ConcurrencyAndBufferSize(64-67)ProxyConfig(73-78)ModelProvider(4-16)Key(43-51)core/schemas/provider.go (3)
DefaultRequestTimeoutInSeconds(14-14)ErrProviderJSONMarshaling(24-24)PostHookRunner(129-129)core/schemas/bifrost.go (11)
ModelProvider(38-38)Cerebras(52-52)ModelParameters(166-183)BifrostResponse(376-389)BifrostError(681-690)BifrostResponseChoice(549-555)BifrostNonStreamResponseChoice(558-562)BifrostMessage(287-295)LogProbs(460-464)BifrostResponseExtraFields(654-662)BifrostStream(670-673)
🪛 LanguageTool
tests/core-providers/README.md
[grammar] ~18-~18: Add a comma
Context: ...** - OSS models - Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Te...
(QB_NEW_EN_OTHER_ERROR_IDS_22)
[grammar] ~18-~18: Use correct spacing
Context: ... Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Tests ### Development wit...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
docs/usage/go-package/README.md
[grammar] ~218-~218: There might be a problem here.
Context: ...n](../providers.md) - All 12+ providers setup ### Development - Logging - Debug visibilit...
(QB_NEW_EN_MERGED_MATCH)
🔇 Additional comments (48)
ci/npx/package.json (1)
4-4: LGTM on updated provider count in descriptionThe description accurately reflects the 12+ provider expansion.
core/schemas/bifrost.go (1)
52-52: LGTM: added Cerebras provider enumEnum addition is correct and consistent with UI/types.
transports/bifrost-http/integrations/utils.go (2)
645-645: Cerebras added to ValidProviders: good additionThis enables provider-prefix parsing for "cerebras/" and O(1) validation. No concerns.
676-727: Provider auto-detection may misroute plain Cerebras model IDs (e.g., "llama-3.3-70b") to OpenAIGetProviderFromModel defaults unknown patterns to OpenAI. If callers pass bare Cerebras model IDs without provider prefix and without an explicit provider, requests could be routed to OpenAI. Consider either:
- Documenting that callers should use "cerebras/" or set Provider explicitly, or
- Adding Cerebras-specific detection logic (only if unambiguous for your deployment), or
- Avoid defaulting to OpenAI for truly unknown models.
If you prefer a defensive tweak now, minimally avoid defaulting to OpenAI for unknown models so upstream must provide a provider or prefix:
- // Default to OpenAI for unknown models (most LiteLLM compatible) - return schemas.OpenAI + // Default to empty/unknown provider to force caller to be explicit, + // otherwise we risk misrouting to a wrong provider. + return ""This would require upstream to set a default provider elsewhere (e.g., route-level default) if desired.
Would you like me to wire a small test that verifies "llama-3.3-70b" without a prefix does not silently route to OpenAI?
tests/core-providers/go.mod (1)
10-11: Local replace for core in tests: LGTM and consistent with repo normsThe replace to ../../core is appropriate for this internal test module (per past decisions).
core/providers/openai.go (3)
68-72: Pre-warm removal: OKDropping pool pre-warm is consistent with the pool removal. No functional impact.
281-285: Direct allocation for Embedding: LGTMAllocation change is correct; ExtraFields.Provider is set later, maintaining parity.
21-41: Response pooling removal verified—no active acquire/release calls remainAll references to
acquireOpenAIResponseandreleaseOpenAIResponseare commented out; there are no live calls in the codebase. Pool removal is complete.core/bifrost.go (2)
690-692: Cerebras switch-case wiring added: looks correctFactory now returns the Cerebras provider. This aligns with the new provider integration and error-handling pattern in this function.
642-695: No changes needed: Cerebras is already correctly handled by key helpersBoth utility functions already treat Cerebras appropriately:
providerRequiresKeyreturns true for Cerebras (it’s not one of the keyless providers).canProviderKeyValueBeEmptyreturns false for Cerebras (it’s not listed among providers that allow empty keys).No updates are required.
ui/app/layout.tsx (1)
24-24: Polish metadata description; fix capitalization (MCP) and tighten phrasing.The copy is wordy and uses lowercase “mcp”. Suggest a concise, polished description with proper capitalization.
- 'Production-ready fastest LLM gateway that connects to 12+ providers through a single API. Get automatic failover, load balancing, mcp support and zero-downtime deployments.', + 'The fastest, production-ready LLM gateway connecting to 12+ providers through a single API. Automatic failover, load balancing, MCP support, and zero-downtime deployments.',[ suggest_nitpick ]
[ duplicate_comment ]docs/usage/go-package/schemas.md (1)
804-807: LGTM: Providers list correctly includes Parasail, SGLang, and Cerebras.The expanded ModelProvider list is accurate and matches the broader PR changes. No further edits needed here.
transports/README.md (2)
55-55: LGTM: Multi-Provider Support now reflects Cerebras.Provider list update is accurate and consistent with the PR’s scope.
295-295: LGTM: Link text updated to reflect 12+ providers.Docs link now matches the new provider count.
README.md (3)
78-78: TOC anchor addition looks correct.Anchor matches the section header and improves navigation.
251-251: LGTM: Updated Multi-Provider Support link text to 12+.Consistent with the new provider count.
13-13: No remaining “10+ providers” references foundI ran a repository-wide search for “10+ providers” (and variants) and found no occurrences—README and messaging are consistent.
ui/README.md (1)
84-84: LGTM: Added Cerebras to Supported Providers listThe addition is accurate and consistent with the rest of the PR.
tests/core-providers/config/account.go (3)
73-74: LGTM: Cerebras added to configured providers listCorrectly registers Cerebras for tests that enumerate providers.
172-179: LGTM: Key sourcing for CerebrasUses CEREBRAS_API_KEY with sane defaults.
287-291: LGTM: Default network and concurrency for CerebrasMatches established defaults across providers.
core/providers/ollama.go (1)
148-152: LGTM: Switched to per-request allocationConsistent with the repo-wide move away from object pools and aligns with handleProviderResponse usage/contract.
ui/lib/types/config.ts (1)
4-16: LGTM: ModelProvider union extended with 'cerebras'Matches backend schemas and UI additions (icons/labels). Past feedback about icon/label coverage has been addressed.
docs/usage/providers.md (1)
18-21: LGTM: Added Parasail, SGLang, and Cerebras to Supported Providers tableAccurately reflects the new provider set.
core/providers/sgl.go (2)
154-158: Per-request allocation replaces pooling — LGTMDirect allocation with response := &schemas.BifrostResponse{} is consistent with the refactor across providers and avoids the prior reuse hazards. The subsequent handleProviderResponse usage and ExtraFields wiring look correct.
64-67: Commented pre-warm loop uses invalid Go syntax — fix if/when re-enabledIf you bring this back, the current pattern won’t compile. It should iterate an integer with a counter.
Apply this diff to correct the commented example:
-// for range config.ConcurrencyAndBufferSize.Concurrency { -// sglResponsePool.Put(&schemas.BifrostResponse{}) -// } +// for i := 0; i < config.ConcurrencyAndBufferSize.Concurrency; i++ { +// sglResponsePool.Put(&schemas.BifrostResponse{}) +// }⛔ Skipped due to learnings
Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.23 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.22 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable. This feature requires Go module version 1.22 or later.Learnt from: TejasGhatte PR: maximhq/bifrost#204 File: plugins/circuitbreaker/plugin_test.go:259-269 Timestamp: 2025-08-04T05:25:37.749Z Learning: In the Bifrost project, TejasGhatte prefers using modern Go 1.22+ syntax like `for range n` for iterating a specific number of times, considering it sufficiently explicit without needing traditional indexed loops for readability.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#83 File: core/providers/mistral.go:68-72 Timestamp: 2025-06-15T13:52:45.723Z Learning: Go 1.22+ supports `for range N` syntax where N is an integer, which iterates from 0 to N-1. This is valid Go syntax and does not require conversion to an indexed loop.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#88 File: core/providers/bedrock.go:185-189 Timestamp: 2025-06-15T16:10:41.196Z Learning: Go 1.22+ supports range over integer syntax: `for range n` iterates n times from 0 to n-1. This is valid Go syntax introduced in Go 1.22 (February 2024) and not a compilation error in modern Go versions.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#83 File: core/providers/mistral.go:68-72 Timestamp: 2025-06-15T13:52:45.723Z Learning: Go 1.22+ supports `for range N` syntax where N is an integer, which iterates from 0 to N-1. This is valid Go syntax and does not require conversion to an indexed loop. Example: `for range 10` iterates 10 times with values 0-9.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#88 File: core/providers/bedrock.go:185-189 Timestamp: 2025-06-15T16:10:41.196Z Learning: Go 1.22+ supports range over integer syntax: `for range n` iterates n times from 0 to n-1. This is valid Go syntax and not a compilation error in modern Go versions.core/providers/mistral.go (3)
146-149: Per-request allocation replaces pooling — LGTMSwitching to a fresh BifrostResponse per request is correct and consistent with the refactor. handleProviderResponse usage and ExtraFields wiring remain intact.
242-245: Embedding path allocation aligns with new pattern — LGTMSame as ChatCompletion: allocation and usage are consistent and correct.
64-68: Commented pre-warm loop uses invalid Go syntax — fix if/when re-enabledMirrors the SGL note: update to a proper integer loop if you ever re-enable pre-warming.
Apply this diff to correct the commented example:
-// for range config.ConcurrencyAndBufferSize.Concurrency { -// mistralResponsePool.Put(&schemas.BifrostResponse{}) -// } +// for i := 0; i < config.ConcurrencyAndBufferSize.Concurrency; i++ { +// mistralResponsePool.Put(&schemas.BifrostResponse{}) +// }⛔ Skipped due to learnings
Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.23 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.22 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable. This feature requires Go module version 1.22 or later.Learnt from: TejasGhatte PR: maximhq/bifrost#204 File: plugins/circuitbreaker/plugin_test.go:259-269 Timestamp: 2025-08-04T05:25:37.749Z Learning: In the Bifrost project, TejasGhatte prefers using modern Go 1.22+ syntax like `for range n` for iterating a specific number of times, considering it sufficiently explicit without needing traditional indexed loops for readability.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#83 File: core/providers/mistral.go:68-72 Timestamp: 2025-06-15T13:52:45.723Z Learning: Go 1.22+ supports `for range N` syntax where N is an integer, which iterates from 0 to N-1. This is valid Go syntax and does not require conversion to an indexed loop.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#88 File: core/providers/bedrock.go:185-189 Timestamp: 2025-06-15T16:10:41.196Z Learning: Go 1.22+ supports range over integer syntax: `for range n` iterates n times from 0 to n-1. This is valid Go syntax introduced in Go 1.22 (February 2024) and not a compilation error in modern Go versions.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#88 File: core/providers/bedrock.go:185-189 Timestamp: 2025-06-15T16:10:41.196Z Learning: Go 1.22+ supports range over integer syntax: `for range n` iterates n times from 0 to n-1. This is valid Go syntax and not a compilation error in modern Go versions.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#83 File: core/providers/mistral.go:68-72 Timestamp: 2025-06-15T13:52:45.723Z Learning: Go 1.22+ supports `for range N` syntax where N is an integer, which iterates from 0 to N-1. This is valid Go syntax and does not require conversion to an indexed loop. Example: `for range 10` iterates 10 times with values 0-9.tests/core-providers/cerebras_test.go (3)
1-1: Test package name is correct per repository conventionUsing package tests matches the established convention for tests/core-providers.
11-18: Setup/teardown sequencing looks solidSetupTest error handling plus deferred cancel and client.Cleanup are in the right order. No issues.
23-37: Tool & Streaming Integrations Confirmed in Cerebras ProviderBoth
ChatCompletion(lines 222–228) andChatCompletionStream(lines 303–309) invokeprepareOpenAIChatRequest, and the stream handler useshandleOpenAIStreaming(line 322). No additional changes are required.core/providers/groq.go (3)
64-67: Pre-warm logic correctly disabled alongside pool removal.Commenting out the pre-warm loop is consistent with disabling the pool and prevents unnecessary object allocations.
146-149: Direct per-request allocation looks good and matches handleProviderResponse expectations.Switching to
&schemas.BifrostResponse{}is consistent with the pool deprecation and the shared parser behavior.
188-196: SSE headers look correct; Authorization handled explicitly.Headers cover content type, SSE accept, and cache control; the bearer token is set separately. This aligns with the OpenAI-compatible streaming utility.
core/providers/parasail.go (2)
65-67: Pre-warm loop disabled in line with pool removal.Avoids dead code paths and unnecessary allocations.
146-149: Direct allocation is consistent with the pool removal and the shared response handler.No issues found.
core/providers/azure.go (1)
53-73: Azure chat pooling removal is consistent and avoids mixed pooling strategies.Removing the chat pool while keeping the AzureTextResponse pool for text completions is coherent and aligns with prior learnings on using provider-specific response types.
ui/lib/constants/logs.ts (2)
1-14: Provider list updated to include Cerebras — type union updated accordingly.This ensures UI logs support the new provider, and the Provider type widens automatically.
42-43: Provider label added for Cerebras — consistent with existing naming.No issues.
core/providers/vertex.go (3)
69-71: OpenAI-style response pre-warm correctly disabled; Anthropic pre-warm retained.This keeps the minimal necessary pool (Anthropic) while aligning non-Claude paths with per-request allocation.
279-284: RawResponse is now gated behind sendBackRawResponse — good separation of concerns.Prevents unnecessary payload bloat when not requested.
293-314: Non-Claude path switched to direct allocation with proper ExtraFields population — LGTM.Provider is set to Vertex, RawResponse is conditional, and Params are preserved.
core/providers/cerebras.go (5)
81-84: SSE may be prematurely terminated by http.Client.TimeoutFor streaming, a per-request timeout on http.Client often cuts streams. Prefer Timeout = 0 and rely on ctx cancellation and server-side timeouts.
- streamClient := &http.Client{ - Timeout: time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds), - } + streamClient := &http.Client{ + // For SSE, avoid hard timeouts; rely on ctx cancellation and transport timeouts. + Timeout: 0, + }Please confirm this aligns with how other providers handle streaming timeouts in this codebase (consistency matters). I can update all providers in a follow-up if desired.
166-169: LGTM: Pool usage confined to internal text response and properly releasedAcquiring from the text pool and releasing via defer is appropriate here since you copy out the needed fields into a new BifrostResponse.
269-273: Good fix: avoid pooling for returned chat responsesAllocating a fresh BifrostResponse for chat completions eliminates use-after-free risks associated with pooled objects returned to callers.
312-321: SSE headers look good; ensure Authorization is always presentHeaders for SSE are correct and Authorization is set. If ExtraHeaders could contain an Authorization override, verify that handleOpenAIStreaming merges maps with the explicit Authorization taking precedence.
86-90: Compile-time bug: cannot range over an int (pre-warm loop)In Go, you cannot range over an int. Use a counted for-loop.
- // Pre-warm response pools - for range config.ConcurrencyAndBufferSize.Concurrency { + // Pre-warm response pools + for i := 0; i < config.ConcurrencyAndBufferSize.Concurrency; i++ { // cerebrasChatResponsePool.Put(&schemas.BifrostResponse{}) cerebrasTextResponsePool.Put(&AzureTextResponse{}) }⛔ Skipped due to learnings
Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.23 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.22 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable. This feature requires Go module version 1.22 or later.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#88 File: core/providers/bedrock.go:185-189 Timestamp: 2025-06-15T16:10:41.196Z Learning: Go 1.22+ supports range over integer syntax: `for range n` iterates n times from 0 to n-1. This is valid Go syntax introduced in Go 1.22 (February 2024) and not a compilation error in modern Go versions.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#88 File: core/providers/bedrock.go:185-189 Timestamp: 2025-06-15T16:10:41.196Z Learning: Go 1.22+ supports range over integer syntax: `for range n` iterates n times from 0 to n-1. This is valid Go syntax and not a compilation error in modern Go versions.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#83 File: core/providers/mistral.go:68-72 Timestamp: 2025-06-15T13:52:45.723Z Learning: Go 1.22+ supports `for range N` syntax where N is an integer, which iterates from 0 to N-1. This is valid Go syntax and does not require conversion to an indexed loop.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#83 File: core/providers/mistral.go:68-72 Timestamp: 2025-06-15T13:52:45.723Z Learning: Go 1.22+ supports `for range N` syntax where N is an integer, which iterates from 0 to N-1. This is valid Go syntax and does not require conversion to an indexed loop. Example: `for range 10` iterates 10 times with values 0-9.Learnt from: TejasGhatte PR: maximhq/bifrost#204 File: plugins/circuitbreaker/plugin_test.go:259-269 Timestamp: 2025-08-04T05:25:37.749Z Learning: In the Bifrost project, TejasGhatte prefers using modern Go 1.22+ syntax like `for range n` for iterating a specific number of times, considering it sufficiently explicit without needing traditional indexed loops for readability.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#214 File: core/providers/azure.go:49-50 Timestamp: 2025-08-05T20:43:59.593Z Learning: In core/providers/azure.go, the azureTextCompletionResponsePool should use AzureTextResponse type, not schemas.BifrostResponse, to maintain consistency with the acquireAzureTextResponse() and releaseAzureTextResponse() functions that work with *AzureTextResponse objects.
122ad1e to
45da154
Compare
Merge activity
|
There was a problem hiding this comment.
Actionable comments posted: 16
🔭 Outside diff range comments (4)
tests/core-providers/go.mod (1)
3-3: Use a valid Go directive (major.minor only) or add a toolchain directiveThe
godirective must be major.minor (no patch).go 1.24.1will be normalized/flagged by tooling. Usego 1.24. If you want to mandate a specific patch, add a separatetoolchaindirective.Apply one of the following:
Option A (simplest):
-go 1.24.1 +go 1.24Option B (pin toolchain):
-go 1.24.1 +go 1.24 +toolchain go1.24.1core/providers/openai.go (3)
357-366: Use providerType for error attribution and avoid stringifying JSON bytesErrors in this OpenAI-compatible helper should be attributed to the invoking provider, not hardcoded to OpenAI.
- jsonBody, err := sonic.Marshal(requestBody) + jsonBody, err := sonic.Marshal(requestBody) if err != nil { - return nil, newBifrostOperationError(schemas.ErrProviderJSONMarshaling, err, schemas.OpenAI) + return nil, newBifrostOperationError(schemas.ErrProviderJSONMarshaling, err, providerType) } // Create HTTP request for streaming - req, err := http.NewRequestWithContext(ctx, "POST", url, strings.NewReader(string(jsonBody))) + req, err := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(jsonBody)) if err != nil { - return nil, newBifrostOperationError("failed to create HTTP request", err, schemas.OpenAI) + return nil, newBifrostOperationError("failed to create HTTP request", err, providerType) }
376-381: Propagate correct provider in request errorsEnsure non-OpenAI providers using this helper get correctly attributed errors.
- resp, err := httpClient.Do(req) + resp, err := httpClient.Do(req) if err != nil { - return nil, newBifrostOperationError(schemas.ErrProviderRequest, err, schemas.OpenAI) + return nil, newBifrostOperationError(schemas.ErrProviderRequest, err, providerType) }
395-401: Increase bufio.Scanner buffer to handle large SSE linesDefault 64KB token limit can truncate or error on larger chunks. Bump the max to avoid flaky streams.
- scanner := bufio.NewScanner(resp.Body) + scanner := bufio.NewScanner(resp.Body) + // Increase max token size to handle large SSE data frames + scanner.Buffer(make([]byte, 64*1024), 10*1024*1024)
♻️ Duplicate comments (20)
docs/usage/go-package/README.md (1)
218-218: Grammar polish: rephrase “All 12+ providers setup.”Use a clearer noun phrase.
- - [Provider Configuration](../providers.md) - All 12+ providers setup + - [Provider Configuration](../providers.md) - Configuration for all 12+ providersdocs/usage/go-package/account.md (1)
582-582: Consistency with other docs: prefer “Configuration for all 12+ providers.”Align phrasing with the Go package README.
- - **[🔗 Provider Configuration](../providers.md)** - Settings for all 12+ providers + - **[🔗 Provider Configuration](../providers.md)** - Configuration for all 12+ providersui/app/layout.tsx (1)
24-24: Tighten wording and capitalize MCP in metadata description.Crisper phrasing; fix “mcp” → “MCP”.
- 'Production-ready fastest LLM gateway that connects to 12+ providers through a single API. Get automatic failover, load balancing, mcp support and zero-downtime deployments.', + 'The fastest, production-ready LLM gateway connecting to 12+ providers through a single API. Automatic failover, load balancing, MCP support, and zero-downtime deployments.',docs/quickstart/go-package.md (1)
223-223: Fix Providers link path (points to the wrong location).The Providers page lives under docs/usage/providers.md.
-| **Add all 12+ providers** | [🔗 Providers](../providers.md) | 5 min | +| **Add all 12+ providers** | [🔗 Providers](../usage/providers.md) | 5 min |Run to verify the target exists:
#!/bin/bash # Confirm providers.md lives under docs/usage/ fd -a "providers.md" | sed -n 'p'core/providers/sgl.go (1)
17-36: Add a TODO explaining why pooling is disabled and the intended reintroductionCommented-out pooling is fine given lifetime issues; add a brief TODO to document intent for future reference (ref counting or safer ownership).
Apply within this block:
+// TODO: Pooling is intentionally disabled to avoid response lifetime/ownership issues. +// Reintroduce with reference counting or a safer ownership model when feasible. // // sglResponsePool provides a pool for SGL response objects. // var sglResponsePool = sync.Pool{ // New: func() interface{} { // return &schemas.BifrostResponse{} // }, // }docs/usage/providers.md (1)
475-486: Fix Vision support for Cerebras (text-only models; no image input)Cerebras (e.g., Llama 3.3 70B) is text-only in this integration. Update the Vision cell for Cerebras to ❌.
-| **Vision** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | +| **Vision** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |core/providers/groq.go (1)
17-36: Add a TODO note explaining why pooling is disabled and when to revisit.This mirrors the transitional approach elsewhere; add a brief TODO to document the intent.
-// // groqResponsePool provides a pool for Groq response objects. +// TODO: Revisit response pooling after implementing safe lifetime management (e.g., ref-counting). +// Disabled now to avoid correctness risks during migration away from pooling. +// // groqResponsePool provides a pool for Groq response objects. // var groqResponsePool = sync.Pool{ // New: func() interface{} { // return &schemas.BifrostResponse{} // }, // } // // acquireGroqResponse gets a Groq response from the pool and resets it. // func acquireGroqResponse() *schemas.BifrostResponse { // resp := groqResponsePool.Get().(*schemas.BifrostResponse) // *resp = schemas.BifrostResponse{} // Reset the struct // return resp // } // // releaseGroqResponse returns a Groq response to the pool. // func releaseGroqResponse(resp *schemas.BifrostResponse) { // if resp != nil { // groqResponsePool.Put(resp) // } // }core/providers/azure.go (3)
114-119: Tighten the pre-warm comment and drop the stale commented line.Only the text completion pool is pre-warmed now.
- // Pre-warm response pools + // Pre-warm Azure text completion response pool for range config.ConcurrencyAndBufferSize.Concurrency { - // azureChatResponsePool.Put(&schemas.BifrostResponse{}) azureTextCompletionResponsePool.Put(&AzureTextResponse{}) }
311-315: Fix misleading comment; pooling is no longer used.Reflect direct allocation and remove dead references.
- // Create response object from pool - // response := acquireAzureChatResponse() - // defer releaseAzureChatResponse(response) - - response := &schemas.BifrostResponse{} + // Create response object + response := &schemas.BifrostResponse{}
364-368: Same nit in Embedding path: remove pool references and clarify comment.Keep comments consistent with current behavior.
- // Pre-allocate response structs from pools - // response := acquireAzureChatResponse() - // defer releaseAzureChatResponse(response) - - response := &schemas.BifrostResponse{} + // Create response object + response := &schemas.BifrostResponse{}tests/core-providers/README.md (1)
15-18: Grammar nit and missing env var/test snippet for Cerebras.
- Use a serial comma in the Cerebras descriptor.
- Add CEREBRAS_API_KEY to the Prerequisites env block and a TestCerebras command in the examples, so tests don’t get skipped.
- - **Cerebras** - Llama, Qwen and GPT-OSS models + - **Cerebras** - Llama, Qwen, and GPT-OSS modelsAdd to the “Prerequisites” env block (below existing providers):
# Cerebras export CEREBRAS_API_KEY="your-cerebras-key"Add to “Run Specific Provider Tests”:
# Test only Cerebras go test -v ./tests/core-providers/ -run TestCerebrastransports/go.mod (1)
20-21: Guard the local replace; document intent or block it before releaseThis local replace will shadow the published core module for consumers. If this is intended only for monorepo/local/CI builds, add an inline note and ensure it’s removed/overridden before tagging a release.
Apply this diff to document the intent inline:
+// NOTE: Local monorepo development only. Remove or override this in release tags. replace github.com/maximhq/bifrost/core => ../corecore/providers/ollama.go (3)
17-36: Delete dead, commented-out pooling codeThese commented pool helpers are obsolete after moving to per-request allocations. Remove to reduce noise and avoid confusion.
-// // ollamaResponsePool provides a pool for Ollama response objects. -// var ollamaResponsePool = sync.Pool{ -// New: func() interface{} { -// return &schemas.BifrostResponse{} -// }, -// } - -// // acquireOllamaResponse gets a Ollama response from the pool and resets it. -// func acquireOllamaResponse() *schemas.BifrostResponse { -// resp := ollamaResponsePool.Get().(*schemas.BifrostResponse) -// *resp = schemas.BifrostResponse{} // Reset the struct -// return resp -// } - -// // releaseOllamaResponse returns a Ollama response to the pool. -// func releaseOllamaResponse(resp *schemas.BifrostResponse) { -// if resp != nil { -// ollamaResponsePool.Put(resp) -// } -// }
64-67: Remove pre-warm remnantsPre-warm code is commented and no longer used; safe to delete.
- // // Pre-warm response pools - // for range config.ConcurrencyAndBufferSize.Concurrency { - // ollamaResponsePool.Put(&schemas.BifrostResponse{}) - // }
149-151: Clean up commented pool usage in ChatCompletionThese lines can be removed now that you allocate directly.
- // response := acquireOllamaResponse() - // defer releaseOllamaResponse(response)Also, consider updating the NewOllamaProvider comment that says “sets up response pools” since pooling was removed.
tests/core-providers/cerebras_test.go (1)
22-22: Fix likely model name typo ("llama3.1-8b" → "llama-3.1-8b")This will otherwise cause model-not-found errors.
- TextModel: "llama3.1-8b", + TextModel: "llama-3.1-8b",core/providers/mistral.go (2)
17-36: Add a concise TODO noting pooling is disabled intentionally and how to reintroduce safelyKeeps intent clear for future refactors while aligning with the team’s temporary safety-first approach.
+// TODO: Pooling disabled intentionally for safety; consider reintroducing with explicit lifetime +// management (e.g., reference counting) and thorough tests to avoid reuse/data races. // // mistralResponsePool provides a pool for Mistral response objects. // var mistralResponsePool = sync.Pool{ // New: func() interface{} { // return &schemas.BifrostResponse{} // }, // }
64-67: Document pre-warm pool disablementSmall TODO helps track future reintroduction.
- // for range config.ConcurrencyAndBufferSize.Concurrency { - // mistralResponsePool.Put(&schemas.BifrostResponse{}) - // } + // TODO: Pre-warm disabled along with pooling; re-enable if/when safe pooling returns. + // for range config.ConcurrencyAndBufferSize.Concurrency { + // mistralResponsePool.Put(&schemas.BifrostResponse{}) + // }core/providers/parasail.go (2)
17-36: Add a concise TODO noting pooling is disabled intentionally and conditions to reintroduce
Same rationale as other providers.+// TODO: Pooling disabled intentionally; reintroduce only with robust lifetime management and tests. // // parasailResponsePool provides a pool for Parasail response objects. // var parasailResponsePool = sync.Pool{ // New: func() interface{} { // return &schemas.BifrostResponse{} // }, // }
64-67: Document disabled pre-warm step
Track the intent for future work.- // for range config.ConcurrencyAndBufferSize.Concurrency { - // parasailResponsePool.Put(&schemas.BifrostResponse{}) - // } + // TODO: Pre-warm disabled with pooling; consider re-enabling alongside safe pooling. + // for range config.ConcurrencyAndBufferSize.Concurrency { + // parasailResponsePool.Put(&schemas.BifrostResponse{}) + // }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/core-providers/go.sumis excluded by!**/*.sum
📒 Files selected for processing (30)
README.md(3 hunks)ci/npx/package.json(1 hunks)core/bifrost.go(1 hunks)core/providers/azure.go(4 hunks)core/providers/cerebras.go(1 hunks)core/providers/groq.go(4 hunks)core/providers/mistral.go(4 hunks)core/providers/ollama.go(3 hunks)core/providers/openai.go(5 hunks)core/providers/parasail.go(3 hunks)core/providers/sgl.go(3 hunks)core/providers/vertex.go(3 hunks)core/schemas/bifrost.go(1 hunks)docs/quickstart/go-package.md(1 hunks)docs/usage/go-package/README.md(1 hunks)docs/usage/go-package/account.md(1 hunks)docs/usage/go-package/schemas.md(1 hunks)docs/usage/providers.md(3 hunks)tests/core-providers/README.md(1 hunks)tests/core-providers/cerebras_test.go(1 hunks)tests/core-providers/config/account.go(3 hunks)tests/core-providers/go.mod(3 hunks)transports/README.md(2 hunks)transports/bifrost-http/integrations/utils.go(1 hunks)transports/go.mod(1 hunks)ui/README.md(1 hunks)ui/app/layout.tsx(1 hunks)ui/lib/constants/icons.tsx(2 hunks)ui/lib/constants/logs.ts(2 hunks)ui/lib/types/config.ts(1 hunks)
👮 Files not reviewed due to content moderation or server errors (1)
- ui/lib/constants/icons.tsx
🧰 Additional context used
🧠 Learnings (23)
📓 Common learnings
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#55
File: core/providers/anthropic.go:358-388
Timestamp: 2025-06-04T05:37:59.699Z
Learning: User Pratham-Mishra04 prefers not to extract small code duplications (around 2 lines) into helper functions, considering the overhead not worth it for such minor repetition.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:25-31
Timestamp: 2025-08-13T07:56:05.626Z
Learning: Pratham-Mishra04 prefers temporary solutions that prevent correctness issues (like commenting out problematic defer statements) when planning to implement better long-term solutions (like reference counting) in the future, accepting trade-offs like temporary memory leaks for safety.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#102
File: README.md:62-66
Timestamp: 2025-06-19T17:03:03.639Z
Learning: Pratham-Mishra04 prefers using the implicit 'latest' tag for the maximhq/bifrost Docker image rather than pinning to specific versions.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#143
File: core/mcp.go:155-196
Timestamp: 2025-07-08T15:33:47.698Z
Learning: Pratham-Mishra04 prefers not to add explanatory comments for obvious code patterns, such as the unlock/lock strategy around network I/O operations, considering them self-explanatory to experienced developers.
📚 Learning: 2025-08-12T20:32:24.408Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: tests/core-providers/cerebras_test.go:1-1
Timestamp: 2025-08-12T20:32:24.408Z
Learning: In the Bifrost project, test files in tests/core-providers directory should use `package tests` rather than `package main`, as confirmed by Pratham-Mishra04 in PR #267 for tests/core-providers/cerebras_test.go.
Applied to files:
tests/core-providers/cerebras_test.gotests/core-providers/go.mod
📚 Learning: 2025-08-05T20:43:59.593Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#214
File: core/providers/azure.go:49-50
Timestamp: 2025-08-05T20:43:59.593Z
Learning: In core/providers/azure.go, the azureTextCompletionResponsePool should use AzureTextResponse type, not schemas.BifrostResponse, to maintain consistency with the acquireAzureTextResponse() and releaseAzureTextResponse() functions that work with *AzureTextResponse objects.
Applied to files:
tests/core-providers/cerebras_test.gocore/providers/openai.gocore/providers/sgl.gocore/providers/groq.gocore/providers/mistral.gocore/providers/azure.gocore/providers/ollama.gocore/providers/parasail.gocore/providers/cerebras.gocore/providers/vertex.go
📚 Learning: 2025-06-16T04:55:11.886Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/automatic_function_calling.go:22-22
Timestamp: 2025-06-16T04:55:11.886Z
Learning: In the Bifrost test suite (tests/core-providers), parallel tests using t.Parallel() are not being implemented currently. The team plans to add parallel test execution in future enhancements.
Applied to files:
tests/core-providers/cerebras_test.go
📚 Learning: 2025-06-20T16:21:18.912Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#103
File: .github/workflows/transport-dependency-update.yml:53-75
Timestamp: 2025-06-20T16:21:18.912Z
Learning: In the bifrost repository's transport dependency update workflow, when updating the core dependency to a new version using `go get`, the go.mod and go.sum files will always change in normal operation, making the safety check for changes more of a defensive programming practice rather than handling a common scenario.
Applied to files:
transports/go.modtests/core-providers/go.mod
📚 Learning: 2025-06-27T17:07:39.462Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#135
File: docs/core-package.md:105-116
Timestamp: 2025-06-27T17:07:39.462Z
Learning: In Go, when importing a package, the identifier used to access it is determined by the `package` declaration in the Go source files, not the directory name. For the Bifrost project, the core directory files declare `package bifrost`, so importing `"github.com/maximhq/bifrost/core"` automatically makes it accessible as `bifrost.Init`, `bifrost.Ptr`, etc., without needing an explicit alias.
Applied to files:
transports/go.modtests/core-providers/go.mod
📚 Learning: 2025-06-15T14:34:29.401Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: core/providers/ollama.go:59-61
Timestamp: 2025-06-15T14:34:29.401Z
Learning: The `CheckAndSetDefaults()` method on `*schemas.ProviderConfig` in Go does not return any error - it has a void return type and only sets default values on the configuration struct.
Applied to files:
tests/core-providers/config/account.go
📚 Learning: 2025-06-04T09:22:18.123Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#54
File: core/schemas/bifrost.go:46-49
Timestamp: 2025-06-04T09:22:18.123Z
Learning: In core/schemas/bifrost.go, the RequestInput struct uses ChatCompletionInput *[]BifrostMessage (pointer-to-slice) rather than []BifrostMessage to properly represent union type semantics. For text completion requests, ChatCompletionInput should be nil to indicate "no chat payload at all", while for chat completion requests it should be non-nil (even if empty slice). This distinguishes between different request types rather than just empty vs non-empty chat messages.
Applied to files:
core/providers/openai.gocore/providers/cerebras.go
📚 Learning: 2025-07-29T16:10:52.088Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#196
File: core/providers/openai.go:180-183
Timestamp: 2025-07-29T16:10:52.088Z
Learning: In the Bifrost provider architecture, `handleProviderResponse` is a utility function that only parses and returns raw response data when the `sendBackRawResponse` flag is true. It's the responsibility of each individual provider (OpenAI, Anthropic, etc.) to conditionally set `response.ExtraFields.RawResponse` using the returned raw response data based on their `sendBackRawResponse` flag. This represents a separation of concerns where the utility handles parsing and the provider handles response object construction.
Applied to files:
core/providers/openai.gocore/providers/sgl.gocore/providers/groq.gocore/providers/mistral.gocore/providers/azure.gocore/providers/ollama.gocore/providers/parasail.gocore/providers/vertex.go
📚 Learning: 2025-07-22T12:14:08.826Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#177
File: transports/bifrost-http/handlers/completions.go:248-264
Timestamp: 2025-07-22T12:14:08.826Z
Learning: In transports/bifrost-http/handlers/completions.go, for speech completion requests, the user prefers to let the provider handle ResponseFormat validation rather than validating supported audio formats ("mp3", "opus", "aac", "flac") at the HTTP transport layer. This follows the architectural pattern of delegating domain-specific validation to providers rather than duplicating validation logic in the transport layer.
Applied to files:
core/providers/openai.go
📚 Learning: 2025-08-12T20:27:20.757Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:18-23
Timestamp: 2025-08-12T20:27:20.757Z
Learning: In Bifrost providers, it's intentional to reuse compatible response types across different providers (e.g., using AzureTextResponse in Cerebras provider) when they follow the same struct, rather than creating provider-specific aliases. This avoids unnecessary type duplication for identical API response structures.
Applied to files:
core/providers/openai.gocore/bifrost.gocore/providers/sgl.goREADME.mdcore/providers/groq.gocore/providers/mistral.gocore/providers/azure.gocore/providers/ollama.gocore/providers/parasail.gocore/providers/cerebras.gocore/providers/vertex.go
📚 Learning: 2025-08-08T07:07:08.828Z
Learnt from: TejasGhatte
PR: maximhq/bifrost#219
File: core/providers/openai.go:459-459
Timestamp: 2025-08-08T07:07:08.828Z
Learning: In core/providers/utils.go, processAndSendResponse and processAndSendError now take exactly five parameters: (ctx, postHookRunner, response|err, responseChan, logger). All providers (OpenAI, Anthropic, Cohere, Bedrock) must call these with the logger as the fifth argument; a repo-wide check in PR maximhq/bifrost#219 confirmed this is already consistent.
Applied to files:
core/providers/openai.go
📚 Learning: 2025-06-16T03:54:48.005Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/go.mod:38-38
Timestamp: 2025-06-16T03:54:48.005Z
Learning: The `core-providers-test` module in `tests/core-providers/` is an internal testing module that will never be consumed as a dependency by external projects, so the replace directive pointing to `../../core` is acceptable for local development and testing purposes.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:12:05.427Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/complete_end_to_end.go:39-41
Timestamp: 2025-06-16T04:12:05.427Z
Learning: In the Bifrost system, error returns are of type `BifrostError` rather than the standard Go `error` interface. Therefore, use `require.Nilf(t, err, ...)` instead of `require.NoError(t, err)` when checking for errors in Bifrost function calls.
Applied to files:
tests/core-providers/go.mod
📚 Learning: 2025-06-16T04:13:55.437Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#81
File: tests/core-providers/scenarios/end_to_end_tool_calling.go:43-45
Timestamp: 2025-06-16T04:13:55.437Z
Learning: In the Bifrost codebase, errors returned from client methods like ChatCompletionRequest are of type BifrostError, not the standard error interface. For testing these errors, use require.Nilf instead of require.NoErrorf since BifrostError doesn't work with the standard error assertion methods.
Applied to files:
tests/core-providers/go.modcore/providers/azure.go
📚 Learning: 2025-06-09T16:46:32.018Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/anthropic/router.go:19-33
Timestamp: 2025-06-09T16:46:32.018Z
Learning: In the GenericRouter (transports/bifrost-http/integrations), ResponseFunc is not called if the BifrostResponse parameter is nil, providing built-in protection against nil response marshaling.
Applied to files:
core/providers/sgl.gocore/providers/groq.gocore/providers/mistral.go
📚 Learning: 2025-06-15T14:24:49.882Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#83
File: core/providers/mistral.go:168-170
Timestamp: 2025-06-15T14:24:49.882Z
Learning: In the Bifrost codebase, performance is prioritized over defensive copying for HTTP service operations. Specifically, shallow slice assignments in provider response handling are acceptable due to object pool reset patterns and JSON unmarshaling behavior that minimize practical data corruption risks.
Applied to files:
core/providers/sgl.gocore/providers/groq.gocore/providers/mistral.gocore/providers/cerebras.go
📚 Learning: 2025-07-17T08:56:59.907Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#169
File: docs/usage/http-transport/openapi.json:1352-1358
Timestamp: 2025-07-17T08:56:59.907Z
Learning: In the Bifrost project, the fallback format has been updated from object structure {"provider": "...", "model": "..."} to a simpler string format "provider/model" (e.g., "anthropic/claude-3-sonnet-20240229"). The current OpenAPI schema correctly reflects this new format.
Applied to files:
README.md
📚 Learning: 2025-06-15T16:05:13.489Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#84
File: transports/bifrost-http/main.go:2-2
Timestamp: 2025-06-15T16:05:13.489Z
Learning: For the Bifrost project, HTTP transport integration routers for new providers (like Mistral and Ollama) are implemented in separate PRs from the core provider support, following a focused PR strategy.
Applied to files:
README.md
📚 Learning: 2025-06-16T06:56:55.290Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#88
File: core/providers/mistral.go:170-176
Timestamp: 2025-06-16T06:56:55.290Z
Learning: When JSON unmarshaling into pooled structs, slice fields like `Choices []schemas.BifrostResponseChoice` get fresh heap memory allocations from `json.Unmarshal()`. The slice data is not part of the pooled struct's memory, so defensive copying is unnecessary. Resetting pooled structs with `*resp = ResponseType{}` only clears slice headers, not the underlying data.
Applied to files:
core/providers/parasail.gocore/providers/cerebras.go
📚 Learning: 2025-06-09T17:33:52.234Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#65
File: transports/bifrost-http/integrations/utils.go:169-173
Timestamp: 2025-06-09T17:33:52.234Z
Learning: The ChatCompletionRequest method in the Bifrost client follows a contract where the result parameter will never be nil if the error parameter is nil. This means when error checking passes (err == nil), the result is guaranteed to be valid and can be safely used without additional nil checks.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-08-13T07:56:05.626Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#267
File: core/providers/cerebras.go:25-31
Timestamp: 2025-08-13T07:56:05.626Z
Learning: Pratham-Mishra04 prefers temporary solutions that prevent correctness issues (like commenting out problematic defer statements) when planning to implement better long-term solutions (like reference counting) in the future, accepting trade-offs like temporary memory leaks for safety.
Applied to files:
core/providers/cerebras.go
📚 Learning: 2025-07-08T18:30:08.258Z
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#141
File: core/bifrost.go:198-272
Timestamp: 2025-07-08T18:30:08.258Z
Learning: Pratham-Mishra04 follows a pattern of implementing core functionality first and deferring non-critical improvements (like race condition fixes, optimizations) to later PRs. This is a reasonable development approach that prioritizes getting the main feature working before addressing edge cases.
Applied to files:
core/providers/cerebras.go
🧬 Code Graph Analysis (15)
tests/core-providers/cerebras_test.go (3)
tests/core-providers/config/setup.go (1)
SetupTest(51-59)tests/core-providers/config/account.go (2)
ComprehensiveTestConfig(38-46)TestScenarios(17-35)core/schemas/bifrost.go (1)
Cerebras(52-52)
tests/core-providers/config/account.go (4)
core/schemas/bifrost.go (1)
Cerebras(52-52)core/schemas/account.go (1)
Key(8-16)transports/bifrost-http/lib/config.go (1)
ProviderConfig(33-39)core/schemas/provider.go (5)
ProviderConfig(87-94)NetworkConfig(32-40)DefaultNetworkConfig(43-48)ConcurrencyAndBufferSize(51-54)DefaultConcurrencyAndBufferSize(57-60)
transports/bifrost-http/integrations/utils.go (1)
core/schemas/bifrost.go (1)
Cerebras(52-52)
core/providers/openai.go (3)
core/schemas/bifrost.go (2)
BifrostResponse(376-389)OpenAI(41-41)ui/lib/constants/logs.ts (1)
Provider(76-76)core/schemas/provider.go (1)
Provider(132-151)
core/schemas/bifrost.go (1)
ui/lib/types/config.ts (1)
ModelProvider(4-16)
core/bifrost.go (1)
core/schemas/bifrost.go (1)
Cerebras(52-52)
core/providers/sgl.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
ui/lib/types/config.ts (1)
core/schemas/bifrost.go (1)
ModelProvider(38-38)
core/providers/groq.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/mistral.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/azure.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/ollama.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/parasail.go (1)
core/schemas/bifrost.go (1)
BifrostResponse(376-389)
core/providers/cerebras.go (3)
ui/lib/types/config.ts (5)
NetworkConfig(54-61)ProviderConfig(81-87)ConcurrencyAndBufferSize(64-67)ModelProvider(4-16)Key(43-51)core/schemas/bifrost.go (3)
ModelProvider(38-38)Cerebras(52-52)BifrostResponse(376-389)ui/lib/constants/logs.ts (1)
Provider(76-76)
core/providers/vertex.go (1)
core/schemas/bifrost.go (2)
Vertex(46-46)BifrostResponse(376-389)
🪛 LanguageTool
docs/usage/go-package/README.md
[grammar] ~218-~218: There might be a problem here.
Context: ...n](../providers.md) - All 12+ providers setup ### Development - Logging - Debug visibilit...
(QB_NEW_EN_MERGED_MATCH)
tests/core-providers/README.md
[grammar] ~18-~18: Add a comma
Context: ...** - OSS models - Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Te...
(QB_NEW_EN_OTHER_ERROR_IDS_22)
[grammar] ~18-~18: Use correct spacing
Context: ... Cerebras - Llama, Qwen and GPT-OSS models ## 🏃♂️ Running Tests ### Development wit...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
README.md
[grammar] ~13-~13: Use correct spacing
Context: ...d zero-downtime deployments in under 30 seconds. 🚀 Just launched: Native MCP (Model ...
(QB_NEW_EN_OTHER_ERROR_IDS_5)
[grammar] ~78-~78: There might be a mistake here.
Context: ... to build AI applications that never go down](#the-fastest-way-to-build-ai-applications-that-never-go-down) - [⚡ Quickstart (30 seconds)](#-quickstart-...
(QB_NEW_EN_OTHER)
🔇 Additional comments (37)
transports/README.md (1)
295-295: LGTM: provider count reflects 12+The feature link correctly advertises 12+ providers.
README.md (3)
78-78: TOC entry looks correctAnchor target matches the section header; no issues.
251-251: LGTM: “12+” reflected in “I want to understand…” sectionAccurate and consistent with provider additions.
13-13: Confirmed: Provider count updated to 12+ with no remaining “10+ providers” referencesThe README headline now accurately reflects “12+ providers” and a quick grep found no stale “10+ providers” occurrences. Messaging is consistent across the PR and documentation—LGTM.
ui/README.md (1)
84-84: LGTM: added Cerebras to supported providersMatches core/types/icons updates.
core/schemas/bifrost.go (1)
52-53: LGTM: Cerebras integration verified across the codebaseAll routing, factory switches, validations, tests, and documentation include the new
Cerebrasprovider:
- core/schemas/bifrost.go: ModelProvider enum updated
- core/bifrost.go (lines 690–691): provider factory switch handles
schemas.Cerebras- core/providers/cerebras.go: full provider implementation present
- transports/bifrost-http/integrations/utils.go (line 645):
ValidProvidersincludesschemas.Cerebras- tests/core-providers: configuration cases and
cerebras_test.gocover Cerebras- transports/README.md & tests/core-providers/README.md: documentation updated
No further changes needed—approving code changes.
core/providers/sgl.go (1)
155-158: LGTM: allocate fresh response instead of using poolMatches the current provider pattern and avoids lifetime issues until pooling is reintroduced.
tests/core-providers/go.mod (2)
10-11: Local replace for core module is appropriate in this internal test moduleThis replace is consistent with the internal testing workflow and learnings for this repo. Good call for exercising local provider changes without publishing a new core version.
36-39: No changes needed aftergo mod tidyI ran
go mod tidyintests/core-providersand confirmed that neithergo.modnorgo.sumchanged. The indirect dependencies are correctly declared and no further updates are required.docs/usage/providers.md (2)
502-503: Audio: marking Cerebras as Not Available is correctAudio Speech and Transcription are correctly set to N/A for Cerebras.
475-481: Docs accuracy confirmed: Text Completion flags match implementationVerified that only Anthropic, Azure, Bedrock, and Cerebras have a
TextCompletionimplementation (all others return an unsupported‐operation error). The table in docs/usage/providers.md correctly reflects this. No changes needed.ui/lib/types/config.ts (1)
4-16: UI union extended with 'cerebras' — looks correctModelProvider now includes 'cerebras', aligning UI configs with the new backend constant. This prevents narrowing issues in discriminated unions and Record<ModelProvider, …> maps.
core/providers/groq.go (1)
188-196: LGTM: Streaming headers setup is clear and OpenAI-compatible.ui/lib/constants/logs.ts (3)
1-14: LGTM: Added 'cerebras' to providers list.Matches backend addition and provider label mapping below.
18-28: LGTM: REQUEST_TYPES export is consistent with labels/colors.Keys are covered in REQUEST_TYPE_LABELS and REQUEST_TYPE_COLORS.
42-43: LGTM: Provider label for Cerebras added.core/bifrost.go (1)
690-692: Cerebras key requirements verifiedproviderRequiresKey( schemas.Cerebras ) returns true and canProviderKeyValueBeEmpty( schemas.Cerebras ) returns false, which matches the intended behavior—no changes required.
transports/bifrost-http/integrations/utils.go (1)
645-646: LGTM: Cerebras added to ValidProvidersValidProviders now recognizes schemas.Cerebras, enabling prefix-based parsing (e.g., "cerebras/llama-3.3-70b") via ParseModelString.
Note: GetProviderFromModel still defaults to OpenAI for unrecognized models without a provider prefix. If you expect model strings like "llama-3.3-70b" without a "cerebras/" prefix, ensure callers set Provider explicitly or use the prefix to avoid misrouting. Do you want a targeted pattern match (e.g., strictly "cerebras/") added for Cerebras? Given overlap with Groq/Bedrock "llama" models, avoiding heuristic matches is reasonable.
tests/core-providers/config/account.go (3)
73-74: Add Cerebras to configured providers: OKProvider enumeration now includes schemas.Cerebras. Matches the PR objective.
288-291: Default config for Cerebras: OKUsing DefaultNetworkConfig and DefaultConcurrencyAndBufferSize is consistent with other providers.
172-179: Key wiring for Cerebras: OK; consider env guard in testsKey sourcing from CEREBRAS_API_KEY mirrors other providers. If the env var isn’t set, tests may fail with 401. If desired, add a test skip helper when the key is empty to produce clearer output.
Example guard (in the Cerebras test, not necessarily here):
- If os.Getenv("CEREBRAS_API_KEY") == "", t.Skip("CEREBRAS_API_KEY not set")
⛔ Skipped due to learnings
Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#82 File: tests/transports-integrations/tests/integrations/test_litellm.py:97-115 Timestamp: 2025-06-16T09:16:15.634Z Learning: In the Bifrost integration tests, the user prefers tests to fail hard when API keys are missing rather than using skip_if_no_api_key decorators for graceful skipping. This applies to LiteLLM tests that depend on OpenAI API keys.tests/core-providers/cerebras_test.go (1)
1-1: Package name is correct per repository conventionConfirmed: tests under tests/core-providers should use package tests.
core/providers/mistral.go (3)
145-149: LGTM: switched to per-request allocationDirect allocation removes object lifetime hazards introduced by pooling and matches the PR-wide pattern.
156-166: LGTM: provider/params/raw-response set consistentlyProvider identity and optional raw response are correctly propagated.
241-245: LGTM: embeddings path allocation and lifecycleConsistent with chat path. Optional raw response and params handling look correct.
core/providers/parasail.go (2)
145-149: LGTM: direct response allocation replaces poolingMatches repository direction and reduces subtle reuse issues.
156-167: LGTM: provider identity, params, and optional raw response setDownstream consumers should get uniform metadata.
core/providers/openai.go (2)
148-169: LGTM: per-request allocation and provider identity setNon-streaming chat now sets ExtraFields.Provider and respects sendBackRawResponse.
283-295: LGTM: embeddings path follows the same allocation and provider metadata patternConsistent with ChatCompletion and other providers.
core/providers/vertex.go (3)
279-284: LGTM: provider identity and RawResponse gating in Claude branchAccurately tags responses and avoids unnecessary raw payloads.
293-314: LGTM: direct allocation for non-Claude and consistent metadataMatches repo-wide pattern; optional raw response and params are handled correctly.
68-72: Integer range loops over int are valid in Go 1.22+
TheConcurrencyfield is indeed anint, and as of Go 1.22 the language supportsfor range non an integer to iterate from 0 to n-1. This pattern is already used consistently throughout the codebase, so no change is necessary. Ensure your go.mod specifies Go 1.22 or later to avoid any compilation issues.core/providers/cerebras.go (5)
166-173: LGTM: safe pooling usage for text response.Acquiring from the pool and deferring release is fine here since you only build a new BifrostResponse and don’t return pooled objects.
199-213: LGTM: defensively copying Usage avoids pointer-to-pooled-memory bugs.Copying Usage out of the pooled AzureTextResponse before release prevents dangling references. Good attention to detail.
275-279: Good: avoid returning pooled objects for chat completions.Allocating a fresh BifrostResponse for ChatCompletion eliminates use-after-release risks observed in earlier iterations.
75-79: Confirm MaxConnsPerHost aligns with intended knob (BufferSize vs Concurrency).You set MaxConnsPerHost from BufferSize. In some providers, this uses Concurrency. Verify this matches your capacity planning to avoid connection starvation or overcommit.
If needed, update to:
- MaxConnsPerHost: config.ConcurrencyAndBufferSize.BufferSize, + MaxConnsPerHost: config.ConcurrencyAndBufferSize.Concurrency,or keep as-is if BufferSize is the intended cap.
86-90: Fix compile-time error: cannot range over int (pre-warm loop).The loop uses range over an integer, which doesn't compile in Go. Switch to a standard counted for-loop.
Apply this diff:
- // Pre-warm response pools - for range config.ConcurrencyAndBufferSize.Concurrency { - // cerebrasChatResponsePool.Put(&schemas.BifrostResponse{}) - cerebrasTextResponsePool.Put(&AzureTextResponse{}) - } + // Pre-warm response pools + for i := 0; i < config.ConcurrencyAndBufferSize.Concurrency; i++ { + // cerebrasChatResponsePool.Put(&schemas.BifrostResponse{}) + cerebrasTextResponsePool.Put(&AzureTextResponse{}) + }⛔ Skipped due to learnings
Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#214 File: core/providers/azure.go:49-50 Timestamp: 2025-08-05T20:43:59.593Z Learning: In core/providers/azure.go, the azureTextCompletionResponsePool should use AzureTextResponse type, not schemas.BifrostResponse, to maintain consistency with the acquireAzureTextResponse() and releaseAzureTextResponse() functions that work with *AzureTextResponse objects.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.23 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable.Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#84 File: core/providers/ollama.go:68-72 Timestamp: 2025-06-15T14:31:59.649Z Learning: Go 1.22 introduced the ability to range over integers. The syntax `for range n` where n is an integer will iterate from 0 to n-1, making `for range config.ConcurrencyAndBufferSize.Concurrency` valid syntax for iterating a specific number of times without needing an explicit counter variable. This feature requires Go module version 1.22 or later.
| "name": "@maximhq/bifrost", | ||
| "version": "1.0.4", | ||
| "description": "High-performance AI gateway CLI - connect to 10+ providers through a single API", | ||
| "description": "High-performance AI gateway CLI - connect to 12+ providers through a single API", |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Minor copy edit for the CLI description.
Stylistic: em dash and “via” read cleaner.
- "description": "High-performance AI gateway CLI - connect to 12+ providers through a single API",
+ "description": "High-performance AI gateway CLI — connect to 12+ providers via a single API",Optional: add “cerebras” to keywords for discoverability (no diff shown since it’s outside the changed line).
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "description": "High-performance AI gateway CLI - connect to 12+ providers through a single API", | |
| "description": "High-performance AI gateway CLI — connect to 12+ providers via a single API", |
🤖 Prompt for AI Agents
In ci/npx/package.json around line 4, the CLI description should be copy-edited:
replace "connect to 12+ providers through a single API" with "connect to 12+
providers — via a single API" (use an em dash and "via" for cleaner style). Also
optionally add "cerebras" to the keywords array for discoverability by adding it
to the existing keyword list.
| // // azureChatResponsePool provides a pool for Azure chat response objects. | ||
| // var azureChatResponsePool = sync.Pool{ | ||
| // New: func() interface{} { | ||
| // return &schemas.BifrostResponse{} | ||
| // }, | ||
| // } | ||
|
|
||
| // // acquireAzureChatResponse gets an Azure chat response from the pool and resets it. | ||
| // func acquireAzureChatResponse() *schemas.BifrostResponse { | ||
| // resp := azureChatResponsePool.Get().(*schemas.BifrostResponse) | ||
| // *resp = schemas.BifrostResponse{} // Reset the struct | ||
| // return resp | ||
| // } | ||
|
|
||
| // // releaseAzureChatResponse returns an Azure chat response to the pool. | ||
| // func releaseAzureChatResponse(resp *schemas.BifrostResponse) { | ||
| // if resp != nil { | ||
| // azureChatResponsePool.Put(resp) | ||
| // } | ||
| // } | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Optional: Add a brief TODO above the commented chat pooling to document intent.
Consistent with other providers, a short note helps explain the temporary state.
-// // azureChatResponsePool provides a pool for Azure chat response objects.
+// TODO: Revisit Azure chat response pooling after safe lifetime management (e.g., ref-counting).
+// Disabled now to avoid correctness risks during migration away from pooling.
+// // azureChatResponsePool provides a pool for Azure chat response objects.
// var azureChatResponsePool = sync.Pool{
// New: func() interface{} {
// return &schemas.BifrostResponse{}
// },
// }
// // acquireAzureChatResponse gets an Azure chat response from the pool and resets it.
// func acquireAzureChatResponse() *schemas.BifrostResponse {
// resp := azureChatResponsePool.Get().(*schemas.BifrostResponse)
// *resp = schemas.BifrostResponse{} // Reset the struct
// return resp
// }
// // releaseAzureChatResponse returns an Azure chat response to the pool.
// func releaseAzureChatResponse(resp *schemas.BifrostResponse) {
// if resp != nil {
// azureChatResponsePool.Put(resp)
// }
// }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // // azureChatResponsePool provides a pool for Azure chat response objects. | |
| // var azureChatResponsePool = sync.Pool{ | |
| // New: func() interface{} { | |
| // return &schemas.BifrostResponse{} | |
| // }, | |
| // } | |
| // // acquireAzureChatResponse gets an Azure chat response from the pool and resets it. | |
| // func acquireAzureChatResponse() *schemas.BifrostResponse { | |
| // resp := azureChatResponsePool.Get().(*schemas.BifrostResponse) | |
| // *resp = schemas.BifrostResponse{} // Reset the struct | |
| // return resp | |
| // } | |
| // // releaseAzureChatResponse returns an Azure chat response to the pool. | |
| // func releaseAzureChatResponse(resp *schemas.BifrostResponse) { | |
| // if resp != nil { | |
| // azureChatResponsePool.Put(resp) | |
| // } | |
| // } | |
| // TODO: Revisit Azure chat response pooling after safe lifetime management (e.g., ref-counting). | |
| // Disabled now to avoid correctness risks during migration away from pooling. | |
| // // azureChatResponsePool provides a pool for Azure chat response objects. | |
| // var azureChatResponsePool = sync.Pool{ | |
| // New: func() interface{} { | |
| // return &schemas.BifrostResponse{} | |
| // }, | |
| // } | |
| // // acquireAzureChatResponse gets an Azure chat response from the pool and resets it. | |
| // func acquireAzureChatResponse() *schemas.BifrostResponse { | |
| // resp := azureChatResponsePool.Get().(*schemas.BifrostResponse) | |
| // *resp = schemas.BifrostResponse{} // Reset the struct | |
| // return resp | |
| // } | |
| // // releaseAzureChatResponse returns an Azure chat response to the pool. | |
| // func releaseAzureChatResponse(resp *schemas.BifrostResponse) { | |
| // if resp != nil { | |
| // azureChatResponsePool.Put(resp) | |
| // } | |
| // } |
🤖 Prompt for AI Agents
In core/providers/azure.go around lines 53 to 73, the commented-out chat
response pooling code lacks a brief TODO explaining why it’s commented and the
intended future behavior; add a single-line TODO comment above the block noting
that pooling is temporarily disabled, why (e.g., stability/bug/perf reason), and
what should be done to re-enable it (e.g., revisit thread-safety or response
struct changes), keeping the note short and consistent with other providers'
TODO style.
| // // cerebrasChatResponsePool provides a pool for Cerebras chat response objects. | ||
| // var cerebrasChatResponsePool = sync.Pool{ | ||
| // New: func() interface{} { | ||
| // return &schemas.BifrostResponse{} | ||
| // }, | ||
| // } | ||
|
|
||
| // // acquireCerebrasChatResponse gets a Cerebras response from the pool and resets it. | ||
| // func acquireCerebrasChatResponse() *schemas.BifrostResponse { | ||
| // resp := cerebrasChatResponsePool.Get().(*schemas.BifrostResponse) | ||
| // *resp = schemas.BifrostResponse{} // Reset the struct | ||
| // return resp | ||
| // } | ||
|
|
||
| // // releaseCerebrasChatResponse returns a Cerebras response to the pool. | ||
| // func releaseCerebrasChatResponse(resp *schemas.BifrostResponse) { | ||
| // if resp != nil { | ||
| // cerebrasChatResponsePool.Put(resp) | ||
| // } | ||
| // } |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Remove or clearly annotate commented-out chat pooling code.
The commented pool and helpers create noise. Either remove them or leave a TODO referencing the planned ref-counted pooling to avoid confusion.
Option A (remove dead/commented code):
-// // cerebrasChatResponsePool provides a pool for Cerebras chat response objects.
-// var cerebrasChatResponsePool = sync.Pool{
-//
-// New: func() interface{} {
-//
-// return &schemas.BifrostResponse{}
-//
-// },
-//
-// }
-
-// // acquireCerebrasChatResponse gets a Cerebras response from the pool and resets it.
-// func acquireCerebrasChatResponse() *schemas.BifrostResponse {
-//
-// resp := cerebrasChatResponsePool.Get().(*schemas.BifrostResponse)
-//
-// *resp = schemas.BifrostResponse{} // Reset the struct
-//
-// return resp
-//
-// }
-
-// // releaseCerebrasChatResponse returns a Cerebras response to the pool.
-// func releaseCerebrasChatResponse(resp *schemas.BifrostResponse) {
-//
-// if resp != nil {
-//
-// cerebrasChatResponsePool.Put(resp)
-//
-// }
-//
-// }Option B (keep, but annotate intent):
-// // cerebrasChatResponsePool provides a pool for Cerebras chat response objects.
+// // cerebrasChatResponsePool provides a pool for Cerebras chat response objects.
+// // Temporarily disabled to avoid returning pooled objects to callers.
+// // TODO(maxim/bifrost#xxxx): Reintroduce with ref-counted pooling to ensure safety.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // // cerebrasChatResponsePool provides a pool for Cerebras chat response objects. | |
| // var cerebrasChatResponsePool = sync.Pool{ | |
| // New: func() interface{} { | |
| // return &schemas.BifrostResponse{} | |
| // }, | |
| // } | |
| // // acquireCerebrasChatResponse gets a Cerebras response from the pool and resets it. | |
| // func acquireCerebrasChatResponse() *schemas.BifrostResponse { | |
| // resp := cerebrasChatResponsePool.Get().(*schemas.BifrostResponse) | |
| // *resp = schemas.BifrostResponse{} // Reset the struct | |
| // return resp | |
| // } | |
| // // releaseCerebrasChatResponse returns a Cerebras response to the pool. | |
| // func releaseCerebrasChatResponse(resp *schemas.BifrostResponse) { | |
| // if resp != nil { | |
| // cerebrasChatResponsePool.Put(resp) | |
| // } | |
| // } |
🤖 Prompt for AI Agents
In core/providers/cerebras.go around lines 25 to 44 the commented-out sync.Pool
and helper functions create dead code noise; either remove the entire commented
block or replace it with a single clear TODO comment explaining why the pool is
commented out and the intended future approach (e.g., "TODO: implement
ref-counted pooling — see ISSUE/PR # or design doc X; owner: @username"), and
ensure the TODO includes reference links and an owner to avoid confusion.
| // Initialize streaming HTTP client | ||
| streamClient := &http.Client{ | ||
| Timeout: time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds), | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Streaming timeout may prematurely cut off SSE responses.
Setting a finite Timeout on streamClient can abort long-running streams. Consider using no timeout (0) or a separate, higher streaming timeout.
Would you like to make streaming timeout configurable (e.g., NetworkConfig.StreamTimeoutSeconds) or default to 0 for SSE?
🏁 Script executed:
#!/bin/bash
# Locate the NetworkConfig struct and its fields
rg "type NetworkConfig" -A20 .
# Check if any existing stream-specific timeout is defined
rg -n "StreamTimeout" .Length of output: 2178
🏁 Script executed:
#!/bin/bash
# Display context around streamClient in core/providers/cerebras.go
rg -n "streamClient" -B5 -A5 core/providers/cerebras.goLength of output: 1855
Separate streaming timeout from general request timeout
The current code applies DefaultRequestTimeoutInSeconds to both regular and streaming HTTP clients, which risks aborting long-lived SSE streams. To avoid unintended timeouts, introduce a dedicated stream timeout (defaulting to 0 for no timeout) and use it when initializing streamClient.
Suggested changes:
• Add StreamTimeoutInSeconds int (JSON tag stream_timeout_in_seconds) to NetworkConfig, documenting that 0 means no timeout.
• Initialize DefaultNetworkConfig.StreamTimeoutInSeconds to 0.
• In core/providers/cerebras.go, set streamClient.Timeout to config.NetworkConfig.StreamTimeoutInSeconds instead of the default request timeout.
Example diff:
--- a/core/schemas/provider.go
+++ b/core/schemas/provider.go
@@ type NetworkConfig struct {
+ // StreamTimeoutInSeconds specifies the timeout for streaming requests (0 = no timeout)
+ StreamTimeoutInSeconds int `json:"stream_timeout_in_seconds,omitempty"`
// DefaultRequestTimeoutInSeconds applies to non‐streaming requests
DefaultRequestTimeoutInSeconds int `json:"default_request_timeout_in_seconds"`
MaxRetries int
@@ var DefaultNetworkConfig = NetworkConfig{
+ StreamTimeoutInSeconds: 0,
DefaultRequestTimeoutInSeconds: DefaultRequestTimeoutInSeconds,
MaxRetries: DefaultMaxRetries,
RetryBackoffInitial: DefaultRetryBackoffInitial,--- a/core/providers/cerebras.go
+++ b/core/providers/cerebras.go
@@ 81,84c81,86
- // Initialize streaming HTTP client
- streamClient := &http.Client{
- Timeout: time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds),
- }
+ // Initialize streaming HTTP client with dedicated stream timeout (0 = no timeout for SSE)
+ streamClient := &http.Client{
+ Timeout: time.Second * time.Duration(config.NetworkConfig.StreamTimeoutInSeconds),
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Initialize streaming HTTP client | |
| streamClient := &http.Client{ | |
| Timeout: time.Second * time.Duration(config.NetworkConfig.DefaultRequestTimeoutInSeconds), | |
| } | |
| // Initialize streaming HTTP client with dedicated stream timeout (0 = no timeout for SSE) | |
| streamClient := &http.Client{ | |
| Timeout: time.Second * time.Duration(config.NetworkConfig.StreamTimeoutInSeconds), | |
| } |
🤖 Prompt for AI Agents
In core/providers/cerebras.go around lines 81 to 84, the streaming HTTP client
currently reuses DefaultRequestTimeoutInSeconds which can abort long-lived
streams; add a new int field StreamTimeoutInSeconds
`json:"stream_timeout_in_seconds"` to the NetworkConfig (document that 0 means
no timeout), set DefaultNetworkConfig.StreamTimeoutInSeconds = 0, and change
streamClient.Timeout to use time.Second *
time.Duration(config.NetworkConfig.StreamTimeoutInSeconds) so streaming uses the
dedicated timeout (0 = no timeout).
| // Configure proxy if provided | ||
| client = configureProxy(client, config.ProxyConfig, logger) | ||
|
|
||
| // Set default BaseURL if not provided | ||
| if config.NetworkConfig.BaseURL == "" { | ||
| config.NetworkConfig.BaseURL = "https://api.cerebras.ai" | ||
| } | ||
| config.NetworkConfig.BaseURL = strings.TrimRight(config.NetworkConfig.BaseURL, "/") | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Ensure proxies are applied to the streaming client as well.
Only the fasthttp client is proxy-aware. Without configuring streamClient, SSE will ignore proxy settings and break in proxied environments.
Apply this diff within the current block:
// Configure proxy if provided
client = configureProxy(client, config.ProxyConfig, logger)
+ // Apply HTTP proxy to streaming client as well
+ if config.ProxyConfig != nil && config.ProxyConfig.URL != "" {
+ parsed, err := url.Parse(config.ProxyConfig.URL)
+ if err != nil {
+ logger.Warn(fmt.Sprintf("invalid proxy URL for streaming client: %v", err))
+ } else {
+ if transport, ok := streamClient.Transport.(*http.Transport); ok && transport != nil {
+ transport.Proxy = http.ProxyURL(parsed)
+ } else {
+ streamClient.Transport = &http.Transport{Proxy: http.ProxyURL(parsed)}
+ }
+ }
+ }And add this import at the top of the file:
import "net/url"If you’re tracking this across all providers, fine to land in a follow-up; this is the local fix.
🤖 Prompt for AI Agents
In core/providers/cerebras.go around lines 92 to 100, the code only applies
proxy configuration to the primary fasthttp client, leaving the streaming SSE
client unconfigured so SSE ignores proxies; call configureProxy on the
streamClient as well (e.g. streamClient = configureProxy(streamClient,
config.ProxyConfig, logger)) inside the same block where you configure client,
and add import "net/url" at the top of the file as requested so URL handling for
proxy config works; keep the existing BaseURL defaulting and TrimRight logic
unchanged.
| schemas.Parasail, // Parasail models | ||
| schemas.SGLang, // SGLang models | ||
| schemas.Cerebras, // Cerebras models | ||
| } |
There was a problem hiding this comment.
Fix provider constant: use schemas.SGL (not schemas.SGLang) in code snippets
The codebase uses schemas.SGL (see transports/bifrost-http/integrations/utils.go), so the snippet with schemas.SGLang won’t compile as-is. Replace SGLang with SGL.
- schemas.Parasail, // Parasail models
- schemas.SGLang, // SGLang models
- schemas.Cerebras, // Cerebras models
+ schemas.Parasail, // Parasail models
+ schemas.SGL, // SGLang models
+ schemas.Cerebras, // Cerebras models📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| schemas.Parasail, // Parasail models | |
| schemas.SGLang, // SGLang models | |
| schemas.Cerebras, // Cerebras models | |
| } | |
| schemas.Parasail, // Parasail models | |
| schemas.SGL, // SGLang models | |
| schemas.Cerebras, // Cerebras models | |
| } |
🤖 Prompt for AI Agents
docs/usage/go-package/schemas.md around lines 804 to 807: the snippet uses the
non-existent constant schemas.SGLang which will not compile; replace
schemas.SGLang with schemas.SGL in the list so it matches the rest of the
codebase (transports/bifrost-http/integrations/utils.go) and imports, ensuring
the provider constant references the correct identifier.
| | **Parasail** | GPT OSS, Llama, Qwen | Enterprise AI platform | ✅ | | ||
| | **SGLang** | Qwen | Enterprise AI platform | ✅ | | ||
| | **Cerebras** | Llama 3.3 70B | Enterprise AI platform | ✅ | |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Minor docs clarity: call out OpenAI-compatible API for Cerebras
Since Cerebras is integrated via an OpenAI-compatible API in this PR, reflect that in the “Features” column for precision.
-| **Cerebras** | Llama 3.3 70B | Enterprise AI platform | ✅ |
+| **Cerebras** | Llama 3.3 70B | OpenAI-compatible API | ✅ |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | **Parasail** | GPT OSS, Llama, Qwen | Enterprise AI platform | ✅ | | |
| | **SGLang** | Qwen | Enterprise AI platform | ✅ | | |
| | **Cerebras** | Llama 3.3 70B | Enterprise AI platform | ✅ | | |
| | **Parasail** | GPT OSS, Llama, Qwen | Enterprise AI platform | ✅ | | |
| | **SGLang** | Qwen | Enterprise AI platform | ✅ | | |
| | **Cerebras** | Llama 3.3 70B | OpenAI-compatible API | ✅ | |
🤖 Prompt for AI Agents
In docs/usage/providers.md around lines 18 to 20, the Features column for the
Cerebras row doesn't indicate that it is accessed via an OpenAI-compatible API;
update that cell to mention "OpenAI-compatible API" (or similar wording like
"OpenAI-compatible API (Enterprise AI platform)") so the table accurately
reflects integration method for Cerebras.
| Scenarios: config.TestScenarios{ | ||
| TextCompletion: true, | ||
| SimpleChat: true, | ||
| ChatCompletionStream: true, | ||
| MultiTurnConversation: true, | ||
| ToolCalls: true, | ||
| MultipleToolCalls: true, | ||
| End2EndToolCalling: true, | ||
| AutomaticFunctionCall: true, | ||
| ImageURL: false, | ||
| ImageBase64: false, | ||
| MultipleImages: false, | ||
| CompleteEnd2End: true, | ||
| ProviderSpecific: false, | ||
| }, |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Verify enabled scenarios against Cerebras capabilities
If the Cerebras provider doesn’t implement TextCompletion or tool/function calling, these scenarios will fail. Consider disabling non-supported scenarios or adding a SkipReason.
Run this to confirm Cerebras.TextCompletion support and tool-calls handling:
🏁 Script executed:
#!/bin/bash
# 1) Find Cerebras provider implementation
fd -a cerebras.go | sed -n '1p' | xargs -I{} sh -c 'echo "File: {}"; sed -n "1,200p" "{}"'
# 2) Check if TextCompletion is implemented or returns unsupported
rg -n $'func \\(provider \\*CerebrasProvider\\) TextCompletion\\(' -A 8
# 3) Check if ChatCompletion path maps tool_calls (OpenAI-compatible) which tests rely on
rg -n $'ChatCompletion\\(' -A 50 core/providers/cerebras.go
# 4) Optionally, scan for "unsupportedOperationError" in Cerebras provider
rg -n 'unsupportedOperationError\\("text completion", "cerebras"\\)' core/providers/cerebras.goLength of output: 10006
Here’s a deeper dive to confirm tool/function–calling and streaming support in the Cerebras provider:
🏁 Script executed:
#!/bin/bash
# 1) Locate and display implementation of prepareOpenAIChatRequest to see if it wires up functions
rg -n "func prepareOpenAIChatRequest" -A 200 .
# 2) Check whether CerebrasProvider implements ChatCompletionStream (streaming)
rg -n "func (provider \\*CerebrasProvider\\) ChatCompletionStream" -A 20 core/providersLength of output: 13056
Disable unsupported streaming scenario in Cerebras tests
We’ve confirmed that:
TextCompletionand OpenAI‐style tool/function calls are implemented inCerebrasProvider.- There is no
ChatCompletionStreammethod onCerebrasProvider, so enablingChatCompletionStreamin tests will cause failures.
Please update tests/core-providers/cerebras_test.go to disable or skip the streaming scenario. For example:
--- a/tests/core-providers/cerebras_test.go
+++ b/tests/core-providers/cerebras_test.go
@@ Scenarios: config.TestScenarios{
- ChatCompletionStream: true,
+ ChatCompletionStream: false, // CerebrasProvider does not support streaming
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Scenarios: config.TestScenarios{ | |
| TextCompletion: true, | |
| SimpleChat: true, | |
| ChatCompletionStream: true, | |
| MultiTurnConversation: true, | |
| ToolCalls: true, | |
| MultipleToolCalls: true, | |
| End2EndToolCalling: true, | |
| AutomaticFunctionCall: true, | |
| ImageURL: false, | |
| ImageBase64: false, | |
| MultipleImages: false, | |
| CompleteEnd2End: true, | |
| ProviderSpecific: false, | |
| }, | |
| Scenarios: config.TestScenarios{ | |
| TextCompletion: true, | |
| SimpleChat: true, | |
| ChatCompletionStream: false, // CerebrasProvider does not support streaming | |
| MultiTurnConversation: true, | |
| ToolCalls: true, | |
| MultipleToolCalls: true, | |
| End2EndToolCalling: true, | |
| AutomaticFunctionCall: true, | |
| ImageURL: false, | |
| ImageBase64: false, | |
| MultipleImages: false, | |
| CompleteEnd2End: true, | |
| ProviderSpecific: false, | |
| }, |
🤖 Prompt for AI Agents
In tests/core-providers/cerebras_test.go around lines 23 to 37, the test
scenarios enable ChatCompletionStream but CerebrasProvider does not implement a
ChatCompletionStream method; change the Scenarios config to disable streaming by
setting ChatCompletionStream: false (or removing that flag) so the streaming
scenario is not run; keep other scenario flags as-is and save the file to
prevent streaming tests from failing.
| | ----------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------- | | ||
| | **🖥️ Built-in Web UI** | Visual configuration, live monitoring, request logs, and analytics | Open `http://localhost:8080` after startup | | ||
| | **🔄 Multi-Provider Support** | OpenAI, Anthropic, Azure, Bedrock, Vertex, Cohere, Mistral, Ollama, Groq, Parasail, SGLang | [Provider Setup](../docs/usage/providers.md) | | ||
| | **🔄 Multi-Provider Support** | OpenAI, Anthropic, Azure, Bedrock, Vertex, Cohere, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras | [Provider Setup](../docs/usage/providers.md) | |
There was a problem hiding this comment.
🧹 Nitpick (assertive)
Provider list update is correct; consider consistent naming across docs
List looks accurate and includes Cerebras and SGLang. Minor naming nit for consistency with UI/docs elsewhere:
- Use "Azure OpenAI" (vs "Azure")
- Use "AWS Bedrock" (vs "Bedrock")
- Use "Google Vertex AI" (vs "Vertex")
Apply this tweak for consistency:
-| **🔄 Multi-Provider Support** | OpenAI, Anthropic, Azure, Bedrock, Vertex, Cohere, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras | [Provider Setup](../docs/usage/providers.md) |
+| **🔄 Multi-Provider Support** | OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Cohere, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras | [Provider Setup](../docs/usage/providers.md) |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | **🔄 Multi-Provider Support** | OpenAI, Anthropic, Azure, Bedrock, Vertex, Cohere, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras | [Provider Setup](../docs/usage/providers.md) | | |
| | **🔄 Multi-Provider Support** | OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex AI, Cohere, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras | [Provider Setup](../docs/usage/providers.md) | |
🤖 Prompt for AI Agents
In transports/README.md around line 55, the provider names in the table are
inconsistent with other docs/UI; update the entries to use the consistent names:
change "Azure" to "Azure OpenAI", "Bedrock" to "AWS Bedrock", and "Vertex" to
"Google Vertex AI" so the table reads accordingly and matches the rest of the
documentation.
💡 Verification agent
🧩 Analysis chain
Docs potentially outdated on fallback format
Separate from this line: the Quick Start “Multi-Provider Fallbacks” example still uses the old object format. Per project docs, fallbacks are now string entries like "provider/model". Consider updating that example to the new string format.
Updated example outside this hunk for reference:
"fallbacks": [
"anthropic/claude-3-5-sonnet-20241022",
"bedrock/anthropic.claude-3-sonnet-20240229-v1:0"
]To spot other places still using the old object format:
🏁 Script executed:
#!/bin/bash
# Find fallback examples that use object entries with "provider"/"model" keys
rg -n -A 4 -B 2 $'"fallbacks"\\s*:\\s*\\[' | sed -n 'p;n;n;n;n'
rg -n $'"provider"\\s*:\\s*"[^"]+"\\s*,\\s*"model"\\s*:\\s*"[^"]+"' || trueLength of output: 1138
Update fallback examples to string format in transports/README.md
The object-style fallbacks in your Quick Start docs still use { "provider": …, "model": … }. They should be simple strings "provider/model".
Files to update:
- transports/README.md, lines 103–104
Suggested diff:
- {"provider": "anthropic", "model": "claude-3-5-sonnet-20241022"},
- {"provider": "bedrock", "model": "anthropic.claude-3-sonnet-20240229-v1:0"}
+ "anthropic/claude-3-5-sonnet-20241022",
+ "bedrock/anthropic.claude-3-sonnet-20240229-v1:0"Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In transports/README.md around lines 103–104, the Quick Start fallback examples
use object-style JSON like { "provider": ..., "model": ... } but should be
simple string fallbacks "provider/model"; update those examples to use the
string format (including quotes and the slash) wherever the object form appears,
keeping surrounding text the same.
| Manage all your AI providers from a unified interface: | ||
|
|
||
| - **Supported Providers**: OpenAI, Azure OpenAI, Anthropic, AWS Bedrock, Cohere, Google Vertex AI, Mistral, Ollama, Groq, Parasail, SGLang | ||
| - **Supported Providers**: OpenAI, Azure OpenAI, Anthropic, AWS Bedrock, Cohere, Google Vertex AI, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Outdated count earlier: “Configure 8+ AI providers” should match 12+
At line 12, “Provider Management — Configure 8+ AI providers” is now stale. Update to “12+” to stay consistent with the project-wide messaging.
Suggested edit (outside this hunk):
- From: “Configure 8+ AI providers...”
- To: “Configure 12+ AI providers...”
🏁 Script executed:
#!/bin/bash
# Check for other stale numeric counts in UI docs
rg -n $'\\b8\\+\\s+AI\\s+providers\\b' ui || trueLength of output: 152
Update UI README provider count to 12+
The list at line 12 currently reads “Configure 8+ AI providers”, but the “Supported Providers” section now lists 12 distinct providers. Please update it accordingly.
• ui/README.md:12
- **⚙️ Provider Management** - Configure 8+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.)
+ **⚙️ Provider Management** - Configure 12+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - **Supported Providers**: OpenAI, Azure OpenAI, Anthropic, AWS Bedrock, Cohere, Google Vertex AI, Mistral, Ollama, Groq, Parasail, SGLang, Cerebras | |
| **⚙️ Provider Management** - Configure 12+ AI providers (OpenAI, Azure, Anthropic, Bedrock, etc.) |
🤖 Prompt for AI Agents
In ui/README.md around line 12 and line 84, the summary count "Configure 8+ AI
providers" is outdated because the Supported Providers list now contains 12
entries; update the text at line 12 to "Configure 12+ AI providers" (or
"Configure 12 AI providers" if you prefer exact) so the README provider count
matches the Supported Providers section and ensure any other occurrences of "8+"
in the README are updated for consistency.
…267) ## Add Cerebras provider support This PR adds support for Cerebras as a new AI provider in Bifrost, increasing our provider count from 10+ to 12+. Cerebras offers Llama 3.3 70B and other models through an OpenAI-compatible API. ## Changes - Added new Cerebras provider implementation in `core/providers/cerebras.go` - Updated provider enums and constants to include Cerebras - Added Cerebras to the provider list in UI components and documentation - Created comprehensive tests for the Cerebras provider - Updated README and documentation to reflect the new provider count (12+ instead of 10+) ## Type of change - [x] Feature - [x] Documentation ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] UI (Next.js) - [x] Docs ## How to test 1. Set up a Cerebras API key in your environment: ```sh export CEREBRAS_API_KEY=your_api_key_here ``` 2. Run the provider tests: ```sh cd tests/core-providers go test -v -run TestCerebras ``` 3. Test with the HTTP transport by adding a Cerebras provider configuration: ```json { "provider": "cerebras", "keys": [ { "value": "your_api_key_here", "models": ["llama-3.3-70b"] } ] } ``` ## Breaking changes - [x] No ## Related issues Enhances provider coverage as requested in various community discussions. ## Security considerations The implementation follows the same security patterns as other providers, with proper API key handling. ## Checklist - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI)
2 participants
Add Cerebras provider support
This PR adds support for Cerebras as a new AI provider in Bifrost, increasing our provider count from 10+ to 12+. Cerebras offers Llama 3.3 70B and other models through an OpenAI-compatible API.
Changes
core/providers/cerebras.goType of change
Affected areas
How to test
export CEREBRAS_API_KEY=your_api_key_here{ "provider": "cerebras", "keys": [ { "value": "your_api_key_here", "models": ["llama-3.3-70b"] } ] }Breaking changes
Related issues
Enhances provider coverage as requested in various community discussions.
Security considerations
The implementation follows the same security patterns as other providers, with proper API key handling.
Checklist