diff --git a/docs/architecture/framework/model-catalog.mdx b/docs/architecture/framework/model-catalog.mdx index 7f37162d15..26b138e070 100644 --- a/docs/architecture/framework/model-catalog.mdx +++ b/docs/architecture/framework/model-catalog.mdx @@ -321,7 +321,7 @@ You can reload the pricing configuration at runtime if you need to change the pr newConfig := &modelcatalog.Config{ PricingSyncInterval: 12 * time.Hour, } -err := modelCatalog.ReloadPricing(ctx, newConfig) +err := modelCatalog.UpdateSyncConfig(ctx, newConfig) ``` ## Error Handling and Fallbacks diff --git a/docs/changelogs/v1.5.0-prerelease2.mdx b/docs/changelogs/v1.5.0-prerelease2.mdx index 2a860a273b..6b7964f7ee 100644 --- a/docs/changelogs/v1.5.0-prerelease2.mdx +++ b/docs/changelogs/v1.5.0-prerelease2.mdx @@ -49,24 +49,8 @@ description: "v1.5.0-prerelease2 changelog - 2026-04-08" - **Data Race Fix** — Fix race in data reading from fasthttp request for integrations - **Model Listing** — Unify /api/models and /api/models/details listing behavior - -**v1.5.0 contains multiple breaking changes.** See the [v1.5.0 Migration Guide](/migration-guides/v1.5.0) for full before/after examples and a migration checklist. - - -## Breaking Changes in This Release - -This prerelease introduces 3 additional breaking changes on top of those in prerelease1. See the **[v1.5.0 Migration Guide](/migration-guides/v1.5.0)** for full before/after examples, automatic migration details, and a step-by-step checklist. - -| # | Breaking Change | Affected | -|---|---|---| -| [9](/migration-guides/v1.5.0#breaking-change-9-provider-deployments-removed-migrate-to-aliases) | Provider `deployments` removed — migrate Azure, Bedrock, Vertex, and Replicate deployment maps to the unified top-level `aliases` field | `config.json`, REST API, Go SDK | -| [10](/migration-guides/v1.5.0#breaking-change-10-go-sdk-extrafields-model-fields-renamed) | Go SDK: `ExtraFields.ModelRequested` replaced by `OriginalModelRequested` + `ResolvedModelUsed` | Go SDK | -| [11](/migration-guides/v1.5.0#breaking-change-11-go-sdk-streamaccumulatorresult-field-renamed) | Go SDK: `StreamAccumulatorResult.Model` replaced by `RequestedModel` + `ResolvedModel` | Go SDK | - ---- -- feat: add model alias support — map model names to provider-specific identifiers per key - feat: add Fireworks AI as a first-class provider (thanks [@ivanetchart](https://github.com/ivanetchart)!) - feat: add realtime provider interfaces, schemas, and engine hooks - feat: add session log storage and realtime request normalization @@ -89,7 +73,6 @@ This prerelease introduces 3 additional breaking changes on top of those in prer -- feat: add model alias storage and encryption in key config - feat: add per-user OAuth consent flow with identity selection and MCP authentication - feat: add access profiles for fine-grained permission control - feat: add user level OAuth for MCP gateway diff --git a/docs/changelogs/v1.5.0-prerelease3.mdx b/docs/changelogs/v1.5.0-prerelease3.mdx new file mode 100644 index 0000000000..b85f049bfc --- /dev/null +++ b/docs/changelogs/v1.5.0-prerelease3.mdx @@ -0,0 +1,94 @@ +--- +title: "v1.5.0-prerelease3" +description: "v1.5.0-prerelease3 changelog - 2026-04-13" +--- + + + ```bash + npx -y @maximhq/bifrost --transport-version v1.5.0-prerelease3 + ``` + + + ```bash + docker pull maximhq/bifrost:v1.5.0-prerelease3 + docker run -p 8080:8080 maximhq/bifrost:v1.5.0-prerelease3 + ``` + + + + +## ✨ Features + +- **OAuth MCP** - add next-step hints to OAuth MCP client creation response +- **Azure passthrough** - added azure passthrough support +- **272k token tier** - add 272k token tier pricing support in pricing +- **Flex and priority tier support** - added flex and priority tier support in pricing + +## 🐞 Fixed + +- **Streaming Post-Hook Race** — Fix race condition where fasthttp RequestCtx could be recycled before transport post-hooks complete in streaming goroutines; eagerly captures request/response snapshots before handler returns +- **Async User Values** — Propagate user values through all async inference handlers and job submissions +- **Trace Completer Safety** — Refactor trace completer to accept transport logs as parameter instead of reading from potentially recycled context +- **Async Log Store Exceptions** — Fix exception handling in async log store jobs +- **Model Alias Tracking** — Split ModelRequested into OriginalModelRequested and ResolvedModelUsed for accurate model alias resolution tracking +- **MCP Tool Discovery** — Add discovered tools and tool name mapping columns to MCP clients + + + +- refactor: split ModelRequested into OriginalModelRequested and ResolvedModelUsed for model alias tracking +- refactor: simplify Azure passthrough by removing redundant config nil checks +- refactor: simplify Mistral error parsing signature +- fix: carry ProviderResponseHeaders through text completion response conversion + + + +- feat: add MCP client discovered tools and tool name mapping migration +- fix: exception handling in async log store jobs +- refactor: model catalog Init API to use SetShouldSyncGate method +- refactor: rename DefaultPricingSyncInterval to DefaultSyncInterval + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + + +- chore: upgraded core to v1.5.2 and framework to v1.3.2 + + diff --git a/docs/docs.json b/docs/docs.json index e9dbee41d9..0a537b322c 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -489,6 +489,7 @@ "item": "Open Source", "icon": "rocket", "pages": [ + "changelogs/v1.5.0-prerelease3", "changelogs/v1.5.0-prerelease2", "changelogs/v1.5.0-prerelease1", "changelogs/v1.4.22", diff --git a/docs/enterprise/setting-up-okta.mdx b/docs/enterprise/setting-up-okta.mdx index 7b6f25f0bc..6435459f7c 100644 --- a/docs/enterprise/setting-up-okta.mdx +++ b/docs/enterprise/setting-up-okta.mdx @@ -13,7 +13,7 @@ This guide walks you through configuring Okta as your identity provider for Bifr - An Okta organization with admin access - Bifrost Enterprise deployed and accessible - The redirect URI for your Bifrost instance (e.g., `https://your-bifrost-domain.com/login`) - +- Ensure you have created all the [roles in Bifrost](/enterprise/rbac) that you are aiming to map to with Okta. --- ## Step 1: Create an OIDC Application @@ -71,39 +71,12 @@ Configure the following settings for your application: --- -## Step 3: Configure Authorization Server (optional) +## Step 3: Create Custom Role Attribute -The default authorization server (`/oauth2/default`) is available to all Okta plans and **supports custom claims**, including role claims. The API Access Management paid add-on is only required to create additional custom authorization servers beyond the default. +You can use both roles and/or groups for assigning roles to users. You can learn more about [RBAC](/enterprise/rbac) docs. Roles take precedence over groups in role assignment. -Bifrost uses Okta's Authorization Server to issue tokens. You have three options: - -1. **Use `/oauth2/default` with role claims (recommended)** — Complete Steps 4-7 to configure custom role claims on the default authorization server. This enables automatic RBAC synchronization. - -2. **Use `/oauth2/default` without role claims** — Skip Steps 4-7. The first user to sign in automatically receives the Admin role and can manage RBAC for all subsequent users through the Bifrost dashboard. - -3. **Skip Step 3 entirely** — Authorization is not configured through Okta. You'll need an alternative authentication mechanism. - -### Configuring the Authorization Server - -1. Navigate to **Security** → **API** -2. Click on **Authorization Servers** - - - Okta Authorization Servers - - -3. Note the **Issuer URI** for your authorization server (e.g., `https://your-domain.okta.com/oauth2/default`) - - -The Issuer URI is used as the `issuerUrl` in your Bifrost configuration. Make sure to use the full URL including `/oauth2/default` (or your custom authorization server path). - - ---- - -## Step 4: Create Custom Role Attribute - To map Okta users to Bifrost roles (Admin, Developer, Viewer), you need to create a custom attribute. 1. Navigate to **Directory** → **Profile Editor** @@ -133,7 +106,7 @@ To map Okta users to Bifrost roles (Admin, Developer, Viewer), you need to creat --- -## Step 5: Add Role Claim to Tokens +## Step 4: Add Role Claim to Tokens Configure the authorization server to include the role in the access token. @@ -164,11 +137,11 @@ If you named your custom attribute differently, update the Value expression acco --- -## Step 6: Configure Groups for Team and Role Synchronization +## Step 5: Configure Groups Bifrost can automatically sync Okta groups for two purposes: - **Team synchronization** — Groups are synced as Bifrost teams -- **Role mapping** — Groups can be mapped to Bifrost roles (Admin, Developer, Viewer) using Group-to-Role Mappings in the Bifrost UI +- **Role mapping** — Groups can be mapped to Bifrost roles (Admin, Developer, Viewer) using Group-to-Role Mappings in the Bifrost UI. ### Create Groups in Okta @@ -191,31 +164,6 @@ Use a consistent naming convention for your groups. This makes it easier to conf ### Add Groups Claim to Tokens -You have two options for configuring the groups claim. Choose the one that best fits your Okta plan and requirements. - -#### Option A: Using App-Level Groups Claim (All Okta Plans) - -This approach configures the groups claim directly in your application's settings and works with all Okta plans, including free tiers. - -1. Navigate to your application's **Sign On** tab -2. Scroll down to the **OpenID Connect ID Token** section -3. Click **Edit** to modify the settings -4. Configure the **Groups claim filter**: - - **Groups claim type**: Filter - - **Groups claim filter**: Set a claim name (e.g., `groups`) and filter condition (e.g., "Starts with" `bifrost-staging`) - - - Application Groups claim configuration - - -5. Click **Save** - - -The filter ensures only relevant groups are included in the token. Adjust the filter condition based on your group naming convention. - - -#### Option B: Using Authorization Server Groups Claim - This approach adds the groups claim through your authorization server, providing more flexibility for complex configurations. 1. Navigate to **Security** → **API** → **Authorization Servers** @@ -235,25 +183,9 @@ Configure the groups claim: 5. Click **Create** -You can also configure an additional groups claim in the application's Sign On settings: - -1. Navigate to your application's **Sign On** tab - - - Application Sign On configuration - - -2. Under **OpenID Connect ID Token**, configure: - - **Groups claim type**: Expression - - **Groups claim expression**: `Arrays.flatten(Groups.startsWith("OKTA", "bifrost", 100))` - - -Adjust the group filter expression based on your naming convention. The example above includes groups starting with "bifrost". - - --- -## Step 7: Assign Users to the Application +## Step 6: Assign Users to the Application 1. Navigate to your application's **Assignments** tab @@ -263,7 +195,9 @@ Adjust the group filter expression based on your naming convention. The example 2. Click **Assign** → **Assign to People** or **Assign to Groups** -3. For each user, set their **bifrostRole**: +### For Assigning Roles + +For each user, set their **bifrostRole** (if you are planning to do role-level mapping): Assign custom role to user @@ -277,6 +211,22 @@ Role claims are available only when you configure custom claims on your authoriz --- +## Step 7: Create API token for bulk user and team sync + +To create an API token, navigate to **Security** → **API** → **Tokens**. + + +Okta API tokens screen + + +1. Click on "Create token" + + + Create token dialog in Okta + + +2. Copy token to be used in the next step. + ## Step 8: Configure Bifrost Now configure Bifrost to use Okta as the identity provider. @@ -297,9 +247,9 @@ Now configure Bifrost to use Okta as the identity provider. 4. Toggle **Enabled** to activate the provider 5. Click **Save Configuration** -### Group-to-Role Mappings (Optional) +### Group-to-Role Mappings -If you configured groups in Okta (Step 6), you can map Okta group names directly to Bifrost roles. This is an alternative to using custom role claims (Steps 4-5) and works with all Okta plans. +If you configured groups in Okta (Step 5), you can map Okta group names directly to Bifrost roles. This is an alternative to using custom role claims (Steps 3-4) and works with all Okta plans. 1. In the User Provisioning configuration, scroll down to **Group-to-Role Mappings** 2. Click **Add Mapping** diff --git a/docs/features/litellm-compat.mdx b/docs/features/litellm-compat.mdx index 51cd26dcd9..490a37efa4 100644 --- a/docs/features/litellm-compat.mdx +++ b/docs/features/litellm-compat.mdx @@ -9,8 +9,10 @@ icon: "train" The LiteLLM compatibility plugin provides two transformations: 1. **Text-to-Chat Conversion** - Automatically converts text completion requests to chat completion format for models that only support chat APIs +2. **Chat-to-Responses Conversion** - Automatically converts chat completion requests to responses format for models that only support responses APIs +3. **Drop Unsupported Params** - Automatically drops unsupported parameters if the model doesn't support them -When either transformation is applied, responses include `extra_fields.litellm_compat: true`. +When either transformation is applied, responses include `extra_fields.converted_request_type: `. If request parameters are dropped, the keys are added in `extra_fields.dropped_compat_plugin_params`. --- @@ -55,6 +57,36 @@ F --> G - `object: "chat.completion"` → `object: "text_completion"` - Usage statistics and metadata are preserved +## 2. Chat-to-Responses Conversion + +Some AI models (like OpenAI o1-pro) only support the responses API and don't support native chat completion endpoints. LiteLLM compatibility mode automatically handles this by: + +1. Checking if the model supports chat completion natively (using the model catalog) +2. If not supported, converting your chat message to responses API format +3. Calling the responses endpoint internally +4. Transforming the response back to chat completion format + + +**Smart Conversion**: The conversion only happens when the model doesn't support chat completions natively. If a model has native chat completion support (like OpenAI's gpt-4 models), Bifrost uses the chat completion endpoint directly without any conversion. + + +This allows you to use a unified chat completion interface across all providers, even those that only support responses API. + +## How It Works + +When LiteLLM compatibility is enabled and you make a chat completion request, Bifrost first checks if the model supports chat completion: + +```mermaid +flowchart LR +A[Chat Completion Request] --> B{Model Supports Chat Completion?} +B -->|Yes| C[Call Chat Completion API] +B -->|No| D[Convert to Responses Message] +D --> E[Call Responses API] +E --> F[Transform Response] +C --> G[Chat Completion Response] +F --> G +``` + ## Enabling LiteLLM Compatibility @@ -63,7 +95,10 @@ F --> G 1. Open the Bifrost dashboard 2. Navigate to **Settings** → **Client Configuration** -3. Enable **LiteLLM Fallbacks** +3. Expand **LiteLLM Compat** and enable the features you need: + - **Convert Text to Chat** — converts text completion requests to chat for models that only support chat + - **Convert Chat to Responses** — converts chat completion requests to responses for models that only support responses + - **Drop Unsupported Params** — drops unsupported parameters based on model catalog allowlist 4. Save your configuration @@ -73,7 +108,11 @@ F --> G ```json { "client_config": { - "enable_litellm_fallbacks": true + "compat": { + "convert_text_to_chat": true, + "convert_chat_to_responses": true, + "should_drop_params": true + } } } ``` @@ -84,9 +123,9 @@ F --> G ## Supported Providers -LiteLLM compatibility mode works with any provider that supports chat completions but lacks native text completion support: +Text completion to chat completion conversion works with any provider that supports chat completions but lacks native text completion support: -| Provider | Native Text Completion | LiteLLM Fallback | +| Provider | Native Text Completion | With Fallback | |----------|----------------------|------------------| | OpenAI (GPT-4, GPT-3.5-turbo) | No | Yes | | Anthropic (Claude) | No | Yes | @@ -95,6 +134,12 @@ LiteLLM compatibility mode works with any provider that supports chat completion | Mistral | No | Yes | | Bedrock | Varies by model | Yes | +Chat completion to responses conversion works with any provider that supports responses but lacks native chat completion support: + +| Provider | Native Chat Completion | With Fallback | +|----------|----------------------|------------------| +| OpenAI (o1-pro) | No | Yes | + ## Behavior Details **Model Capability Detection:** @@ -117,13 +162,19 @@ LiteLLM compatibility mode works with any provider that supports chat completion | Response | `choices[0].message.content` | `choices[0].text` | | Response | `object: "chat.completion"` | `object: "text_completion"` | +### Transformation 2: Chat-to-Responses Conversion + +**Applies to:** Chat completion requests on responses-only models + +| Phase | Original | Transformed | +|-------|----------|-------------| +| Request | Chat message with `role: "user"` | Responses input with `role: "user"` | +| Request | `chat_completion` request type | `responses` request type | ### Metadata Set on Transformed Responses When either transformation is applied: -- `extra_fields.litellm_compat`: Set to `true` -- `extra_fields.provider`: The provider that handled the request - `extra_fields.request_type`: Reflects the original request type - `extra_fields.original_model_requested`: The originally requested model - `extra_fields.resolved_model_used`: The actual provider API identifier used (equals original_model_requested when no alias mapping exists) @@ -131,8 +182,11 @@ When either transformation is applied: ### Error Handling When errors occur on transformed requests: -- `extra_fields.litellm_compat` is set to `true` - Original request type and model are preserved in error metadata +- `extra_fields.converted_request_type`: Set to type of request that was converted to (i.e., `chat_completion` or `responses`) +- `extra_fields.provider`: The provider that handled the request +- `extra_fields.original_model_requested`: The originally requested model +- `extra_fields.dropped_compat_plugin_params`: If any unsupported parameters were dropped, the keys are added here ## What's Preserved @@ -145,7 +199,7 @@ When errors occur on transformed requests: **Good Use Cases:** - Migrating from LiteLLM to Bifrost without code changes -- Maintaining backward compatibility with text completion interfaces +- Maintaining backward compatibility with text completion interfaces or chat completion interfaces - Using a unified API across providers with different capabilities **Consider Alternatives When:** @@ -157,4 +211,4 @@ When errors occur on transformed requests: - [Fallbacks](/features/fallbacks) - Automatic provider failover - [Drop-in Replacement](/features/drop-in-replacement) - Use existing SDKs with Bifrost -- [LiteLLM Integration](/integrations/litellm-sdk) - Using LiteLLM SDK with Bifrost +- [LiteLLM Integration](/integrations/litellm-sdk) - Using LiteLLM SDK with Bifrost \ No newline at end of file diff --git a/docs/migration-guides/v1.5.0.mdx b/docs/migration-guides/v1.5.0.mdx index de384bfeca..7d1933aaac 100644 --- a/docs/migration-guides/v1.5.0.mdx +++ b/docs/migration-guides/v1.5.0.mdx @@ -319,15 +319,6 @@ The database migration runs automatically on startup, migrating existing deploym "gpt-4o-mini": "my-mini-deployment" } }] - "ollama": { - "keys": [ - { - "id": "ollama-local", - "models": ["*"], - "weight": 1.0, - "ollama_key_config": { "url": "http://localhost:11434" } - } - ] } } } @@ -471,6 +462,42 @@ result.ResolvedModel // actual model identifier used by the provider --- +## Opting Out: `version: 1` Compatibility Mode + +If you are not ready to adopt the new deny-by-default semantics, you can add a single field to `config.json` to restore v1.4.x behavior for all allow-list fields loaded from that file: + +```json +{ + "version": 1, + "providers": { ... } +} +``` + +| Value | Behavior | +|---|---| +| `2` (default, omitted) | v1.5.0 semantics — empty = deny all, `["*"]` = allow all | +| `1` | v1.4.x semantics — empty = allow all | + +**What `version: 1` normalizes at startup** (before any other processing): + +| Field | Without `version: 1` | With `version: 1` | +|---|---|---| +| Provider key `models: []` | Deny all models | Allow all models (→ `["*"]`) | +| VK `provider_configs: []` | No providers allowed | All configured providers added with `allowed_models: ["*"]` | +| VK provider config `allowed_models: []` | Deny all models | Allow all models (→ `["*"]`) | +| VK provider config `key_ids: []` | No keys allowed | All keys allowed (→ `key_ids: ["*"]`) | +| VK `mcp_configs: []` | No MCP tools allowed | All configured MCP clients added with `tools_to_execute: ["*"]` | + + +`version: 1` only applies to configuration loaded from `config.json`. Virtual Keys created or updated via the REST API always use v1.5.0 semantics regardless of this setting. The automatic database migration that runs on startup is also unaffected. + + + +`version: 1` is a temporary compatibility shim. Plan to migrate your `config.json` to explicit `["*"]` wildcards and remove the `version` field before the next major release. + + +--- + ## Complete Migration Checklist diff --git a/docs/openapi/schemas/management/config.yaml b/docs/openapi/schemas/management/config.yaml index 2c54b3979d..eaafb3821f 100644 --- a/docs/openapi/schemas/management/config.yaml +++ b/docs/openapi/schemas/management/config.yaml @@ -44,9 +44,24 @@ ClientConfig: max_request_body_size_mb: type: integer description: Maximum request body size in MB - enable_litellm_fallbacks: - type: boolean - description: Whether LiteLLM fallbacks are enabled + compat: + type: object + description: Compat plugin configuration + properties: + convert_text_to_chat: + type: boolean + description: Convert text completion requests to chat + convert_chat_to_responses: + type: boolean + description: Convert chat completion requests to responses + should_drop_params: + type: boolean + description: Drop unsupported parameters based on model catalog + should_convert_params: + type: boolean + default: false + description: Converts model parameter values that are not supported by the model + additionalProperties: false log_retention_days: type: integer description: Number of days to retain logs diff --git a/docs/providers/custom-pricing.mdx b/docs/providers/custom-pricing.mdx index 13c883773f..43931fc93a 100644 --- a/docs/providers/custom-pricing.mdx +++ b/docs/providers/custom-pricing.mdx @@ -268,6 +268,8 @@ Only fields with non-zero values are applied. All values are cost **per unit** i | `output_cost_per_token_batches` | Output token cost for batch requests | | `input_cost_per_token_priority` | Input token cost for priority requests | | `output_cost_per_token_priority` | Output token cost for priority requests | +| `input_cost_per_token_flex` | Input token cost for flex requests | +| `output_cost_per_token_flex` | Output token cost for flex requests | | `input_cost_per_character` | Input cost per character (character-billed models) | ### Token tier costs @@ -277,7 +279,13 @@ Only fields with non-zero values are applied. All values are cost **per unit** i | `input_cost_per_token_above_128k_tokens` | Input cost above 128k context | | `output_cost_per_token_above_128k_tokens` | Output cost above 128k context | | `input_cost_per_token_above_200k_tokens` | Input cost above 200k context | +| `input_cost_per_token_above_200k_tokens_priority` | Input cost above 200k context for priority requests | | `output_cost_per_token_above_200k_tokens` | Output cost above 200k context | +| `output_cost_per_token_above_200k_tokens_priority` | Output cost above 200k context for priority requests | +| `input_cost_per_token_above_272k_tokens` | Input cost above 272k context | +| `input_cost_per_token_above_272k_tokens_priority` | Input cost above 272k context for priority requests | +| `output_cost_per_token_above_272k_tokens` | Output cost above 272k context | +| `output_cost_per_token_above_272k_tokens_priority` | Output cost above 272k context for priority requests | ### Cache costs @@ -287,7 +295,11 @@ Only fields with non-zero values are applied. All values are cost **per unit** i | `cache_read_input_token_cost` | Cost to read a cached input token | | `cache_creation_input_token_cost_above_200k_tokens` | Cache creation above 200k context | | `cache_read_input_token_cost_above_200k_tokens` | Cache read above 200k context | +| `cache_read_input_token_cost_above_200k_tokens_priority` | Cache read above 200k context for priority requests | | `cache_read_input_token_cost_priority` | Priority cache read cost | +| `cache_read_input_token_cost_flex` | Flex cache read cost | +| `cache_read_input_token_cost_above_272k_tokens` | Cache read above 272k context | +| `cache_read_input_token_cost_above_272k_tokens_priority` | Cache read above 272k context for priority requests | | `cache_read_input_image_token_cost` | Cache read cost for image tokens | | `cache_creation_input_audio_token_cost` | Cache creation cost for audio tokens | diff --git a/docs/providers/supported-providers/overview.mdx b/docs/providers/supported-providers/overview.mdx index b3ae42f62f..98d13ffa73 100644 --- a/docs/providers/supported-providers/overview.mdx +++ b/docs/providers/supported-providers/overview.mdx @@ -48,7 +48,7 @@ The following table summarizes which operations are supported by each provider v Some operations are not supported by the downstream provider, and their internal implementation in Bifrost is optional. 🟡 -Like Text completions are not supported by Groq, but Bifrost can emulate them internally using the Chat Completions API. This feature is disabled by default, but it can be enabled by setting the `enable_litellm_fallbacks` flag to `true` in the client configuration. +Like Text completions are not supported by Groq, but Bifrost can emulate them internally using the Chat Completions API. This feature is disabled by default, but it can be enabled by setting `compat.convert_text_to_chat` to `true` in the client configuration. We do not promote using such fallbacks, since text completions and chat completions are fundamentally different. However, this option is available to help users migrating from LiteLLM (which does support these fallbacks).