diff --git a/docs/features/litellm-compat.mdx b/docs/features/litellm-compat.mdx index 490a37efa4..a4e262b080 100644 --- a/docs/features/litellm-compat.mdx +++ b/docs/features/litellm-compat.mdx @@ -11,6 +11,7 @@ The LiteLLM compatibility plugin provides two transformations: 1. **Text-to-Chat Conversion** - Automatically converts text completion requests to chat completion format for models that only support chat APIs 2. **Chat-to-Responses Conversion** - Automatically converts chat completion requests to responses format for models that only support responses APIs 3. **Drop Unsupported Params** - Automatically drops unsupported parameters if the model doesn't support them +4. **Unsupported Parameter Conversion** - Automatically converts unsupported parameters to their supported equivalents When either transformation is applied, responses include `extra_fields.converted_request_type: `. If request parameters are dropped, the keys are added in `extra_fields.dropped_compat_plugin_params`. @@ -94,13 +95,16 @@ F --> G 1. Open the Bifrost dashboard -2. Navigate to **Settings** → **Client Configuration** -3. Expand **LiteLLM Compat** and enable the features you need: +2. Navigate to **Settings** → **Compatibility** +3. There you can enable the features you need: - **Convert Text to Chat** — converts text completion requests to chat for models that only support chat - **Convert Chat to Responses** — converts chat completion requests to responses for models that only support responses - **Drop Unsupported Params** — drops unsupported parameters based on model catalog allowlist + - **Convert Unsupported Params Values** — converts unsupported parameters values to their supported equivalents. See the list of [supported parameters](#supported-parameters) below. 4. Save your configuration +![Compatibility Settings](../media/compatibility-settings.png) + @@ -111,7 +115,8 @@ F --> G "compat": { "convert_text_to_chat": true, "convert_chat_to_responses": true, - "should_drop_params": true + "should_drop_params": true, + "should_convert_params": true } } } @@ -121,6 +126,14 @@ F --> G +## Supported Parameters + +The following parameters are converted to their supported equivalents: + +| Parameter | Original Value | Transformed Value | Provider | +|-----------|----------------|-------------------|----------| +| messages.role | developer | system | Gemini, Vertex, Bedrock | + ## Supported Providers Text completion to chat completion conversion works with any provider that supports chat completions but lacks native text completion support: @@ -188,6 +201,12 @@ When errors occur on transformed requests: - `extra_fields.original_model_requested`: The originally requested model - `extra_fields.dropped_compat_plugin_params`: If any unsupported parameters were dropped, the keys are added here +## Header Overrides + +To enable compat plugins per request, you can use `x-bf-compat` header: +- When `x-bf-compat: true` or `x-bf-compat: ["*"]`, enables all compat plugins options +- When `x-bf-compat: ["",""]`, enables the specified settings (available settings: `convert_text_to_chat`, `convert_chat_to_responses`, `should_drop_params`, `should_convert_params`) + ## What's Preserved - Model selection and fallback chain diff --git a/docs/media/compatibility-settings.png b/docs/media/compatibility-settings.png new file mode 100644 index 0000000000..69b62a60d9 Binary files /dev/null and b/docs/media/compatibility-settings.png differ