Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 22 additions & 3 deletions docs/features/litellm-compat.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ The LiteLLM compatibility plugin provides two transformations:
1. **Text-to-Chat Conversion** - Automatically converts text completion requests to chat completion format for models that only support chat APIs
2. **Chat-to-Responses Conversion** - Automatically converts chat completion requests to responses format for models that only support responses APIs
3. **Drop Unsupported Params** - Automatically drops unsupported parameters if the model doesn't support them
4. **Unsupported Parameter Conversion** - Automatically converts unsupported parameters to their supported equivalents
Comment thread
coderabbitai[bot] marked this conversation as resolved.

When either transformation is applied, responses include `extra_fields.converted_request_type: <transformed_request_type>`. If request parameters are dropped, the keys are added in `extra_fields.dropped_compat_plugin_params`.

Expand Down Expand Up @@ -94,13 +95,16 @@ F --> G
<Tab title="Gateway UI">

1. Open the Bifrost dashboard
2. Navigate to **Settings** → **Client Configuration**
3. Expand **LiteLLM Compat** and enable the features you need:
2. Navigate to **Settings** → **Compatibility**
3. There you can enable the features you need:
- **Convert Text to Chat** — converts text completion requests to chat for models that only support chat
- **Convert Chat to Responses** — converts chat completion requests to responses for models that only support responses
- **Drop Unsupported Params** — drops unsupported parameters based on model catalog allowlist
- **Convert Unsupported Params Values** — converts unsupported parameters values to their supported equivalents. See the list of [supported parameters](#supported-parameters) below.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
4. Save your configuration

![Compatibility Settings](../media/compatibility-settings.png)

</Tab>

<Tab title="Configuration File">
Expand All @@ -111,7 +115,8 @@ F --> G
"compat": {
"convert_text_to_chat": true,
"convert_chat_to_responses": true,
"should_drop_params": true
"should_drop_params": true,
"should_convert_params": true
}
}
}
Expand All @@ -121,6 +126,14 @@ F --> G

</Tabs>

## Supported Parameters

The following parameters are converted to their supported equivalents:

| Parameter | Original Value | Transformed Value | Provider |
|-----------|----------------|-------------------|----------|
| messages.role | developer | system | Gemini, Vertex, Bedrock |

## Supported Providers

Text completion to chat completion conversion works with any provider that supports chat completions but lacks native text completion support:
Expand Down Expand Up @@ -188,6 +201,12 @@ When errors occur on transformed requests:
- `extra_fields.original_model_requested`: The originally requested model
- `extra_fields.dropped_compat_plugin_params`: If any unsupported parameters were dropped, the keys are added here

## Header Overrides

To enable compat plugins per request, you can use `x-bf-compat` header:
- When `x-bf-compat: true` or `x-bf-compat: ["*"]`, enables all compat plugins options
- When `x-bf-compat: ["<setting1>","<setting2>"]`, enables the specified settings (available settings: `convert_text_to_chat`, `convert_chat_to_responses`, `should_drop_params`, `should_convert_params`)

Comment thread
sammaji marked this conversation as resolved.
## What's Preserved

- Model selection and fallback chain
Expand Down
Binary file added docs/media/compatibility-settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading