Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@
"providers/supported-providers/overview",
"providers/supported-providers/anthropic",
"providers/supported-providers/azure",
"providers/supported-providers/azure-model-router",
"providers/supported-providers/bedrock",
"providers/supported-providers/bedrock-mantle",
"providers/supported-providers/cerebras",
Expand Down
129 changes: 129 additions & 0 deletions docs/providers/supported-providers/azure-model-router.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
---
title: "Azure Model Router"
description: "Use Azure model-router deployments with Bifrost - native Chat Completions support and a custom-provider workaround for Responses"
icon: "microsoft"
---

## Overview

Azure model router is an Azure OpenAI deployment family that automatically picks the best underlying model for a request. Bifrost's native Azure provider supports model-router deployments for Chat Completions today.

The Responses API is not yet exposed for model-router deployments through the native Azure provider. Until that lands, you can reach model-router's Responses endpoint directly through your Azure AI Foundry project by configuring **Azure Foundry as a custom provider**, as described below.

<Note>
This page covers model-router specifically. For general endpoint setup, authentication, aliases, and deployment configuration on the native Azure provider, see the main [Azure provider guide](./azure).
</Note>

### Supported operations

| Operation | Support | Notes |
| --- | --- | --- |
| Chat Completions | ✅ | Native Azure provider or Azure Foundry custom provider |
| Chat Completions (stream) | ✅ | Native Azure provider or Azure Foundry custom provider |
| Responses API | ⚠️ | Requires adding Azure Foundry as a custom-provider |
| Responses API (stream) | ⚠️ | Requires adding Azure Foundry as a custom-provider |

---

## Adding Azure Foundry as a custom provider

Azure AI Foundry projects expose an OpenAI-Responses-API-shaped endpoint (`*.ai.azure.com/api/projects/<project>/openai/v1/responses`) that is distinct from the classic Azure OpenAI resource endpoint (`*.openai.azure.com`) used by the native Azure provider. You can reach it by creating a [custom provider](../custom-providers) named e.g. `azure-foundry` with `base_provider_type: "openai"` and a full-URL [request path override](../custom-providers#request-path-overrides) for the `responses` and `responses_stream` request types. A full URL in `request_path_overrides` bypasses `base_url` entirely, so requests go straight to your Azure AI Foundry project's Responses endpoint.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/sh
set -eu
curl -fsSL 'https://learn.microsoft.com/en-us/rest/api/microsoft-foundry/aiproject' |
  grep -F 'services.ai.azure.com/api/projects'

Repository: maximhq/bifrost

Length of output: 35769


🏁 Script executed:

#!/bin/sh
set -eu

file='docs/providers/supported-providers/azure-model-router.mdx'

# Show the relevant sections with line numbers.
sed -n '1,180p' "$file" | cat -n

echo
echo '---'
echo 'Occurrences of ai.azure.com / services.ai.azure.com in this file:'
grep -nE 'ai\.azure\.com|services\.ai\.azure\.com|base_url|request_path_overrides|responses_stream|responses' "$file" || true

Repository: maximhq/bifrost

Length of output: 9228


Use the Foundry project hostname

The examples still point at *.ai.azure.com/api/projects/..., but the Foundry project endpoint uses *.services.ai.azure.com/api/projects/<project>. Update the prose, UI example, and JSON overrides, and either remove the base_url example or call out that the full-URL overrides ignore it.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/providers/supported-providers/azure-model-router.mdx` at line 30, Update
the Azure AI Foundry documentation to use the
*.services.ai.azure.com/api/projects/<project> hostname throughout the prose, UI
example, and JSON request_path_overrides. In the custom provider example, remove
base_url or explicitly state that the full-URL responses and responses_stream
overrides bypass it.


<Tabs group="config-method">
<Tab title="Web UI">

![Custom Provider Configuration Interface](../../media/ui-custom-provider.png)

1. Go to **Providers** in the sidebar and click **Add New Provider**.
2. Name the provider `azure-foundry` (or similar).
3. Set **Base Format** to **OpenAI**.
4. Set **Base URL** to `https://<your-resource>.ai.azure.com/openai`.
5. Under **Allowed Request Types**, toggle on **Responses** and **Responses Stream**.
6. Click the settings icon next to each and enter the full Azure AI Foundry Responses URL in **Custom Path or URL**:
`https://<your-resource>.ai.azure.com/api/projects/<your-project>/openai/v1/responses`
7. Add your Azure API key and save.

</Tab>

<Tab title="API">
Refer to the API documentation for [Provider Keys Management](https://docs.getbifrost.ai/api-reference/providers/create-a-key-for-a-provider).
</Tab>
<Tab title="config.json">

```json
{
"providers": {
"azure-foundry": {
"keys": [
{
"name": "azure-foundry-key-1",
"value": "env.AZURE_API_KEY",
"models": ["*"],
"weight": 1.0
}
],
"network_config": {
"base_url": "https://<your-resource>.ai.azure.com/openai"
},
"custom_provider_config": {
"base_provider_type": "openai",
"allowed_requests": {
"responses": true,
"responses_stream": true
},
"request_path_overrides": {
"responses": "https://<your-resource>.ai.azure.com/api/projects/<your-project>/openai/v1/responses",
"responses_stream": "https://<your-resource>.ai.azure.com/api/projects/<your-project>/openai/v1/responses"
}
}
}
}
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `custom_provider_config.base_provider_type` | string | Yes | Must be `openai` so requests are shaped as OpenAI-compatible calls |
| `custom_provider_config.allowed_requests` | object | No | Restricts this provider to only the request types you enable |
| `custom_provider_config.request_path_overrides` | object | No | Full URL per request type; bypasses `network_config.base_url` |
| `network_config.base_url` | string | No | Used for any request type without a full-URL override |
Comment on lines +84 to +89

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Mark the Responses override as required for this setup.

request_path_overrides is documented as optional, but the page states that the full override is what routes requests to the Azure Foundry Responses endpoint. Without the responses and responses_stream overrides, this workaround has no documented path to that endpoint.

As per path instructions, the field table should make required fields explicit for the documented scenario.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/providers/supported-providers/azure-model-router.mdx` around lines 84 -
89, Update the request_path_overrides entry in the field table to mark it as
required for the documented Azure Foundry Responses setup, and clarify that both
responses and responses_stream overrides must be provided to route requests to
the Responses endpoint. Leave the other field requirements unchanged.

Source: Path instructions


</Tab>
</Tabs>

<Note>
If you also want Chat Completions routed through this same custom provider instead of the native Azure provider, enable `chat_completion` / `chat_completion_stream` in `allowed_requests` and add matching entries in `request_path_overrides`. Otherwise, keep using the native Azure provider for Chat Completions and this custom provider only for Responses.
</Note>

---

## Usage

Call the Responses endpoint using the `azure-foundry` custom provider and your model-router deployment name:

```bash
curl -X POST http://localhost:8080/v1/responses \
-H "Content-Type: application/json" \
-d '{
"model": "azure-foundry/model-router",
"input": "Write a short haiku about gateways."
}'
```
Comment on lines +100 to +111

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add the promised Go SDK example.

The PR objectives call for REST and Go SDK usage examples, but this section only documents curl. Add a Go SDK example using the repository’s established SDK conventions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/providers/supported-providers/azure-model-router.mdx` around lines 100 -
111, Add a Go SDK usage example alongside the existing curl example in the Usage
section, following the repository’s established SDK conventions and using the
same azure-foundry provider, model-router deployment, Responses endpoint, and
input prompt.


<Note>
If you need to inspect provider-specific extra parameters, enable [Send Back Raw Response](/providers/request-options#send-back-raw-response). If you do not want those raw bytes persisted in logs, also set [Store Raw Request/Response](/providers/request-options#store-raw-request/response).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 security Raw Logging Advice Is Reversed

When a user wants to inspect raw responses without persisting them, this note tells them to set Store Raw Request/Response, which enables internal log persistence rather than preventing it. This can store sensitive provider data against the user's intent; the option should remain disabled, and the corrected link should use the generated section fragment.

Suggested change
If you need to inspect provider-specific extra parameters, enable [Send Back Raw Response](/providers/request-options#send-back-raw-response). If you do not want those raw bytes persisted in logs, also set [Store Raw Request/Response](/providers/request-options#store-raw-request/response).
If you need to inspect provider-specific extra parameters, enable [Send Back Raw Response](/providers/request-options#send-back-raw-response). If you do not want those raw bytes persisted in logs, leave [Store Raw Request/Response](/providers/request-options#store-raw-requestresponse) disabled.

</Note>

---

## Limitations

- Chat Completions for model-router deployments works through the native Azure provider; no extra configuration is needed.
- Responses support for model-router requires the custom-provider workaround on this page until native support is added to the Azure provider.
- This page does not change Azure authentication or endpoint setup for the native Azure provider - see the [Azure provider guide](./azure) for that.

## Related docs

- [Azure](./azure)
- [Custom Providers](../custom-providers)
- [Request Options](/providers/request-options)
Loading