Skip to content

chore: update provider management docs #3953

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
akshaydeo merged 6 commits into from
Jun 2, 2026
Merged

chore: update provider management docs #3953

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9efa509
feat: adds support for multiple otel collectors
sammaji May 29, 2026
eb7ebfe
fix: updates config and helm schema
sammaji Jun 1, 2026
747af18
feat: adds tests for multiple otel collectors support
sammaji May 29, 2026
8137423
chore: updated docs for otel plugin
sammaji Apr 14, 2026
48158ed
chore: updated docs for extra params passthrough
sammaji May 29, 2026
c4cac97
chore: update provider management docs
sammaji Jun 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 131 additions & 13 deletions docs/features/observability/otel.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ func main() {
// Initialize OTel plugin
otelPlugin, err := otel.Init(ctx, &otel.Config{
ServiceName: "bifrost",
CollectorURL: "http://localhost:4318",
CollectorURL: "http://localhost:4318/v1/traces",
TraceType: otel.TraceTypeGenAIExtension,
Protocol: otel.ProtocolHTTP,
Headers: map[string]string{
Expand Down Expand Up @@ -153,7 +153,7 @@ For Gateway mode, configure via `config.json`:
"name": "otel",
"config": {
"service_name": "bifrost",
"collector_url": "http://localhost:4318",
"collector_url": "http://localhost:4318/v1/traces",
"trace_type": "genai_extension",
"protocol": "http",
"headers": {
Expand All @@ -165,7 +165,7 @@ For Gateway mode, configure via `config.json`:
}
```

If you need to connect to an OTEL collector that requires TLS, configure `tls_ca_cert`:
If you need to connect to an OTEL collector that requires TLS, configure `tls_ca_cert` and set insecure mode to `false`:

```json
{
Expand All @@ -178,6 +178,7 @@ If you need to connect to an OTEL collector that requires TLS, configure `tls_ca
"collector_url": "localhost:4317",
"trace_type": "genai_extension",
"protocol": "grpc",
"insecure": false,
"tls_ca_cert": "/path/to/your/ca.cert",
"headers": {
"Authorization": "env.OTEL_API_KEY"
Expand All @@ -189,6 +190,65 @@ If you need to connect to an OTEL collector that requires TLS, configure `tls_ca
```

</Tab>
<Tab title="config.json (v1.5.8+)">

For Gateway mode, configure via `config.json`:

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "http://localhost:4318/v1/traces",
"trace_type": "genai_extension",
"protocol": "http",
"headers": {
"Authorization": "env.OTEL_API_KEY"
}
}
]
}
}
]
}
```

If you need to connect to an OTEL collector that requires TLS, configure `tls_ca_cert` and set insecure mode to `false`:

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "localhost:4317",
"trace_type": "genai_extension",
"protocol": "grpc",
"insecure": false,
"tls_ca_cert": "/path/to/your/ca.cert",
"headers": {
"Authorization": "env.OTEL_API_KEY"
}
}
]
}
}
]
}
```
</Tab>

</Tabs>

---
Expand Down Expand Up @@ -221,12 +281,12 @@ services:
tempo:
image: grafana/tempo:latest
container_name: tempo
command: [ "-config.file=/etc/tempo.yaml" ]
command: ["-target=all", "-config.file=/etc/tempo.yaml"]
configs:
- source: tempo-config
target: /etc/tempo.yaml
ports:
- "3200:3200" # tempo HTTP API
- "3200:3200" # Tempo HTTP API
expose:
- "4317" # OTLP gRPC (internal)
volumes:
Expand All @@ -236,21 +296,23 @@ services:
prometheus:
image: prom/prometheus:latest
container_name: prometheus
depends_on:
- otel-collector
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
- "--web.enable-remote-write-receiver"
- "--enable-feature=exemplar-storage"
- "--enable-feature=native-histograms"
ports:
- "9090:9090"
volumes:
- prometheus-data:/prometheus
configs:
- source: prometheus-config
target: /etc/prometheus/prometheus.yml
depends_on:
- otel-collector
restart: unless-stopped

grafana:
Expand All @@ -264,9 +326,8 @@ services:
GF_SECURITY_ADMIN_PASSWORD: admin
GF_AUTH_ANONYMOUS_ENABLED: "true"
GF_AUTH_ANONYMOUS_ORG_ROLE: Viewer
GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: "grafana-pyroscope-app,grafana-exploretraces-app,grafana-metricsdrilldown-app"
GF_PLUGINS_ENABLE_ALPHA: "true"
GF_INSTALL_PLUGINS: ""
GF_FEATURE_TOGGLES_ENABLE: traceqlEditor
ports:
- "4000:3000"
volumes:
Expand Down Expand Up @@ -296,12 +357,12 @@ configs:
namespace: otel
const_labels:
source: otelcol

otlp/tempo:
endpoint: tempo:4317
tls:
insecure: true

debug:
verbosity: detailed

Expand Down Expand Up @@ -346,7 +407,7 @@ configs:
protocols:
grpc:
endpoint: 0.0.0.0:4317

ingester:
max_block_duration: 5m
trace_idle_period: 10s
Expand Down Expand Up @@ -716,7 +777,7 @@ Uses HTTP/1.1 or HTTP/2 with JSON or Protobuf encoding:

```json
{
"collector_url": "http://localhost:4318",
"collector_url": "http://localhost:4318/v1/traces",
"protocol": "http"
}
```
Expand Down Expand Up @@ -778,8 +839,37 @@ The OTel plugin supports **push-based metrics export** via OTLP, which is essent
]
}
```
</Tab>

<Tab title="HTTP Protocol (v1.5+)">

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "http://otel-collector:4318/v1/traces",
"trace_type": "genai_extension",
"protocol": "http",
"metrics_enabled": true,
"metrics_endpoint": "http://otel-collector:4318/v1/metrics",
"metrics_push_interval": 15
}
]
}
}
]
}
```

</Tab>

<Tab title="gRPC Protocol">

```json
Expand All @@ -801,8 +891,36 @@ The OTel plugin supports **push-based metrics export** via OTLP, which is essent
]
}
```
</Tab>

<Tab title="gRPC Protocol (v1.5+)">

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "otel-collector:4317",
"trace_type": "genai_extension",
"protocol": "grpc",
"metrics_enabled": true,
"metrics_endpoint": "otel-collector:4317",
"metrics_push_interval": 15
}
]
}
}
]
}
```
</Tab>

</Tabs>

### Pushed Metrics
Expand Down
31 changes: 27 additions & 4 deletions docs/providers/request-options.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -521,8 +521,13 @@ This flag affects only what is written to the log record (messages, params, tool

Enable passthrough mode for extra parameters. When enabled, any parameters in `extra_params` (or provider-specific extra parameter fields) will be merged directly into the request sent to the provider.

How parameters are collected depends on the entrypoint:

- Standard inference routes such as `/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`, and `/v1/images/generations` collect all unknown top-level JSON fields as extra parameters.
- OpenAI integration routes under `/openai` preserve the OpenAI request shape, so provider-specific fields must be put under `extra_params`.

<Tabs>
<Tab title="Gateway (cURL)">
<Tab title="Standard Inference Routes">
```bash
curl --location 'http://localhost:8080/v1/chat/completions' \
--header 'x-bf-passthrough-extra-params: true' \
Expand All @@ -538,13 +543,31 @@ curl --location 'http://localhost:8080/v1/chat/completions' \
}'
````

</Tab>
<Tab title="OpenAI Integration">
```bash
curl --location 'http://localhost:8080/openai/chat/completions' \
--header 'x-bf-passthrough-extra-params: true' \
--header 'Content-Type: application/json' \
--data '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello!"}],
"extra_params": {
"custom_param": "value",
"nested_param": {
"a": "value",
"b": 123
}
}
}'
````
</Tab>
<Tab title="Go SDK">
```go
ctx := context.Background()
ctx = context.WithValue(ctx, schemas.BifrostContextKeyPassthroughExtraParams, true)
ctx := schemas.NewBifrostContext(context.Background(), schemas.NoDeadline)
ctx.SetValue(schemas.BifrostContextKeyPassthroughExtraParams, true)

response, err := client.ChatCompletionRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), &schemas.BifrostChatRequest{
response, err := client.ChatCompletionRequest(ctx, &schemas.BifrostChatRequest{
Provider: schemas.OpenAI,
Model: "gpt-4o-mini",
Input: messages,
Expand Down
20 changes: 1 addition & 19 deletions docs/providers/supported-providers/anthropic.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,25 +72,7 @@ Configure Anthropic as a provider.

</Tab>
<Tab title="API">

```bash
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{
"provider": "anthropic",
"keys": [
{
"name": "anthropic-key-1",
"value": "env.ANTHROPIC_API_KEY",
"models": [
"*"
],
"weight": 1.0
}
]
}'
```

Refer to the API documentation for [Provider Keys Management](https://docs.getbifrost.ai/api-reference/providers/create-a-key-for-a-provider).
</Tab>
<Tab title="Go SDK">

Expand Down
20 changes: 1 addition & 19 deletions docs/providers/supported-providers/cerebras.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,25 +72,7 @@ Configure Cerebras as a provider.

</Tab>
<Tab title="API">

```bash
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{
"provider": "cerebras",
"keys": [
{
"name": "cerebras-key-1",
"value": "env.CEREBRAS_API_KEY",
"models": [
"*"
],
"weight": 1.0
}
]
}'
```

Refer to the API documentation for [Provider Keys Management](https://docs.getbifrost.ai/api-reference/providers/create-a-key-for-a-provider).
</Tab>
<Tab title="Go SDK">

Expand Down
20 changes: 1 addition & 19 deletions docs/providers/supported-providers/cohere.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,25 +72,7 @@ Configure Cohere as a provider.

</Tab>
<Tab title="API">

```bash
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{
"provider": "cohere",
"keys": [
{
"name": "cohere-key-1",
"value": "env.COHERE_API_KEY",
"models": [
"*"
],
"weight": 1.0
}
]
}'
```

Refer to the API documentation for [Provider Keys Management](https://docs.getbifrost.ai/api-reference/providers/create-a-key-for-a-provider).
</Tab>
<Tab title="Go SDK">

Expand Down
Loading
Loading