Skip to content

chore: updated docs for otel plugin #2712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sammaji wants to merge 1 commit into
base: 04-08-feat_multiple_otel_profiles
Choose a base branch
from
+156 −15
Open

chore: updated docs for otel plugin #2712

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 156 additions & 15 deletions docs/features/observability/otel.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ func main() {
// Initialize OTel plugin
otelPlugin, err := otel.Init(ctx, &otel.Config{
ServiceName: "bifrost",
CollectorURL: "http://localhost:4318",
CollectorURL: "http://localhost:4318/v1/traces",
TraceType: otel.TraceTypeGenAIExtension,
Protocol: otel.ProtocolHTTP,
Headers: map[string]string{
Expand Down Expand Up @@ -152,7 +152,7 @@ For Gateway mode, configure via `config.json`:
"name": "otel",
"config": {
"service_name": "bifrost",
"collector_url": "http://localhost:4318",
"collector_url": "http://localhost:4318/v1/traces",
"trace_type": "genai_extension",
"protocol": "http",
"headers": {
Expand All @@ -164,7 +164,7 @@ For Gateway mode, configure via `config.json`:
}
```

If you need to connect to an OTEL collector that requires TLS, configure `tls_ca_cert`:
If you need to connect to an OTEL collector that requires TLS, configure `tls_ca_cert` and set insecure mode to `false`:
Comment thread
coderabbitai[bot] marked this conversation as resolved.

```json
{
Expand All @@ -177,6 +177,7 @@ If you need to connect to an OTEL collector that requires TLS, configure `tls_ca
"collector_url": "localhost:4317",
"trace_type": "genai_extension",
"protocol": "grpc",
"insecure": false,
"tls_ca_cert": "/path/to/your/ca.cert",
"headers": {
"Authorization": "env.OTEL_API_KEY"
Expand All @@ -188,6 +189,65 @@ If you need to connect to an OTEL collector that requires TLS, configure `tls_ca
```

</Tab>
<Tab title="config.json (v1.5+)">

For Gateway mode, configure via `config.json`:

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "http://localhost:4318/v1/traces",
"trace_type": "genai_extension",
"protocol": "http",
"headers": {
"Authorization": "env.OTEL_API_KEY"
}
}
]
}
}
]
}
```

If you need to connect to an OTEL collector that requires TLS, configure `tls_ca_cert` and set insecure mode to `false`:

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "localhost:4317",
"trace_type": "genai_extension",
"protocol": "grpc",
"insecure": false,
"tls_ca_cert": "/path/to/your/ca.cert",
"headers": {
"Authorization": "env.OTEL_API_KEY"
}
}
]
}
}
]
}
```
</Tab>
Comment on lines +192 to +249
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== OTEL config schema in current branch =="
rg -n 'type Config struct|json:"profiles"|Profiles|service_name|collector_url|trace_type|protocol' plugins/otel/main.go -C2

echo
echo "== OTEL validation requirements (top-level keys) =="
rg -n 'collector url is required|trace type is required|protocol is required|ValidateConfig' plugins/otel/main.go -C2

echo
echo "== Docs references to profiles =="
rg -n '"profiles"|config.json \(v1\.5\+\)' docs/features/observability/otel.mdx -C1

echo
echo "== PR stack metadata for this Graphite-managed PR =="
gh pr view 2712 --repo maximhq/bifrost --json number,title,headRefName,baseRefName,body,url

Repository: maximhq/bifrost

Length of output: 5070

config.profiles[] examples describe a config structure that the current plugin does not support.

The documentation in the v1.5+ section shows nested config.profiles[] configuration, but plugins/otel/main.go defines the Config struct with flat top-level fields only (service_name, collector_url, trace_type, protocol, etc.). The ValidateConfig function validates these top-level fields directly and does not recognize a profiles field. Users who copy these documented examples will fail validation.

This PR updates documentation without a corresponding implementation change to add profile support. Before merging, either:

  • Gate these sections behind a separate feature branch/implementation PR, or
  • Revert the examples to use the flat config structure that the plugin currently accepts
Example using current supported schema 
{
  "plugins": [
    {
      "enabled": true,
      "name": "otel",
      "config": {
        "service_name": "bifrost",
        "collector_url": "http://localhost:4318",
        "trace_type": "genai_extension",
        "protocol": "http",
        "headers": {
          "Authorization": "env.OTEL_API_KEY"
        }
      }
    }
  ]
}

Also applies to lines 868–937 (metrics configuration sections).

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docs/features/observability/otel.mdx` around lines 192 - 249, The docs show a
nested "profiles" array but the otel plugin only supports a flat config; update
the MDX examples to match the current Config struct and ValidateConfig behavior
in plugins/otel/main.go by replacing any "config.profiles[]" examples with the
flat top-level fields (service_name, collector_url, trace_type, protocol,
insecure, tls_ca_cert, headers, etc.) for both traces and metrics sections (also
address the repeated sections around lines 868–937). Reference the Config type
and ValidateConfig function to ensure every documented field maps directly to
the plugin's supported fields and remove or gate any "profiles" examples until a
code change adds profile support.


</Tabs>

---
Expand Down Expand Up @@ -217,39 +277,55 @@ services:
depends_on:
- tempo

redpanda:
image: redpandadata/redpanda:latest
container_name: redpanda
command: >
redpanda start --overprovisioned
--mode=dev-container
--kafka-addr=PLAINTEXT://0.0.0.0:9092
--advertise-kafka-addr=PLAINTEXT://redpanda:9092
ports:
- "9092:9092"
restart: unless-stopped

Comment thread
coderabbitai[bot] marked this conversation as resolved.
tempo:
image: grafana/tempo:latest
container_name: tempo
command: [ "-config.file=/etc/tempo.yaml" ]
command: ["-target=all", "-config.file=/etc/tempo.yaml"]
configs:
- source: tempo-config
target: /etc/tempo.yaml
ports:
- "3200:3200" # tempo HTTP API
- "3200:3200" # Tempo HTTP API
expose:
- "4317" # OTLP gRPC (internal)
volumes:
- tempo-data:/var/tempo
depends_on:
- redpanda
restart: unless-stopped

prometheus:
image: prom/prometheus:latest
container_name: prometheus
depends_on:
- otel-collector
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--web.console.libraries=/usr/share/prometheus/console_libraries"
- "--web.console.templates=/usr/share/prometheus/consoles"
- "--web.enable-remote-write-receiver"
- "--enable-feature=exemplar-storage"
- "--enable-feature=native-histograms"
ports:
- "9090:9090"
volumes:
- prometheus-data:/prometheus
configs:
- source: prometheus-config
target: /etc/prometheus/prometheus.yml
depends_on:
- otel-collector
Comment thread
coderabbitai[bot] marked this conversation as resolved.
restart: unless-stopped
Comment thread
greptile-apps[bot] marked this conversation as resolved.

grafana:
Expand All @@ -263,9 +339,8 @@ services:
GF_SECURITY_ADMIN_PASSWORD: admin
GF_AUTH_ANONYMOUS_ENABLED: "true"
GF_AUTH_ANONYMOUS_ORG_ROLE: Viewer
GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: "grafana-pyroscope-app,grafana-exploretraces-app,grafana-metricsdrilldown-app"
GF_PLUGINS_ENABLE_ALPHA: "true"
GF_INSTALL_PLUGINS: ""
GF_FEATURE_TOGGLES_ENABLE: traceqlEditor
ports:
- "4000:3000"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
volumes:
Expand Down Expand Up @@ -295,12 +370,12 @@ configs:
namespace: otel
const_labels:
source: otelcol

otlp/tempo:
endpoint: tempo:4317
tls:
insecure: true

debug:
verbosity: detailed

Expand Down Expand Up @@ -339,13 +414,21 @@ configs:
http_listen_port: 3200
log_level: info

ingest:
kafka:
address: redpanda:9092
topic: tempo-ingest

block_builder:
consume_cycle_duration: 30s
Comment on lines +417 to +423
Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps Bot Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Kafka ingest path is wired in Tempo but never fed by the collector

ingest.kafka and block_builder are configured in Tempo, but the OTel Collector's pipeline still exports traces directly to tempo:4317 via otlp/tempo. No component in this stack writes to the tempo-ingest Kafka topic on Redpanda, so the block builder will never consume any data and traces will continue to flow through the standard OTLP path — making Redpanda an unused service.

To make the Kafka path actually work, the OTel Collector config (or a new sidecar) needs a kafka exporter writing to redpanda:9092 on the tempo-ingest topic, and the otlp/tempo exporter should be removed from the traces pipeline. Otherwise, the ingest.kafka/block_builder blocks and the Redpanda service can be removed and the docs simplified back to a direct-ingest setup.


distributor:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317

ingester:
max_block_duration: 5m
trace_idle_period: 10s
Expand Down Expand Up @@ -419,11 +502,12 @@ volumes:

This launches:
- **OTel Collector** - Receives traces on ports 4317 (gRPC) and 4318 (HTTP)
- **Redpanda** - Kafka-compatible broker that buffers trace ingestion for Tempo
- **Tempo** - Distributed tracing backend
- **Prometheus** - Metrics collection
- **Grafana** - Visualization dashboard

Access Grafana at `http://localhost:3000` (default credentials: admin/admin)
Access Grafana at `http://localhost:4000` (default credentials: admin/admin)

<Frame>
<img src="/media/grafana-otel-traces.png" alt="Okta Applications page" />
Expand Down Expand Up @@ -709,7 +793,7 @@ Uses HTTP/1.1 or HTTP/2 with JSON or Protobuf encoding:

```json
{
"collector_url": "http://localhost:4318",
"collector_url": "http://localhost:4318/v1/traces",
"protocol": "http"
}
```
Expand Down Expand Up @@ -771,8 +855,37 @@ The OTel plugin supports **push-based metrics export** via OTLP, which is essent
]
}
```
</Tab>

<Tab title="HTTP Protocol (v1.5+)">

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "http://otel-collector:4318/v1/traces",
"trace_type": "genai_extension",
"protocol": "http",
"metrics_enabled": true,
"metrics_endpoint": "http://otel-collector:4318/v1/metrics",
"metrics_push_interval": 15
}
]
}
}
]
}
```

</Tab>

<Tab title="gRPC Protocol">

```json
Expand All @@ -794,8 +907,36 @@ The OTel plugin supports **push-based metrics export** via OTLP, which is essent
]
}
```
</Tab>

<Tab title="gRPC Protocol (v1.5+)">

```json
{
"plugins": [
{
"enabled": true,
"name": "otel",
"config": {
"profiles": [
{
"service_name": "bifrost",
"enabled": true,
"collector_url": "otel-collector:4317",
"trace_type": "genai_extension",
"protocol": "grpc",
"metrics_enabled": true,
"metrics_endpoint": "otel-collector:4317",
"metrics_push_interval": 15
}
]
}
}
]
}
```
</Tab>

</Tabs>

### Pushed Metrics
Expand Down Expand Up @@ -975,4 +1116,4 @@ echo $OTEL_API_KEY

- **[Built-in Observability](./default)** - Local logging for development
- **[Maxim Plugin](./maxim)** - Advanced LLM evaluation and monitoring
- **[Telemetry](../telemetry)** - Prometheus metrics and dashboards
- **[Telemetry](../telemetry)** - Prometheus metrics and dashboards
Loading