docs: add OTEL trace / metrics headers doc and fixes the pushed metrics list

[roroghost17]

Summary

Documents two new OTel plugin capabilities: per-signal headers (trace_headers and metrics_headers) and a traces_enabled flag that enables a metrics-only mode where collector_url is not required.

Changes

• Added traces_enabled field documentation — when set to false, the trace client is never built and collector_url/trace_type become optional, enabling metrics-only profiles
• Added trace_headers and metrics_headers fields — these are overlaid on top of the shared headers field for their respective endpoints, with per-signal values winning on key collision
• Clarified that headers is sent to both trace and metrics endpoints, and that protocol is shared between both signals
• Added a "Per-signal headers" section with a worked example showing Authorization shared via headers and X-Databricks-Table scoped to the metrics endpoint via metrics_headers
• Added a "Metrics-only mode" section with a full JSON configuration example
• Expanded the pushed metrics table to include bifrost_cache_read_input_tokens_total, bifrost_cache_write_input_tokens_total, bifrost_cache_write_input_tokens_5m_total, bifrost_cache_write_input_tokens_1h_total, bifrost_request_retries, and mcp.client.operation.duration
• Added a note clarifying that an unreachable metrics endpoint never blocks Bifrost startup
• Updated env-var substitution docs to include trace_headers and metrics_headers
• Applied the same changes to both the config-json and Helm plugin reference pages

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Review the rendered documentation for the OTel plugin pages:

• docs/features/observability/otel.mdx
• docs/deployment-guides/config-json/plugins.mdx
• docs/deployment-guides/helm/plugins.mdx

Verify that:

1. The traces_enabled: false example produces a valid metrics-only config with no collector_url
2. The per-signal headers example correctly shows Authorization on both endpoints and X-Databricks-Table only on the metrics endpoint
3. All new metrics in the pushed metrics table are accurately described

Breaking changes

• Yes
• No

Related issues

Security considerations

trace_headers and metrics_headers support the env. prefix for environment variable substitution, consistent with the existing headers field. No new secrets are stored in configuration.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Documentation
  • Clarified OpenTelemetry configuration for independently enabling traces and metrics.
  • Documented metrics-only setups and when collector URLs are required.
  • Added guidance for shared and signal-specific headers, protocols, and environment-variable substitution.
  • Explained metrics connection behavior, retry and drop handling, and newly available cache-token, retry, and MCP metrics.

Walkthrough

The OpenTelemetry documentation now covers independent trace control, conditional collector settings, shared and per-signal headers, metrics-only profiles, metrics connection behavior, and additional metrics.

Changes

OpenTelemetry documentation

Layer / File(s)	Summary
Configuration contract and per-signal headers docs/deployment-guides/config-json/plugins.mdx, docs/deployment-guides/helm/plugins.mdx, docs/features/observability/otel.mdx	Documents traces_enabled, conditional collector_url and trace_type, protocol use for both signals, environment-variable substitution, and shared, trace-specific, and metrics-specific headers with override precedence.
Metrics operation and metrics-only mode docs/features/observability/otel.mdx	Documents lazy metrics collector dialing, failed-push handling, retry behavior, and profiles that enable metrics without traces.
Additional metric definitions docs/features/observability/otel.mdx	Adds cache token, cache-write TTL, request retry, and MCP client operation duration metrics.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• maximhq/bifrost#5939: Introduces the independent trace and metrics OTel configuration documented here.
• maximhq/bifrost#5940: Introduces the per-signal trace and metrics header support documented here.
• maximhq/bifrost#5941: Implements the OTel configuration fields and validation behavior documented here.

Suggested reviewers: akshaydeo, bearts, pratham-mishra04

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The description covers the purpose, documentation changes, affected areas, validation steps, security considerations, and breaking-change status.
Title check	✅ Passed	The title accurately identifies the OTel header documentation and pushed metrics updates, which are central parts of the changeset.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 08-07-docs_add_otel_trace___metrics_headers_doc_and_fixes_the_pushed_metrics_list

---

_{Comment @coderabbitai help to get the list of available commands.}

[CLAassistant]

All committers have signed the CLA.

[roroghost17]

• docs: add OTEL trace / metrics headers doc and fixes the pushed metrics list #5942 👈 (View in Graphite)
• chore: update helm-charts for OTEL trace / metrics headers changes #5941
• feat: add separate headers support for traces and metrics in OTEL collector #5940
• feat: add support for separate metrics tab independent of traces for OTEL #5939
• chore: doc fixes for config.schema.json #5938
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/features/observability/otel.mdx`:
- Line 41: Align the documented transport configuration requirements with
transports/config.schema.json: in docs/features/observability/otel.mdx at lines
41-41, state that protocol is required only when tracing or metrics is enabled;
in docs/deployment-guides/config-json/plugins.mdx at lines 132-135, document the
same conditional protocol requirement and mark metrics_endpoint required when
metrics_enabled is true; in docs/deployment-guides/helm/plugins.mdx at lines
263-266, document the conditional requirements for trace_type when tracing is
enabled, shared protocol when tracing or metrics is enabled, and
metrics_endpoint when metrics is enabled.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 59af9222-af74-4bce-89be-a834b8c335b5

📥 Commits

Reviewing files that changed from the base of the PR and between e718291 and 0a8c411.

📒 Files selected for processing (3)

• docs/deployment-guides/config-json/plugins.mdx
• docs/deployment-guides/helm/plugins.mdx
• docs/features/observability/otel.mdx

[akshaydeo]

Merge activity

• Aug 10, 3:13 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Aug 10, 3:21 PM UTC: @akshaydeo merged this pull request with Graphite.

The base branch was changed.