feat: add separate headers support for traces and metrics in OTEL collector

[roroghost17]

Summary

Adds support for per-signal HTTP headers in OTel profiles, allowing separate headers to be sent exclusively to the trace endpoint or the metrics endpoint, in addition to the existing shared headers field. This is particularly useful when a metrics collector requires a signal-specific header (e.g. a Databricks table name) that should not be forwarded to the trace endpoint.

Changes

• Added trace_headers and metrics_headers fields to the Profile struct and profileForStorage struct, alongside the existing headers field.
• headers continues to apply to both endpoints. trace_headers and metrics_headers are overlaid on top of the common headers at build time, with per-signal keys winning on collision.
• Introduced mergedResolvedHeaders to merge common and per-signal header maps and resolve env.VAR_NAME references without mutating the inputs.
• Extracted redactHeaderMap to eliminate duplicated redaction logic and applied it to all three header maps in Redacted().
• Updated the JSON schema (config.schema.json) with descriptions for all three header fields.
• Updated the UI form to render separate HeadersTable inputs for common, trace-only, and metrics-only headers, each with descriptive labels and FormDescription text.
• Updated the Zod schema and form serialization to include trace_headers and metrics_headers.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

# Core/Transports
go test ./plugins/otel/...

# UI
cd ui
pnpm i || npm i
pnpm test || npm test
pnpm build || npm run build

Configure an OTel profile with all three header fields:

{
  "headers": { "Authorization": "env.OTEL_TOKEN" },
  "trace_headers": { "X-Trace-Only": "trace-value" },
  "metrics_headers": { "X-Databricks-Table": "my_table" }
}

Verify that:

• Trace requests include Authorization and X-Trace-Only but not X-Databricks-Table.
• Metrics requests include Authorization and X-Databricks-Table but not X-Trace-Only.
• env.OTEL_TOKEN is resolved from the environment on both endpoints.
• Redacted config masks literal header values and preserves env. references across all three maps.

Breaking changes

• Yes
• No

Security considerations

All three header maps (headers, trace_headers, metrics_headers) are subject to the same redaction logic in Redacted(). Literal header values are masked and env. references are preserved as-is, consistent with prior behavior. No new secret storage paths are introduced.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• New Features
  • Added separate header configuration for OTEL trace and metrics endpoints.
  • Shared headers apply to both endpoints, while signal-specific headers override shared values.
  • Added dedicated trace and metrics header sections to observability profile forms.
  • Environment-based header values remain supported, with secret values protected during storage and display.
  • Headers for inactive signals no longer prevent profile initialization.

Walkthrough

OTEL profiles now support shared, trace-specific, and metrics-specific headers. The backend persists, redacts, resolves, and applies each map independently. The UI and configuration schemas expose the new fields with secret-variable support.

Changes

OTEL signal-specific headers

Layer / File(s)	Summary
Header contracts and storage plugins/otel/main.go, transports/config.schema.json, ui/lib/types/schemas.ts	Profiles and schemas define optional trace and metrics header maps. Storage persists them, and redaction masks literal values while preserving environment references.
Signal header resolution plugins/otel/main.go, plugins/otel/profiles_test.go	Shared headers are merged with signal-specific headers. Signal-specific values override shared values. Environment references resolve without mutating inputs. Trace clients and metrics exporters receive their respective resolved maps. Tests cover storage, redaction, precedence, and inactive-signal errors.
Header configuration UI ui/app/workspace/observability/fragments/otelFormFragment.tsx, ui/app/workspace/observability/views/plugins/otelView.tsx	The OTEL form adds secret-capable trace and metrics header tables. Save logic converts all header maps to backend-compatible string maps.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant OTEL Profile
  participant Header Resolver
  participant Trace Client
  participant Metrics Exporter
  OTEL Profile->>Header Resolver: resolve shared and trace headers
  Header Resolver->>Trace Client: provide resolved trace headers
  OTEL Profile->>Header Resolver: resolve shared and metrics headers
  Header Resolver->>Metrics Exporter: provide resolved metrics headers

Loading

Possibly related PRs

• maximhq/bifrost#5939: Modifies related OTEL profile handling, storage, schema, UI, and tests.
• maximhq/bifrost#5941: Extends Helm schemas and templates for OTEL signal-specific headers.
• maximhq/bifrost#5942: Adds related per-signal OTEL header configuration and documentation.

Suggested reviewers: akshaydeo, bearts

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly describes the primary change: adding separate trace and metrics headers for OTEL collectors.
Description check	✅ Passed	The description covers the purpose, implementation, affected areas, testing steps, security impact, and breaking-change status.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 08-07-feat_add_separate_headers_support_for_traces_and_metrics_in_otel_collector

---

_{Comment @coderabbitai help to get the list of available commands.}

[roroghost17]

• docs: add OTEL trace / metrics headers doc and fixes the pushed metrics list #5942
• chore: update helm-charts for OTEL trace / metrics headers changes #5941
• feat: add separate headers support for traces and metrics in OTEL collector #5940 👈 (View in Graphite)
• feat: add support for separate metrics tab independent of traces for OTEL #5939
• chore: doc fixes for config.schema.json #5938
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

plugins/otel/profiles_test.go (1)

531-538: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover trace-header redaction.

The test verifies literal-value masking only for MetricsHeaders. Add a literal TraceHeaders value and assert that Redacted() masks it. This detects a regression where trace headers bypass redactHeaderMap.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/otel/profiles_test.go` around lines 531 - 538, Add a literal entry to
the test profile’s TraceHeaders alongside the existing MetricsHeaders fixture,
then extend the redacted profile assertions to verify that Redacted() masks the
trace-header value rather than preserving the literal. Keep the existing
environment-reference preservation and metrics-header checks unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/otel/main.go`:
- Around line 557-563: Update the profile header resolution flow around
mergedResolvedHeaders so trace headers are resolved only when traces are enabled
and metrics headers only when metrics are enabled; preserve common headers and
enabled-signal merging, while ignoring unset environment references in inactive
signal maps. Add trace-only and metrics-only tests covering an unset environment
reference in the disabled signal’s headers.

---

Nitpick comments:
In `@plugins/otel/profiles_test.go`:
- Around line 531-538: Add a literal entry to the test profile’s TraceHeaders
alongside the existing MetricsHeaders fixture, then extend the redacted profile
assertions to verify that Redacted() masks the trace-header value rather than
preserving the literal. Keep the existing environment-reference preservation and
metrics-header checks unchanged.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 79a11d6b-e1ff-4424-9fbb-b8f164412bd0

📥 Commits

Reviewing files that changed from the base of the PR and between a8828bc and e48d172.

📒 Files selected for processing (6)

• plugins/otel/main.go
• plugins/otel/profiles_test.go
• transports/config.schema.json
• ui/app/workspace/observability/fragments/otelFormFragment.tsx
• ui/app/workspace/observability/views/plugins/otelView.tsx
• ui/lib/types/schemas.ts

[CLAassistant]

All committers have signed the CLA.

[akshaydeo]

Merge activity

• Aug 10, 3:13 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Aug 10, 3:18 PM UTC: Graphite rebased this pull request as part of a merge.
• Aug 10, 3:19 PM UTC: @akshaydeo merged this pull request with Graphite.

The base branch was changed.