Skip to content

feat: adds plugin_span_filters to config json and helm charts - #4201

Merged
Pratham-Mishra04 merged 1 commit into
devfrom
06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts
Jun 9, 2026
Merged

feat: adds plugin_span_filters to config json and helm charts#4201
Pratham-Mishra04 merged 1 commit into
devfrom
06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts

Conversation

@roroghost17

@roroghost17 roroghost17 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Extends plugin_span_filter support to the Datadog observability connector and introduces a new BigQuery traces connector, while renaming the shared otelPluginSpanFilter / otel_plugin_span_filter schema definition to the more generic pluginSpanFilter / plugin_span_filter so it can be reused across all observability connectors.

Changes

  • Added plugin_span_filter support to the Datadog plugin in both the Helm chart template (_helpers.tpl) and its schema/values definitions, matching the existing pattern used by OTEL connectors.
  • Renamed the otelPluginSpanFilter / otel_plugin_span_filter schema $defs entry to pluginSpanFilter / plugin_span_filter in both values.schema.json and transports/config.schema.json, and updated all $ref usages accordingly. The description was also updated to clarify that the filter applies to any observability connector, not just OTEL.
  • Added a full JSON schema definition for a new bigquery observability connector in transports/config.schema.json, including fields for project_id, dataset_id, table_id, location, service_account_key (with ADC fallback), flush_interval_seconds, buffer_size, custom_labels, disable_content_logging, request_headers, and plugin_span_filter.
  • Added the bigquery plugin to the Helm chart (values.yaml, values.schema.json, and _helpers.tpl) with the same version validation guard used by other built-in plugins.
  • Added commented-out plugin_span_filter examples to values.yaml for both the Datadog and BigQuery plugins to aid discoverability.

Type of change

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Chore/CI

Affected areas

  • Core (Go)
  • Transports (HTTP)
  • Providers/Integrations
  • Plugins
  • UI (React)
  • Docs

How to test

  1. Deploy the Helm chart with a Datadog plugin config that includes plugin_span_filter:

    bifrost:
      plugins:
        datadog:
          config:
            plugin_span_filter:
              mode: "exclude"
              plugins: ["logging"]

    Verify the rendered manifest includes plugin_span_filter in the Datadog plugin config.

  2. Deploy the Helm chart with the BigQuery plugin enabled:

    bifrost:
      plugins:
        bigquery:
          enabled: true
          version: 1
          config:
            project_id: "my-gcp-project"
            dataset_id: "bifrost_traces"
            table_id: "traces"

    Verify the rendered manifest includes the BigQuery plugin config with the expected fields.

  3. Validate transports/config.schema.json against a BigQuery connector config:

    {
      "name": "bigquery",
      "config": {
        "project_id": "my-gcp-project",
        "plugin_span_filter": { "mode": "include", "plugins": ["auth"] }
      }
    }
  4. Confirm that no dangling $ref entries referencing the old otelPluginSpanFilter / otel_plugin_span_filter names remain in either schema file.

Breaking changes

  • Yes
  • No

The otelPluginSpanFilter / otel_plugin_span_filter $defs keys have been renamed to pluginSpanFilter / plugin_span_filter. Any external tooling or configs that reference these definition names directly will need to be updated.

Related issues

Security considerations

The BigQuery connector schema supports service_account_key via an environment variable reference (env.MY_VAR) or Application Default Credentials, avoiding the need to embed raw credentials in config files. Care should be taken to ensure service account keys are not logged or exposed through the custom_labels or request_headers fields.

Checklist

  • I read docs/contributing/README.md and followed the guidelines
  • I added/updated tests where appropriate
  • I updated documentation where needed
  • I verified builds succeed (Go and UI)
  • I verified the CI pipeline passes locally if applicable

Summary by CodeRabbit

  • New Features

    • Added BigQuery plugin for exporting spans and traces with configurable project, dataset, table, location, buffering, and custom labels.
    • Added plugin_span_filter option to Datadog plugin to control which spans are exported.
  • Documentation

    • Updated Helm chart documentation and configuration schemas with new plugin options.

@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Generalizes the OTEL-only plugin span filter schema into a shared definition, wires plugin_span_filter into Datadog and BigQuery plugin schemas, adds a BigQuery plugin to Helm templates/values with validation, and updates the 2.1.22 changelog.

Changes

Plugin Span Filtering Generalization and BigQuery Plugin Addition

Layer / File(s) Summary
Shared plugin span filter schema definition
helm-charts/bifrost/values.schema.json, transports/config.schema.json
Replaced the OTEL-specific otelPluginSpanFilter / otel_plugin_span_filter with a shared pluginSpanFilter / plugin_span_filter $defs and updated descriptions.
Plugin schema fields and OTEL references
helm-charts/bifrost/values.schema.json, transports/config.schema.json
Added plugin_span_filter to plugins.datadog.config and plugins.bigquery.config; updated otelProfileConfig and otelProfilesConfig to reference the shared schema.
Helm template rendering, validation, and value examples
helm-charts/bifrost/templates/_helpers.tpl, helm-charts/bifrost/values.yaml
Datadog plugin rendering includes plugin_span_filter when present; a BigQuery plugin block is conditionally built and appended with mapped fields and optional integer version; BigQuery version bounds and required project_id validation were added; values.yaml includes commented examples.
Release notes and changelog
helm-charts/bifrost/README.md
Changelog for v2.1.22 documents Datadog plugin_span_filter support, the new BigQuery plugin and its options (e.g., disable_content_logging), and the internal refactor to a shared schema.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • maximhq/bifrost#3944: Previous work on the OTEL plugin_span_filter shapes and multi-profile handling that this PR generalizes.
  • maximhq/bifrost#4199: Introduced shared PluginSpanFilter schema in core/schemas and overlaps on schema wiring extended here.

Suggested reviewers

  • danpiths
  • akshaydeo
  • Pratham-Mishra04

"I hopped through schemas bright and merry,
Shared filters now guide spans we carry,
Datadog and BigQuery join the dance,
OTEL’s shape reused at a glance,
A rabbit cheers the unified ferry."

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main feature additions: plugin_span_filters support across Datadog and BigQuery plugins, plus config schema updates.
Description check ✅ Passed The PR description addresses most template sections with clear, detailed explanations of changes, testing instructions, and breaking changes disclosure, though some checklist items remain unchecked.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts

Comment @coderabbitai help to get the list of available commands and usage tips.

roroghost17 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

@CLAassistant

CLAassistant commented Jun 9, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@roroghost17
roroghost17 force-pushed the 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts branch from e8c3643 to 1d37af0 Compare June 9, 2026 10:31
@roroghost17
roroghost17 marked this pull request as ready for review June 9, 2026 10:32
@roroghost17
roroghost17 requested a review from a team as a code owner June 9, 2026 10:32
@greptile-apps

greptile-apps Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Confidence Score: 5/5

Schema-only and Helm-template change; no Go runtime code touched; all previous reviewer concerns have been addressed.

The PR touches only JSON schema files, Helm templates, and values files. The three issues flagged in previous review threads — project_id silently dropped, missing required in the Helm schema, and numeric fields being dropped for a zero value — are all explicitly addressed: a Helm fail guard fires at deploy time when project_id is absent, required: ["project_id"] is present in both schema files, and flush_interval_seconds/buffer_size correctly use hasKey. No Go code is changed, so there are no runtime regressions to evaluate.

No files require special attention.

Important Files Changed

Filename Overview
helm-charts/bifrost/templates/_helpers.tpl Adds BigQuery plugin rendering and validation guards; numeric fields use hasKey, boolean fields use hasKey, previous reviewer concerns all addressed
helm-charts/bifrost/values.schema.json Renames otelPluginSpanFilter to pluginSpanFilter, adds BigQuery config schema with required project_id, adds plugin_span_filter to Datadog config
transports/config.schema.json Renames otel_plugin_span_filter $defs key to plugin_span_filter, adds full BigQuery connector schema with required project_id and additionalProperties: false
helm-charts/bifrost/values.yaml Adds BigQuery plugin defaults with sensible values; commented-out plugin_span_filter examples for Datadog and BigQuery
helm-charts/bifrost/README.md Changelog updated for pluginSpanFilter rename, Datadog plugin_span_filter, and BigQuery plugin additions

Reviews (5): Last reviewed commit: "feat: adds plugin_span_filters to config..." | Re-trigger Greptile

Comment thread helm-charts/bifrost/templates/_helpers.tpl
Comment thread helm-charts/bifrost/values.schema.json

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
helm-charts/bifrost/values.schema.json (1)

903-966: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add conditional required validation for BigQuery core configuration fields.

BigQuery cannot operate without project_id, dataset_id, and table_id, but the schema does not enforce these as required. Other plugins (e.g., semanticCache at lines 822-835) use if/then conditionals to require essential fields when enabled: true.

Add a validation guard to ensure these core fields are present when the BigQuery plugin is enabled.

🛡️ Proposed validation guard
             "bigquery": {
               "type": "object",
               "properties": {
                 "enabled": {
                   "type": "boolean"
                 },
                 "version": {
                   "type": "integer",
                   "minimum": 1
                 },
                 "config": {
                   ...
                 }
-              }
+              },
+              "if": {
+                "properties": {
+                  "enabled": {
+                    "const": true
+                  }
+                }
+              },
+              "then": {
+                "properties": {
+                  "config": {
+                    "required": ["project_id", "dataset_id", "table_id"]
+                  }
+                }
+              }
             },
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@helm-charts/bifrost/values.schema.json` around lines 903 - 966, The BigQuery
schema block for "bigquery" is missing an if/then guard to require core config
fields when enabled; update the "bigquery" object to add an "if" that checks {
"properties": { "enabled": { "const": true } } } and a corresponding "then" that
enforces "config" -> "required": ["project_id","dataset_id","table_id"] (and/or
require "config" itself) similar to the existing semanticCache pattern;
reference the "bigquery" schema, its "enabled" property and the "config"
properties (project_id, dataset_id, table_id) when adding this conditional
validation so those fields are mandatory only when bigquery.enabled is true.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@helm-charts/bifrost/values.schema.json`:
- Around line 928-940: The current schema for service_account_key allows value,
env_var and from_env to coexist; change the inner object schema so the object
enforces mutual exclusivity by replacing the "properties" block with a oneOf
that defines two clear alternatives: one alternative requires ["value"] (type:
string) and disallows other properties, the other requires
["env_var","from_env"] (env_var: string, from_env: boolean) and disallows other
properties; keep the outer anyOf that accepts a plain string or this new
oneOf-object to ensure either a direct credential or environment indirection is
supplied but not both.

---

Outside diff comments:
In `@helm-charts/bifrost/values.schema.json`:
- Around line 903-966: The BigQuery schema block for "bigquery" is missing an
if/then guard to require core config fields when enabled; update the "bigquery"
object to add an "if" that checks { "properties": { "enabled": { "const": true }
} } and a corresponding "then" that enforces "config" -> "required":
["project_id","dataset_id","table_id"] (and/or require "config" itself) similar
to the existing semanticCache pattern; reference the "bigquery" schema, its
"enabled" property and the "config" properties (project_id, dataset_id,
table_id) when adding this conditional validation so those fields are mandatory
only when bigquery.enabled is true.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0f23d8d9-b007-46d2-81b5-8eece4dcb5db

📥 Commits

Reviewing files that changed from the base of the PR and between 7e12811 and 1d37af0.

📒 Files selected for processing (5)
  • helm-charts/bifrost/README.md
  • helm-charts/bifrost/templates/_helpers.tpl
  • helm-charts/bifrost/values.schema.json
  • helm-charts/bifrost/values.yaml
  • transports/config.schema.json

Comment thread helm-charts/bifrost/values.schema.json
@roroghost17
roroghost17 force-pushed the 06-09-feat_adds_plugin_span_filtering_for_multiple_connectors branch from 7e12811 to 7473d46 Compare June 9, 2026 11:03
@roroghost17
roroghost17 force-pushed the 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts branch from 1d37af0 to 45d25de Compare June 9, 2026 11:03
Comment thread helm-charts/bifrost/templates/_helpers.tpl Outdated
@roroghost17
roroghost17 force-pushed the 06-09-feat_adds_plugin_span_filtering_for_multiple_connectors branch from 7473d46 to a02e997 Compare June 9, 2026 11:25
@roroghost17
roroghost17 force-pushed the 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts branch from 45d25de to 1931de3 Compare June 9, 2026 11:25

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@helm-charts/bifrost/README.md`:
- Line 25: Update the changelog/README to explicitly warn external schema
consumers that the internal JSON Schema $defs identifier otelPluginSpanFilter
(and any snake_case variant otel_plugin_span_filter) was renamed to
pluginSpanFilter; although the Helm chart's user-facing config key
plugin_span_filter is unchanged, any external tooling or configs that reference
the old $defs name must be updated to reference pluginSpanFilter to avoid
breakage. Ensure the note mentions all affected plugin configs (OTEL, Datadog,
BigQuery) and that this is an internal schema-definition rename only for Helm
users but a breaking change for external $defs consumers.

In `@helm-charts/bifrost/values.yaml`:
- Around line 539-541: Add an inline security warning comment next to the
custom_labels and request_headers entries (referencing the custom_labels and
request_headers keys) that explicitly warns users not to place secrets or
credential material (e.g., service_account_key, tokens, passwords, private keys)
in those fields because they may be logged or exported to external systems; keep
the comment concise and placed immediately above or beside the existing
custom_labels/request_headers lines so it's visible when editing values.yaml.

In `@transports/config.schema.json`:
- Around line 1707-1709: Update the helm chart README to stop referencing the
removed OTEL schema name `otel_plugin_span_filter` /
`"$defs/otel_plugin_span_filter"` and instead reference the shared schema
`plugin_span_filter` (`$defs/plugin_span_filter`); search for any occurrences of
`otel_plugin_span_filter`, `"$defs/otel_plugin_span_filter"`, or wording that
implies a separate OTEL-specific schema in `helm-charts/bifrost/README.md` and
replace them with `plugin_span_filter` and `"$defs/plugin_span_filter"`,
updating any explanatory text so it correctly describes using the shared
`plugin_span_filter` shape.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: b22b66d0-f2bb-47ce-92e7-69bfeb30431e

📥 Commits

Reviewing files that changed from the base of the PR and between 45d25de and 1931de3.

📒 Files selected for processing (5)
  • helm-charts/bifrost/README.md
  • helm-charts/bifrost/templates/_helpers.tpl
  • helm-charts/bifrost/values.schema.json
  • helm-charts/bifrost/values.yaml
  • transports/config.schema.json

Comment thread helm-charts/bifrost/README.md Outdated
Comment thread helm-charts/bifrost/values.yaml
Comment thread transports/config.schema.json
@roroghost17
roroghost17 force-pushed the 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts branch from 1931de3 to 7350823 Compare June 9, 2026 12:21
@coderabbitai
coderabbitai Bot requested review from akshaydeo and danpiths June 9, 2026 12:24
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 9, 2026

Pratham-Mishra04 commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Merge activity

  • Jun 9, 12:40 PM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jun 9, 12:43 PM UTC: Graphite rebased this pull request as part of a merge.
  • Jun 9, 12:44 PM UTC: @Pratham-Mishra04 merged this pull request with Graphite.

@Pratham-Mishra04
Pratham-Mishra04 changed the base branch from 06-09-feat_adds_plugin_span_filtering_for_multiple_connectors to graphite-base/4201 June 9, 2026 12:40
@Pratham-Mishra04
Pratham-Mishra04 changed the base branch from graphite-base/4201 to dev June 9, 2026 12:41
@Pratham-Mishra04
Pratham-Mishra04 dismissed coderabbitai[bot]’s stale review June 9, 2026 12:41

The base branch was changed.

@Pratham-Mishra04
Pratham-Mishra04 force-pushed the 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts branch from 7350823 to 3e5da32 Compare June 9, 2026 12:42
@Pratham-Mishra04
Pratham-Mishra04 merged commit fc02026 into dev Jun 9, 2026
14 of 15 checks passed
@Pratham-Mishra04
Pratham-Mishra04 deleted the 06-09-feat_adds_plugin_span_filters_to_config_json_and_helm_charts branch June 9, 2026 12:44
akshaydeo pushed a commit that referenced this pull request Jun 12, 2026
## Summary

Extends `plugin_span_filter` support to the Datadog observability connector and introduces a new BigQuery traces connector, while renaming the shared `otelPluginSpanFilter` / `otel_plugin_span_filter` schema definition to the more generic `pluginSpanFilter` / `plugin_span_filter` so it can be reused across all observability connectors.

## Changes

- Added `plugin_span_filter` support to the Datadog plugin in both the Helm chart template (`_helpers.tpl`) and its schema/values definitions, matching the existing pattern used by OTEL connectors.
- Renamed the `otelPluginSpanFilter` / `otel_plugin_span_filter` schema `$defs` entry to `pluginSpanFilter` / `plugin_span_filter` in both `values.schema.json` and `transports/config.schema.json`, and updated all `$ref` usages accordingly. The description was also updated to clarify that the filter applies to any observability connector, not just OTEL.
- Added a full JSON schema definition for a new `bigquery` observability connector in `transports/config.schema.json`, including fields for `project_id`, `dataset_id`, `table_id`, `location`, `service_account_key` (with ADC fallback), `flush_interval_seconds`, `buffer_size`, `custom_labels`, `disable_content_logging`, `request_headers`, and `plugin_span_filter`.
- Added the `bigquery` plugin to the Helm chart (`values.yaml`, `values.schema.json`, and `_helpers.tpl`) with the same `version` validation guard used by other built-in plugins.
- Added commented-out `plugin_span_filter` examples to `values.yaml` for both the Datadog and BigQuery plugins to aid discoverability.

## Type of change

- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [ ] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

1. Deploy the Helm chart with a Datadog plugin config that includes `plugin_span_filter`:
   ```yaml
   bifrost:
     plugins:
       datadog:
         config:
           plugin_span_filter:
             mode: "exclude"
             plugins: ["logging"]
   ```
   Verify the rendered manifest includes `plugin_span_filter` in the Datadog plugin config.

2. Deploy the Helm chart with the BigQuery plugin enabled:
   ```yaml
   bifrost:
     plugins:
       bigquery:
         enabled: true
         version: 1
         config:
           project_id: "my-gcp-project"
           dataset_id: "bifrost_traces"
           table_id: "traces"
   ```
   Verify the rendered manifest includes the BigQuery plugin config with the expected fields.

3. Validate `transports/config.schema.json` against a BigQuery connector config:
   ```json
   {
     "name": "bigquery",
     "config": {
       "project_id": "my-gcp-project",
       "plugin_span_filter": { "mode": "include", "plugins": ["auth"] }
     }
   }
   ```

4. Confirm that no dangling `$ref` entries referencing the old `otelPluginSpanFilter` / `otel_plugin_span_filter` names remain in either schema file.

## Breaking changes

- [x] Yes
- [ ] No

The `otelPluginSpanFilter` / `otel_plugin_span_filter` `$defs` keys have been renamed to `pluginSpanFilter` / `plugin_span_filter`. Any external tooling or configs that reference these definition names directly will need to be updated.

## Related issues

## Security considerations

The BigQuery connector schema supports `service_account_key` via an environment variable reference (`env.MY_VAR`) or Application Default Credentials, avoiding the need to embed raw credentials in config files. Care should be taken to ensure service account keys are not logged or exposed through the `custom_labels` or `request_headers` fields.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
akhsaul pushed a commit to akhsaul/bifrost that referenced this pull request Aug 27, 2026
…q#4201)

## Summary

Extends `plugin_span_filter` support to the Datadog observability connector and introduces a new BigQuery traces connector, while renaming the shared `otelPluginSpanFilter` / `otel_plugin_span_filter` schema definition to the more generic `pluginSpanFilter` / `plugin_span_filter` so it can be reused across all observability connectors.

## Changes

- Added `plugin_span_filter` support to the Datadog plugin in both the Helm chart template (`_helpers.tpl`) and its schema/values definitions, matching the existing pattern used by OTEL connectors.
- Renamed the `otelPluginSpanFilter` / `otel_plugin_span_filter` schema `$defs` entry to `pluginSpanFilter` / `plugin_span_filter` in both `values.schema.json` and `transports/config.schema.json`, and updated all `$ref` usages accordingly. The description was also updated to clarify that the filter applies to any observability connector, not just OTEL.
- Added a full JSON schema definition for a new `bigquery` observability connector in `transports/config.schema.json`, including fields for `project_id`, `dataset_id`, `table_id`, `location`, `service_account_key` (with ADC fallback), `flush_interval_seconds`, `buffer_size`, `custom_labels`, `disable_content_logging`, `request_headers`, and `plugin_span_filter`.
- Added the `bigquery` plugin to the Helm chart (`values.yaml`, `values.schema.json`, and `_helpers.tpl`) with the same `version` validation guard used by other built-in plugins.
- Added commented-out `plugin_span_filter` examples to `values.yaml` for both the Datadog and BigQuery plugins to aid discoverability.

## Type of change

- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [ ] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

1. Deploy the Helm chart with a Datadog plugin config that includes `plugin_span_filter`:
   ```yaml
   bifrost:
     plugins:
       datadog:
         config:
           plugin_span_filter:
             mode: "exclude"
             plugins: ["logging"]
   ```
   Verify the rendered manifest includes `plugin_span_filter` in the Datadog plugin config.

2. Deploy the Helm chart with the BigQuery plugin enabled:
   ```yaml
   bifrost:
     plugins:
       bigquery:
         enabled: true
         version: 1
         config:
           project_id: "my-gcp-project"
           dataset_id: "bifrost_traces"
           table_id: "traces"
   ```
   Verify the rendered manifest includes the BigQuery plugin config with the expected fields.

3. Validate `transports/config.schema.json` against a BigQuery connector config:
   ```json
   {
     "name": "bigquery",
     "config": {
       "project_id": "my-gcp-project",
       "plugin_span_filter": { "mode": "include", "plugins": ["auth"] }
     }
   }
   ```

4. Confirm that no dangling `$ref` entries referencing the old `otelPluginSpanFilter` / `otel_plugin_span_filter` names remain in either schema file.

## Breaking changes

- [x] Yes
- [ ] No

The `otelPluginSpanFilter` / `otel_plugin_span_filter` `$defs` keys have been renamed to `pluginSpanFilter` / `plugin_span_filter`. Any external tooling or configs that reference these definition names directly will need to be updated.

## Related issues

## Security considerations

The BigQuery connector schema supports `service_account_key` via an environment variable reference (`env.MY_VAR`) or Application Default Credentials, avoiding the need to embed raw credentials in config files. Care should be taken to ensure service account keys are not logged or exposed through the `custom_labels` or `request_headers` fields.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
occcat pushed a commit to occcat/bifrost that referenced this pull request Sep 2, 2026
…q#4201)

## Summary

Extends `plugin_span_filter` support to the Datadog observability connector and introduces a new BigQuery traces connector, while renaming the shared `otelPluginSpanFilter` / `otel_plugin_span_filter` schema definition to the more generic `pluginSpanFilter` / `plugin_span_filter` so it can be reused across all observability connectors.

## Changes

- Added `plugin_span_filter` support to the Datadog plugin in both the Helm chart template (`_helpers.tpl`) and its schema/values definitions, matching the existing pattern used by OTEL connectors.
- Renamed the `otelPluginSpanFilter` / `otel_plugin_span_filter` schema `$defs` entry to `pluginSpanFilter` / `plugin_span_filter` in both `values.schema.json` and `transports/config.schema.json`, and updated all `$ref` usages accordingly. The description was also updated to clarify that the filter applies to any observability connector, not just OTEL.
- Added a full JSON schema definition for a new `bigquery` observability connector in `transports/config.schema.json`, including fields for `project_id`, `dataset_id`, `table_id`, `location`, `service_account_key` (with ADC fallback), `flush_interval_seconds`, `buffer_size`, `custom_labels`, `disable_content_logging`, `request_headers`, and `plugin_span_filter`.
- Added the `bigquery` plugin to the Helm chart (`values.yaml`, `values.schema.json`, and `_helpers.tpl`) with the same `version` validation guard used by other built-in plugins.
- Added commented-out `plugin_span_filter` examples to `values.yaml` for both the Datadog and BigQuery plugins to aid discoverability.

## Type of change

- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI

## Affected areas

- [ ] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs

## How to test

1. Deploy the Helm chart with a Datadog plugin config that includes `plugin_span_filter`:
   ```yaml
   bifrost:
     plugins:
       datadog:
         config:
           plugin_span_filter:
             mode: "exclude"
             plugins: ["logging"]
   ```
   Verify the rendered manifest includes `plugin_span_filter` in the Datadog plugin config.

2. Deploy the Helm chart with the BigQuery plugin enabled:
   ```yaml
   bifrost:
     plugins:
       bigquery:
         enabled: true
         version: 1
         config:
           project_id: "my-gcp-project"
           dataset_id: "bifrost_traces"
           table_id: "traces"
   ```
   Verify the rendered manifest includes the BigQuery plugin config with the expected fields.

3. Validate `transports/config.schema.json` against a BigQuery connector config:
   ```json
   {
     "name": "bigquery",
     "config": {
       "project_id": "my-gcp-project",
       "plugin_span_filter": { "mode": "include", "plugins": ["auth"] }
     }
   }
   ```

4. Confirm that no dangling `$ref` entries referencing the old `otelPluginSpanFilter` / `otel_plugin_span_filter` names remain in either schema file.

## Breaking changes

- [x] Yes
- [ ] No

The `otelPluginSpanFilter` / `otel_plugin_span_filter` `$defs` keys have been renamed to `pluginSpanFilter` / `plugin_span_filter`. Any external tooling or configs that reference these definition names directly will need to be updated.

## Related issues

## Security considerations

The BigQuery connector schema supports `service_account_key` via an environment variable reference (`env.MY_VAR`) or Application Default Credentials, avoiding the need to embed raw credentials in config files. Care should be taken to ensure service account keys are not logged or exposed through the `custom_labels` or `request_headers` fields.

## Checklist

- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants