feat: adds PubSub & Kafka to config json with plugin span filter - #4331
Merged
akshaydeo merged 1 commit intoJun 12, 2026
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughAdds Kafka and Google Cloud Pub/Sub telemetry exporters: transport JSON schemas, Helm values schema and disabled-by-default defaults, and Helm template logic + validation to instantiate them when enabled. ChangesKafka and Pub/Sub Telemetry Exporters
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
|
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
Contributor
Confidence Score: 5/5Safe to merge — adds Helm chart support for two new disabled-by-default plugins with no changes to runtime Go code or existing plugin behavior. All four changed files are Helm chart artifacts. The rendering logic, validation fail-directives, and JSON Schema conditional blocks faithfully follow the existing BigQuery plugin pattern. Required-field enforcement is handled both at the schema level and by template fail-directives, so no misconfiguration can silently reach the runtime. No files require special attention. Important Files Changed
Reviews (2): Last reviewed commit: "feat: adds PubSub & Kafka to config json..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@helm-charts/bifrost/templates/_helpers.tpl`:
- Around line 1326-1353: When bifrost.plugins.pubsub.enabled is true the
template doesn't validate required fields and version bounds; update the
bifrost.validate template (insert after the Kafka validation block, before the
final {{- end -}}) to check .Values.bifrost.plugins.pubsub.enabled and then
assert that .Values.bifrost.plugins.pubsub.config.project_id and
.Values.bifrost.plugins.pubsub.config.topic_id are present (non-empty), and if
.Values.bifrost.plugins.pubsub.version is provided ensure it's an integer
between 1 and 32767; fail-fast by emitting a validation error when these checks
fail so the $pubsubConfig/$plugin block doesn't render invalid config (use the
same validation pattern used for Kafka validation and reference
transports/config.schema.json as the source of truth).
- Around line 1280-1325: When bifrost.plugins.kafka.enabled is true the template
must validate required fields to fail fast: inside the bifrost.validate template
add checks that .Values.bifrost.plugins.kafka.config.brokers exists and is a
non-empty array and that .Values.bifrost.plugins.kafka.config.topic exists and
is non-empty, and if .Values.bifrost.plugins.kafka.version is set ensure it
parses to an integer between 1 and 32767; implement these using Helm helpers
(required/fail and int/gt/lt or comparisons) referencing the same symbols used
in the plugin block ($kafkaConfig, $plugin, .Values.bifrost.plugins.kafka) so
the chart errors during rendering rather than letting Bifrost fail at runtime.
In `@helm-charts/bifrost/values.schema.json`:
- Around line 1018-1102: The Helm JSON schema lacks conditional validation to
require transport identifiers when enabled; add JSON Schema if/then blocks so
when kafka.enabled is true the kafka.config must require brokers and topic
(e.g., add an "if": {"properties": {"enabled": {"const": true}}} with a "then":
{"properties": {"config": {"required": ["brokers","topic"]}}}) and likewise for
the pubsub object (when pubsub.enabled is true require config.project_id and
config.topic_id). Reference the top-level "kafka" object and its
"config.brokers"/"config.topic" keys and the "pubsub" object with
"config.project_id"/"config.topic_id" when adding these conditional required
rules; keep additionalProperties and existing $ref entries intact.
- Around line 1044-1058: The Helm values schema restricts
kafka.config.sasl.username, kafka.config.sasl.password and kafka.config.ca_cert
to plain strings but the transport contract accepts the EnvVar object shape
{value, env_var, from_env}; update the schema definitions for "sasl.username",
"sasl.password" and the "ca_cert" property to accept either a string OR the
canonical EnvVar object (match the shape used in transports/config.schema.json)
so Helm validation does not reject valid transport configs; ensure you reuse the
same property name tokens (kafka.config.sasl.username,
kafka.config.sasl.password, kafka.config.ca_cert) and mirror the exact EnvVar
object property names/required fields from the transport schema.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: dc07b2e6-657c-4be4-94a8-c6ebda61f583
📒 Files selected for processing (4)
helm-charts/bifrost/templates/_helpers.tplhelm-charts/bifrost/values.schema.jsonhelm-charts/bifrost/values.yamltransports/config.schema.json
roroghost17
force-pushed
the
06-12-feat_adds_pubsub_kafka_to_config_json_with_plugin_span_filter
branch
from
June 12, 2026 09:03
44f14ea to
2b9a709
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@helm-charts/bifrost/values.schema.json`:
- Around line 1028-1133: kafka.config and pubsub.config objects in the Helm
values schema currently allow unknown keys which diverges from the strict
transport contract; update the "config" object definitions (the ones containing
properties like brokers, topic, sasl, tls_enabled, ca_cert, compression,
batch_size, flush_interval_ms, auto_create_topic, disable_content_logging,
request_headers, plugin_span_filter) to be strict by adding
"additionalProperties": false so unknown keys are rejected, and ensure the
resulting schema matches transports/config.schema.json for each transport config
shape.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: b2df4487-9779-47f9-b2f9-65e96601ac7b
📒 Files selected for processing (4)
helm-charts/bifrost/templates/_helpers.tplhelm-charts/bifrost/values.schema.jsonhelm-charts/bifrost/values.yamltransports/config.schema.json
Contributor
Merge activity
|
akshaydeo
deleted the
06-12-feat_adds_pubsub_kafka_to_config_json_with_plugin_span_filter
branch
June 12, 2026 09:12
akshaydeo
pushed a commit
that referenced
this pull request
Jun 12, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 15, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 16, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 19, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 19, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 21, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 21, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 21, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 21, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 22, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 22, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 24, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 24, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 25, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jun 25, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 1, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 1, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 4, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 4, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 5, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 5, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 9, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 9, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 9, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 9, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 9, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 9, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 13, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 13, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 14, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 14, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 24, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 24, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 29, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 29, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 30, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 30, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 30, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 30, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 30, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
pushed a commit
that referenced
this pull request
Jul 30, 2026
## Summary Adds Helm chart support for two new Enterprise trace-publishing plugins: **Kafka** and **Google Cloud Pub/Sub**. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively. ## Changes - Added Kafka plugin rendering logic to `_helpers.tpl`, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options. - Added Pub/Sub plugin rendering logic to `_helpers.tpl`, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering. - Added JSON schema definitions for both `kafka` and `pubsub` plugin configs in `values.schema.json`, including field-level descriptions, enums, and `anyOf` patterns for env-var-substitutable fields. - Added conditional schema validation blocks in `transports/config.schema.json` for both plugins, enforcing required fields (`brokers`/`topic` for Kafka; `project_id`/`topic_id` for Pub/Sub). - Added default-disabled entries for both plugins in `values.yaml` with inline documentation comments covering all supported options. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Enable the Kafka plugin in your `values.yaml` and confirm the rendered config contains the expected plugin block: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.kafka.enabled=true \ --set bifrost.plugins.kafka.config.brokers[0]=localhost:9092 \ --set bifrost.plugins.kafka.config.topic=traces ``` Enable the Pub/Sub plugin and verify the rendered output: ```sh helm template bifrost ./helm-charts/bifrost \ --set bifrost.plugins.pubsub.enabled=true \ --set bifrost.plugins.pubsub.config.project_id=my-project \ --set bifrost.plugins.pubsub.config.topic_id=my-topic ``` Validate the schema against a config file: ```sh # Validate transports config schema npx ajv validate -s transports/config.schema.json -d <your-config.json> ``` **New config fields (Kafka):** | Field | Description | Default | |---|---|---| | `brokers` | Kafka broker addresses | required | | `topic` | Topic to publish traces to | required | | `sasl_enabled` | Enable SASL authentication | `false` | | `tls_enabled` | Enable TLS for broker connections | `false` | | `compression` | Codec: `none`, `gzip`, `snappy`, `lz4`, `zstd` | `none` | | `batch_size` | Max messages per batch | `100` | | `flush_interval_ms` | Max ms before flushing a batch | `1000` | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip message content from traces | `false` | **New config fields (Pub/Sub):** | Field | Description | Default | |---|---|---| | `project_id` | GCP project ID | required | | `topic_id` | Pub/Sub topic ID | required | | `service_account_key` | SA key JSON or `env.VAR`; omit for ADC | — | | `auto_create_topic` | Create topic at startup if missing | `false` | | `disable_content_logging` | Strip content from traces | `false` | ## Breaking changes - [ ] Yes - [x] No ## Security considerations - SASL credentials (`username`, `password`) and the Pub/Sub `service_account_key` support `env.VAR_NAME` substitution to avoid embedding secrets directly in config files. - TLS CA certificates for Kafka also support env-var substitution. - Both plugins support `disable_content_logging` to prevent PII or sensitive payload data from being published to the message broker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Kafka and Pub/Sub as telemetry export backends for distributed tracing. * Both plugins are disabled by default and expose configuration for brokers/project/topic, optional auth (SASL/TLS/service account), compression/batching/flush, topic auto-creation, request header capture, and content/span filtering. * Chart/values and schema updates add configuration stubs and validation for these plugins. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Helm chart support for two new Enterprise trace-publishing plugins: Kafka and Google Cloud Pub/Sub. These plugins allow Bifrost to publish completed traces as JSON messages to a Kafka topic or a GCP Pub/Sub topic respectively.
Changes
_helpers.tpl, supporting broker addresses, topic, SASL authentication, TLS, compression, batching, and span filtering options._helpers.tpl, supporting GCP project/topic IDs, service account key authentication (or ADC), auto topic creation, content logging controls, and span filtering.kafkaandpubsubplugin configs invalues.schema.json, including field-level descriptions, enums, andanyOfpatterns for env-var-substitutable fields.transports/config.schema.jsonfor both plugins, enforcing required fields (brokers/topicfor Kafka;project_id/topic_idfor Pub/Sub).values.yamlwith inline documentation comments covering all supported options.Type of change
Affected areas
How to test
Enable the Kafka plugin in your
values.yamland confirm the rendered config contains the expected plugin block:Enable the Pub/Sub plugin and verify the rendered output:
Validate the schema against a config file:
New config fields (Kafka):
brokerstopicsasl_enabledfalsetls_enabledfalsecompressionnone,gzip,snappy,lz4,zstdnonebatch_size100flush_interval_ms1000auto_create_topicfalsedisable_content_loggingfalseNew config fields (Pub/Sub):
project_idtopic_idservice_account_keyenv.VAR; omit for ADCauto_create_topicfalsedisable_content_loggingfalseBreaking changes
Security considerations
username,password) and the Pub/Subservice_account_keysupportenv.VAR_NAMEsubstitution to avoid embedding secrets directly in config files.disable_content_loggingto prevent PII or sensitive payload data from being published to the message broker.Checklist
docs/contributing/README.mdand followed the guidelinesSummary by CodeRabbit