Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions helm-charts/bifrost/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Official Helm charts for deploying [Bifrost](https://github.com/maximhq/bifrost)
- Added `bifrost.plugins.splunk` — the Splunk HTTP Event Collector (HEC) observability connector (Enterprise): one flattened event per request to `events_index` plus the derived metric set to `metrics_index`, with TLS (`ca_cert` / `insecure_skip_verify`), a content toggle (`disable_content_logging`), request-header capture, and indexer acknowledgement (`indexer_ack`, `ack_poll_interval_ms`, `ack_timeout_ms`, `max_ack_attempts`). Renders into the `splunk` plugin config.
- Added `bifrost.plugins.otel.config.semaphore_size` and `inject_timeout` (plugin-level, both legacy and `profiles` wrapper shapes, default `10000` / `5`) — cap on concurrent in-flight trace injects and the timeout for a single inject call, so a hung collector can't hold its concurrency slot indefinitely. Renders into `semaphore_size` / `inject_timeout`. `bifrost.plugins.logging.config` accepts the same two keys (`inject_timeout` as a duration string, e.g. `"5s"`), passed through as-is.
- Added `postgresql.primary.nodeSelector`, `postgresql.primary.tolerations`, and `postgresql.primary.affinity` to the hosted PostgreSQL deployment. Previously only the Bifrost pod itself could be steered (top-level `nodeSelector`/`tolerations`/`affinity`), so on clusters that mix long-lived services with ephemeral autoscaled workloads the hosted database could not be kept off nodes that scale in — and draining the single-replica Postgres takes the gateway down with it. All three default to empty, so rendering is unchanged unless set.
- Added `storage.logsStore.postgres` to point the logs store at a **separate external PostgreSQL** (different host and/or database) than the config store, instead of forcing both onto the shared top-level `postgresql` connection. Only applies when the logs store resolves to postgres; `enabled: false` (default) preserves existing behavior. Fields mirror `postgresql.external` (`host`, `port`, `user`, `password`, `passwordCommand`, `database`, `sslMode`, `connMaxLifetime`, `existingSecret`, `passwordKey`); with `existingSecret` the password is injected as `BIFROST_LOGS_POSTGRES_PASSWORD`. Renders into `logs_store.config`.

### 2.1.36

Expand Down Expand Up @@ -672,6 +673,7 @@ Bifrost supports two storage backends (SQLite and PostgreSQL) that can be config
| `storage.configStore.type` | Config store backend: `sqlite`, `postgres`, or `""` | `""` (uses `storage.mode`) |
| `storage.logsStore.enabled` | Enable logs store | `true` |
| `storage.logsStore.type` | Logs store backend: `sqlite`, `postgres`, or `""` | `""` (uses `storage.mode`) |
| `storage.logsStore.postgres.enabled` | Point the logs store at a separate external PostgreSQL than the config store (only applies when the logs store is postgres). When `false`, a postgres logs store shares the top-level `postgresql` connection. | `false` |
| `storage.logsStore.objectStorageExcludeFields` | Payload DB fields to keep in DB instead of offloading to object storage | `[]` |

#### Mixed Backend Example
Expand All @@ -693,6 +695,47 @@ postgresql:
# ... PostgreSQL configuration for logs store
```

#### Separate PostgreSQL for Logs

When both stores use PostgreSQL, they share the single top-level `postgresql`
connection by default. To send high-volume logs to a **different** external
PostgreSQL (a separate host and/or database) while the config store keeps using
the shared `postgresql` block, set `storage.logsStore.postgres`. Its fields mirror
`postgresql.external`; when `existingSecret` is set the password is injected as
`BIFROST_LOGS_POSTGRES_PASSWORD`.

```yaml
storage:
mode: postgres
configStore:
enabled: true
type: postgres # Config -> shared postgresql block below
logsStore:
enabled: true
type: postgres # Logs -> separate postgres below
postgres:
enabled: true
host: logs-db.example.com
port: 5432
user: bifrost
database: bifrost_logs
sslMode: require
existingSecret: bifrost-logs-postgres # key: password
# passwordKey: password

postgresql:
external:
enabled: true # Config store's database
host: config-db.example.com
port: 5432
user: bifrost
database: bifrost
sslMode: require
existingSecret: bifrost-config-postgres
```

See `values-examples/separate-logs-postgres.yaml` for a complete example.

### PostgreSQL Configuration

| Parameter | Description | Default |
Expand Down Expand Up @@ -965,6 +1008,7 @@ The chart includes pre-configured examples in `values-examples/`:
| `sqlite-only.yaml` | Simple setup with SQLite (local development) |
| `postgres-only.yaml` | PostgreSQL for config and logs |
| `mixed-backend.yaml` | SQLite for config + PostgreSQL for logs (mixed backend) |
| `separate-logs-postgres.yaml` | Config and logs on separate PostgreSQL databases |
| `postgres-weaviate.yaml` | PostgreSQL + Weaviate for semantic caching |
| `postgres-redis.yaml` | PostgreSQL + Redis for semantic caching |
| `postgres-qdrant.yaml` | PostgreSQL + Qdrant for semantic caching |
Expand Down
69 changes: 68 additions & 1 deletion helm-charts/bifrost/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,63 @@ disable
{{- end -}}
{{- end -}}

{{- /*
Logs-store PostgreSQL helpers. When storage.logsStore.postgres.enabled is true,
the logs store points at a separate external PostgreSQL; otherwise every helper
falls back to the shared bifrost.postgresql.* helpers so behavior is unchanged.
*/ -}}
{{- define "bifrost.logsPostgresql.host" -}}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore -}}
{{- .Values.storage.logsStore.postgres.host -}}
{{- else -}}
{{- include "bifrost.postgresql.host" . -}}
{{- end -}}
{{- end -}}

{{- define "bifrost.logsPostgresql.port" -}}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore -}}
{{- .Values.storage.logsStore.postgres.port -}}
{{- else -}}
{{- include "bifrost.postgresql.port" . -}}
{{- end -}}
{{- end -}}

{{- define "bifrost.logsPostgresql.database" -}}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore -}}
{{- .Values.storage.logsStore.postgres.database -}}
{{- else -}}
{{- include "bifrost.postgresql.database" . -}}
{{- end -}}
{{- end -}}

{{- define "bifrost.logsPostgresql.username" -}}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore -}}
{{- .Values.storage.logsStore.postgres.user -}}
{{- else -}}
{{- include "bifrost.postgresql.username" . -}}
{{- end -}}
{{- end -}}

{{- define "bifrost.logsPostgresql.password" -}}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore -}}
{{- if .Values.storage.logsStore.postgres.existingSecret -}}
env.BIFROST_LOGS_POSTGRES_PASSWORD
{{- else -}}
{{- .Values.storage.logsStore.postgres.password -}}
{{- end -}}
{{- else -}}
{{- include "bifrost.postgresql.password" . -}}
{{- end -}}
{{- end -}}

{{- define "bifrost.logsPostgresql.sslMode" -}}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore -}}
{{- .Values.storage.logsStore.postgres.sslMode -}}
{{- else -}}
{{- include "bifrost.postgresql.sslMode" . -}}
{{- end -}}
{{- end -}}

{{- define "bifrost.weaviate.host" -}}
{{- if .Values.vectorStore.weaviate.external.enabled }}
{{- .Values.vectorStore.weaviate.external.host }}
Expand Down Expand Up @@ -903,14 +960,24 @@ false
{{- if .Values.storage.logsStore.enabled }}
{{- $logsStoreType := .Values.storage.logsStore.type | default .Values.storage.mode }}
{{- if eq $logsStoreType "postgres" }}
{{- $pgConfig := dict "host" (include "bifrost.postgresql.host" .) "port" (include "bifrost.postgresql.port" .) "db_name" (include "bifrost.postgresql.database" .) "user" (include "bifrost.postgresql.username" .) "password" (include "bifrost.postgresql.password" .) "ssl_mode" (include "bifrost.postgresql.sslMode" .) }}
{{- $pgConfig := dict "host" (include "bifrost.logsPostgresql.host" .) "port" (include "bifrost.logsPostgresql.port" .) "db_name" (include "bifrost.logsPostgresql.database" .) "user" (include "bifrost.logsPostgresql.username" .) "password" (include "bifrost.logsPostgresql.password" .) "ssl_mode" (include "bifrost.logsPostgresql.sslMode" .) }}
{{- if dig "postgres" "enabled" false .Values.storage.logsStore }}
{{- if .Values.storage.logsStore.postgres.passwordCommand }}
{{- $_ := set $pgConfig "password_command" .Values.storage.logsStore.postgres.passwordCommand }}
{{- $_ := unset $pgConfig "password" }}
{{- end }}
{{- if .Values.storage.logsStore.postgres.connMaxLifetime }}
{{- $_ := set $pgConfig "conn_max_lifetime" .Values.storage.logsStore.postgres.connMaxLifetime }}
{{- end }}
{{- else }}
{{- if and .Values.postgresql.external.enabled .Values.postgresql.external.passwordCommand }}
{{- $_ := set $pgConfig "password_command" .Values.postgresql.external.passwordCommand }}
{{- $_ := unset $pgConfig "password" }}
{{- end }}
{{- if and .Values.postgresql.external.enabled .Values.postgresql.external.connMaxLifetime }}
{{- $_ := set $pgConfig "conn_max_lifetime" .Values.postgresql.external.connMaxLifetime }}
{{- end }}
{{- end }}
{{- if .Values.storage.logsStore.maxIdleConns }}
{{- $_ := set $pgConfig "max_idle_conns" (.Values.storage.logsStore.maxIdleConns | int) }}
{{- end }}
Expand Down
9 changes: 9 additions & 0 deletions helm-charts/bifrost/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,15 @@ spec:
name: {{ .Values.postgresql.auth.existingSecret }}
key: {{ .Values.postgresql.auth.passwordKey | default "password" }}
{{- end }}
{{- /* Separate logs-store PostgreSQL password from existing secret. Only inject when the logs store is enabled and resolves to postgres, mirroring the config block. */ -}}
{{- $logsStoreType := .Values.storage.logsStore.type | default .Values.storage.mode }}
{{- if and .Values.storage.logsStore.enabled (eq $logsStoreType "postgres") (dig "postgres" "enabled" false .Values.storage.logsStore) (dig "postgres" "existingSecret" "" .Values.storage.logsStore) }}
- name: BIFROST_LOGS_POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.storage.logsStore.postgres.existingSecret }}
key: {{ .Values.storage.logsStore.postgres.passwordKey | default "password" }}
{{- end }}
{{- /* Redis password from existing secret */ -}}
{{- if and .Values.vectorStore.enabled (eq .Values.vectorStore.type "redis") .Values.vectorStore.redis.external.enabled .Values.vectorStore.redis.external.existingSecret }}
- name: BIFROST_REDIS_PASSWORD
Expand Down
9 changes: 9 additions & 0 deletions helm-charts/bifrost/templates/stateful.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,15 @@ spec:
name: {{ .Values.postgresql.auth.existingSecret }}
key: {{ .Values.postgresql.auth.passwordKey | default "password" }}
{{- end }}
{{- /* Separate logs-store PostgreSQL password from existing secret. Only inject when the logs store is enabled and resolves to postgres, mirroring the config block. */ -}}
{{- $logsStoreType := .Values.storage.logsStore.type | default .Values.storage.mode }}
{{- if and .Values.storage.logsStore.enabled (eq $logsStoreType "postgres") (dig "postgres" "enabled" false .Values.storage.logsStore) (dig "postgres" "existingSecret" "" .Values.storage.logsStore) }}
- name: BIFROST_LOGS_POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Values.storage.logsStore.postgres.existingSecret }}
key: {{ .Values.storage.logsStore.postgres.passwordKey | default "password" }}
{{- end }}
{{- /* Redis password from existing secret */ -}}
{{- if and .Values.vectorStore.enabled (eq .Values.vectorStore.type "redis") .Values.vectorStore.redis.external.enabled .Values.vectorStore.redis.external.existingSecret }}
- name: BIFROST_REDIS_PASSWORD
Expand Down
55 changes: 55 additions & 0 deletions helm-charts/bifrost/values-examples/separate-logs-postgres.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Configuration: config store and logs store on SEPARATE PostgreSQL databases
# The config store uses the shared `postgresql` connection (here, an external RDS),
# while the logs store points at a different external PostgreSQL via
# storage.logsStore.postgres. Useful when logs are high-volume and you want them on
# a dedicated database/instance, isolated from config.
# Usage: helm install bifrost ./bifrost -f values-examples/separate-logs-postgres.yaml

storage:
mode: postgres
configStore:
enabled: true
type: postgres # Config store -> shared postgresql block below
logsStore:
enabled: true
type: postgres # Logs store -> separate postgres below
# Point logs at a different external PostgreSQL than the config store.
postgres:
enabled: true
host: "logs-db.example.com"
port: 5432
user: bifrost
database: bifrost_logs
sslMode: require
# Provide the password via an existing secret (recommended). existingSecret
# takes precedence: when set, the chart injects BIFROST_LOGS_POSTGRES_PASSWORD
# from this secret and ignores `password` below.
existingSecret: "bifrost-logs-postgres"
passwordKey: "password"
# ... or inline via an env var (avoid in production). To use this, REMOVE
# existingSecret above and supply LOGS_POSTGRES_PASSWORD yourself through the
# chart's top-level env / extraEnv / envFrom (the chart does not set it for you):
# password: "env.LOGS_POSTGRES_PASSWORD"

# Shared PostgreSQL connection used by the config store (external instance here).
postgresql:
enabled: false
external:
enabled: true
host: "config-db.example.com"
port: 5432
user: bifrost
database: bifrost
sslMode: require
existingSecret: "bifrost-config-postgres"
passwordKey: "password"

# No vector store
vectorStore:
enabled: false
type: none

bifrost:
client:
enableLogging: true
providers: {}
101 changes: 101 additions & 0 deletions helm-charts/bifrost/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -4395,6 +4395,107 @@
"description": "How long an idle physical connection is kept before being closed (default 5m). Without this the idle cap alone controls pool size, so bursts above maxIdleConns close and reopen connections. The config store and logs store open separate pools.",
"pattern": "^[1-9][0-9]*(ns|us|µs|ms|s|m|h)$"
},
"postgres": {
"type": "object",
"description": "Point the logs store at a separate external PostgreSQL than the config store (only applies when the logs store resolves to postgres). When enabled:false the postgres logs store shares the top-level postgresql connection.",
"properties": {
"enabled": {
"type": "boolean"
},
"host": {
"type": "string"
},
"port": {
"description": "PostgreSQL port. Accepts an integer (e.g. 5432) or an env.VAR_NAME string for environment-variable substitution.",
"anyOf": [
{
"type": "integer",
"minimum": 1,
"maximum": 65535
},
{
"type": "string"
}
]
},
"user": {
"type": "string"
},
"password": {
"type": "string"
},
"passwordCommand": {
"type": "object",
"description": "Command executed by Bifrost to produce the PostgreSQL password on stdout. The result is cached for cache_ttl and shared across new physical connections",
"properties": {
"command": {
"type": "string",
"minLength": 1,
"pattern": "^\\S+$"
},
"args": {
"type": "array",
"items": {
"type": "string"
}
},
"timeout": {
"type": "string",
"pattern": "^[1-9][0-9]*(ns|us|µs|ms|s|m|h)$"
},
"cache_ttl": {
"type": "string",
"description": "How long a resolved password is reused across new physical connections (default 60s). Keep below the credential's validity window.",
"pattern": "^[1-9][0-9]*(ns|us|µs|ms|s|m|h)$"
}
},
"required": ["command"],
"additionalProperties": false
},
"connMaxLifetime": {
"type": "string",
"pattern": "^[1-9][0-9]*(ns|us|µs|ms|s|m|h)$"
},
"database": {
"type": "string"
},
"sslMode": {
"type": "string",
"enum": ["disable", "allow", "prefer", "require", "verify-ca", "verify-full"]
},
"existingSecret": {
"type": "string"
},
"passwordKey": {
"type": "string"
}
},
"allOf": [
{
"if": {
"properties": {
"enabled": {
"const": true
}
}
},
"then": {
"required": ["host", "port", "user", "database", "sslMode"]
}
},
{
"not": {
"required": ["password", "passwordCommand"]
}
},
{
"not": {
"required": ["existingSecret", "passwordCommand"]
}
}
],
"additionalProperties": false
},
"matviewRefreshInterval": {
"type": "string",
"description": "How often to refresh materialized views. Go duration string (e.g. '30s', '5m', '1h'). Minimum 5s.",
Expand Down
Loading
Loading