Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion core/schemas/bifrost.go
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ const (
BifrostContextKeyGovernanceRoutingRuleID BifrostContextKey = "bifrost-governance-routing-rule-id" // string (to store the routing rule ID (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeyGovernanceRoutingRuleName BifrostContextKey = "bifrost-governance-routing-rule-name" // string (to store the routing rule name (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeyGovernanceComplexityTier BifrostContextKey = "bifrost-governance-complexity-tier" // string (complexity tier computed for routing, e.g. "SIMPLE"/"MEDIUM"/"COMPLEX"; only present when a routing rule referenced complexity_tier and classification produced a tier (set by bifrost routing plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeyGovernanceComplexityMechanism BifrostContextKey = "bifrost-governance-complexity-mechanism" // string (how the complexity tier was classified: "semantic", or "skipped" when classification was demanded but produced no tier; only present when a routing rule referenced complexity_tier (set by bifrost routing plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeyGovernanceComplexityMechanism BifrostContextKey = "bifrost-governance-complexity-mechanism" // string (how the effective complexity tier was determined: "semantic", "llm", "session", or "skipped" when classification was demanded but produced no tier; only present when a routing rule referenced complexity_tier (set by bifrost routing plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeyGovernanceComplexityScore BifrostContextKey = "bifrost-governance-complexity-score" // float64 (classifier score behind the tier: the semantic classifier's similarity to the nearest reference phrase; only present alongside a computed tier (set by bifrost routing plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeyRoutingPinnedAPIKeyID BifrostContextKey = "bifrost-routing-pinned-api-key-id" // string (provider key ID pinned by a matched routing rule target; resolved against the configured key pool during key selection and takes precedence over a caller-supplied pin (set by bifrost governance plugin - DO NOT SET THIS MANUALLY))
BifrostContextKeySelectedPromptName BifrostContextKey = "bifrost-selected-prompt-name" // string (display name of the selected prompt (set by prompts plugin - DO NOT SET THIS MANUALLY))
Expand Down
4 changes: 4 additions & 0 deletions docs/deployment-guides/helm/governance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -406,12 +406,16 @@ bifrost:
message_history_count: 1
count_toward_budgets: false
vector_store: "embedded"
session:
enabled: true
keywords:
simple_keywords: ["what is a mutex?", "fix the grammar in this sentence."]
medium_keywords: ["add api-key auth: hash the keys, reject revoked ones, and never log them."]
complex_keywords: ["balance testing, prescribing rules, and staffing against rising resistant infections."]
```

`session.enabled` is optional and defaults to `false`. When enabled, an identified session retains its highest observed tier for 24 hours of inactivity; normally sequential turns can escalate while lower proposals keep the stored tier. Overlapping requests for the same session are best-effort and resolve by last writer wins. The lifetime is built in and is separate from provider prompt-cache TTLs.

<Note>
In the default split mode, runtime UI and API edits are preserved while the matching Helm-rendered section is unchanged. When Helm changes a section, keyword lists are merged additively with stored runtime phrases (union with duplicates removed), and the semantic block is replaced as one unit. Use `bifrost.sourceOfTruth: config.json` only when Helm should replace stored governance state. See [Source of Truth & Reconciliation](/deployment-guides/config-json/source-of-truth) for the full startup rules.
</Note>
Expand Down
63 changes: 43 additions & 20 deletions docs/features/governance/complexity-router.mdx

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/features/observability/datadog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ When a routing rule references `complexity_tier`, two additional tags are set:
- `complexity_tier` - The complexity tier the request was classified into: `SIMPLE`, `MEDIUM`, or `COMPLEX` (Datadog normalizes tag values to lowercase, so query as `simple`/`medium`/`complex`)
- `complexity_mechanism` - How the tier was classified: `semantic`, or `skipped` when classification ran but produced no tier

Use them to attribute cost, latency, and volume to classified complexity, e.g. `sum:bifrost.request.cost.usd{complexity_tier:complex} by {model}`. The raw complexity score is not exported as a tag (its cardinality is unbounded); it is available only in the log store.
Use them to attribute cost, latency, and volume to classified complexity, e.g. `sum:bifrost.request.cost.usd{complexity_tier:complex} by {model}`. The raw complexity score is not exported as a metric tag because its cardinality is unbounded; it remains available in request logs and trace attributes.

---

Expand Down
2 changes: 1 addition & 1 deletion docs/features/observability/prometheus.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ Most request-level Bifrost LLM metrics include these labels (the `bifrost_key_ro
- `routing_engine_used` - Comma-separated list of routing engines that contributed to the decision (e.g. `governance`, `routing-rule`, `loadbalancing`, `model-catalog`, `core`). `core` is emitted when the Bifrost orchestrator itself makes a routing decision — fallback transitions or retry transitions.
- `routing_rule_id` / `routing_rule_name` - Routing rule that matched the request
- `complexity_tier` - Complexity tier used for routing (`SIMPLE` / `MEDIUM` / `COMPLEX`); empty when no routing rule referenced `complexity_tier`
- `complexity_mechanism` - How the complexity tier was classified (`semantic` for the embedding-based classifier, or `skipped` when classification was demanded but produced no tier). The raw complexity score is deliberately not a label because it has unbounded cardinality and lives only in the request logs
- `complexity_mechanism` - How the effective complexity tier was determined (`semantic`, `llm`, `session`, or `skipped` when no tier was produced). The raw complexity score is deliberately not a label because it has unbounded cardinality; it remains available in request logs and trace attributes
- `selected_key_id` / `selected_key_name` - API key that successfully served the request (`""` when all attempts failed)
- `fallback_index` - Fallback position
- `team_id` / `team_name` - Team identifiers (empty when governance is not used)
Expand Down
2 changes: 1 addition & 1 deletion docs/features/telemetry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Base Labels:
- `routing_rule_id`: Routing rule ID that matched the request
- `routing_rule_name`: Routing rule name that matched the request
- `complexity_tier`: Complexity tier used for routing (`SIMPLE` / `MEDIUM` / `COMPLEX`); empty when no routing rule referenced `complexity_tier`
- `complexity_mechanism`: How the complexity tier was classified (`semantic` for the embedding-based classifier, or `skipped` when classification was demanded but produced no tier). The raw complexity score is deliberately not a label because it has unbounded cardinality; it is recorded in request logs and traces
- `complexity_mechanism`: How the effective complexity tier was determined (`semantic`, `llm`, `session`, or `skipped` when no tier was produced). The raw complexity score is deliberately not a label because it has unbounded cardinality; it is recorded in request logs and traces
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- `selected_key_id`: ID of the key that successfully served the request (empty string `""` on final errors)
- `selected_key_name`: Name of the key that successfully served the request (empty string `""` on final errors)
- `fallback_index`: Fallback index (0 for first attempt, 1 for second attempt, etc.)
Expand Down
Binary file modified docs/media/ui-complexity-router-embedding-configuration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/media/ui-complexity-router-semantic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading