Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 15 additions & 10 deletions docs/deployment-guides/helm/governance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -390,29 +390,34 @@ bifrost:

## Complexity Router Configuration

If you use `complexity_tier` in routing rules, you can seed the analyzer thresholds and keyword lists from Helm. The chart renders this block to `governance.complexity_analyzer_config` in `config.json`.
If you use `complexity_tier` in routing rules, configure the semantic embedding model and seed its reference phrases from Helm. The chart renders this block to `governance.complexity_analyzer_config` in `config.json`.

Omit this block, or leave `complexityAnalyzerConfig: null`, to use the built-in defaults.
Omit this block, or leave `complexityAnalyzerConfig: null`, to leave semantic complexity classification disabled. To enable it from Helm, provide the `semantic` block and all three keyword lists. In the default split mode, the supplied phrases merge with the stored built-in defaults; with `sourceOfTruth: config.json`, the supplied lists are the complete phrase set.

```yaml
bifrost:
governance:
complexityAnalyzerConfig:
tier_boundaries:
simple_medium: 0.20
medium_complex: 0.40
semantic:
provider: "openai"
embedding_model: "text-embedding-3-small"
timeout: "1.5s"
min_similarity: 0
message_history_count: 1
count_toward_budgets: false
vector_store: "embedded"
keywords:
simple_keywords: ["hello", "hi", "thanks", "what is", "define"]
medium_keywords: ["function", "api", "debug", "architecture", "kubernetes", "latency"]
complex_keywords: ["step by step", "explain why", "tradeoffs", "root cause analysis"]
simple_keywords: ["what is a mutex?", "fix the grammar in this sentence."]
medium_keywords: ["add api-key auth: hash the keys, reject revoked ones, and never log them."]
complex_keywords: ["balance testing, prescribing rules, and staffing against rising resistant infections."]
```
Comment thread
Madhuvod marked this conversation as resolved.

<Note>
In the default split mode, runtime UI and API edits are preserved while the matching Helm-rendered section is unchanged. When Helm changes a section, tier boundaries are replaced from the rendered `config.json`, while keyword lists are merged additively with stored runtime keywords (union with duplicates removed). Use `bifrost.sourceOfTruth: config.json` only when Helm should replace stored governance state. See [Source of Truth & Reconciliation](/deployment-guides/config-json/source-of-truth) for the full startup rules.
In the default split mode, runtime UI and API edits are preserved while the matching Helm-rendered section is unchanged. When Helm changes a section, keyword lists are merged additively with stored runtime phrases (union with duplicates removed), and the semantic block is replaced as one unit. Use `bifrost.sourceOfTruth: config.json` only when Helm should replace stored governance state. See [Source of Truth & Reconciliation](/deployment-guides/config-json/source-of-truth) for the full startup rules.
</Note>

<Note>
Existing releases that still use `code_keywords`, `technical_keywords`, and `reasoning_keywords` remain valid during upgrade. Bifrost reads them as Medium (`code` + `technical`) and Complex (`reasoning`) signals. New configurations should use the three-list shape shown above.
Existing releases that still use `code_keywords`, `technical_keywords`, and `reasoning_keywords` remain valid during upgrade. Bifrost maps them into the three reference-phrase lists; new configurations should use the three-list shape shown above. Legacy `tier_boundaries` also remain accepted when present, but are optional and ignored by semantic routing.
</Note>

---
Expand Down
305 changes: 147 additions & 158 deletions docs/features/governance/complexity-router.mdx

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion docs/features/observability/prometheus.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,8 @@ The `path` label contains the matched route template (e.g. `/genai/v1beta/models
| `bifrost_provider_key_up` | Gauge | Per-key health. `1` after a successful attempt, `0` after a failed attempt. Labels: `provider`, `key_id`, `key_name`. |
| `bifrost_key_rotation_events_total` | Counter | Key rotations triggered by per-key failures — rate-limit (429), auth (401/403), or billing (402) — see below <sup>v1.5.0-prerelease4+</sup> |
| `bifrost_request_retries` | Histogram | Number of retries used per request (observed once per request; buckets `0,1,2,3,5,10`). |
| `bifrost_routing_embedding_requests_total` | Counter | Embedding calls made by semantic complexity routing. Labels: `provider`, `model` (the embedding provider/model, not the request's), `phase` (`request` classification vs `warmup` exemplar embedding). |
| `bifrost_routing_embedding_cost_total` | Counter | Cost in USD of semantic routing embeddings (same labels as above). Recorded regardless of whether embedding usage counts toward budgets. |

### Bifrost MCP Metrics

Expand All @@ -237,7 +239,7 @@ Most request-level Bifrost LLM metrics include these labels (the `bifrost_key_ro
- `routing_engine_used` - Comma-separated list of routing engines that contributed to the decision (e.g. `governance`, `routing-rule`, `loadbalancing`, `model-catalog`, `core`). `core` is emitted when the Bifrost orchestrator itself makes a routing decision — fallback transitions or retry transitions.
- `routing_rule_id` / `routing_rule_name` - Routing rule that matched the request
- `complexity_tier` - Complexity tier used for routing (`SIMPLE` / `MEDIUM` / `COMPLEX`); empty when no routing rule referenced `complexity_tier`
- `complexity_mechanism` - How the complexity tier was classified (`lexical`, or `skipped` when classification was demanded but produced no tier). The raw complexity score is deliberately not a label — it has unbounded cardinality and lives only in the request logs
- `complexity_mechanism` - How the complexity tier was classified (`semantic` for the embedding-based classifier, or `skipped` when classification was demanded but produced no tier). The raw complexity score is deliberately not a label because it has unbounded cardinality and lives only in the request logs
- `selected_key_id` / `selected_key_name` - API key that successfully served the request (`""` when all attempts failed)
- `fallback_index` - Fallback position
- `team_id` / `team_name` - Team identifiers (empty when governance is not used)
Expand Down
2 changes: 1 addition & 1 deletion docs/features/telemetry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ Base Labels:
- `routing_rule_id`: Routing rule ID that matched the request
- `routing_rule_name`: Routing rule name that matched the request
- `complexity_tier`: Complexity tier used for routing (`SIMPLE` / `MEDIUM` / `COMPLEX`); empty when no routing rule referenced `complexity_tier`
- `complexity_mechanism`: How the complexity tier was classified (`lexical`, or `skipped` when classification was demanded but produced no tier). The raw complexity score is deliberately not a label — unbounded cardinality; it lives only in the request logs
- `complexity_mechanism`: How the complexity tier was classified (`semantic` for the embedding-based classifier, or `skipped` when classification was demanded but produced no tier). The raw complexity score is deliberately not a label because it has unbounded cardinality; it lives only in the request logs
- `selected_key_id`: ID of the key that successfully served the request (empty string `""` on final errors)
- `selected_key_name`: Name of the key that successfully served the request (empty string `""` on final errors)
- `fallback_index`: Fallback index (0 for first attempt, 1 for second attempt, etc.)
Expand Down
Binary file removed docs/media/architecture-complexity-router.png
Binary file not shown.
Binary file removed docs/media/complexity-logic-architecture.png
Binary file not shown.
Binary file removed docs/media/ui-complexity-router-config.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/media/ui-complexity-router-keywords.png
Binary file not shown.
Binary file added docs/media/ui-complexity-router-semantic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/media/ui-routing-logs-complexity.png
Binary file not shown.
Binary file removed docs/media/ui-routing-rule-complexity.png
Binary file not shown.
Loading
Loading