Fix/preserve model metadata - #4301
Open
kristiandrucker wants to merge 112 commits into
Open
Conversation
## Summary Adds an "Allow Private Network" toggle to the custom provider creation form, enabling users to configure whether a custom provider can connect to private network IP ranges (e.g., `192.168.x.x`, `10.x.x.x`). Link-local addresses remain blocked regardless of this setting. ## Changes - Added `allow_private_network` as an optional boolean field to the custom provider form schema, defaulting to `false` - Wired the field value into `network_config.allow_private_network` when saving the provider - Added a labeled toggle switch in the form UI with a description clarifying which address ranges are affected and which remain blocked ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to the custom provider creation sheet in the workspace providers UI. 2. Verify the "Allow Private Network" toggle is visible and defaults to off. 3. Enable the toggle and save the provider — confirm `allow_private_network: true` is included in the saved `network_config`. 4. Disable the toggle and save — confirm `allow_private_network: false` is sent. 5. Verify the toggle is disabled when the user lacks provider create access. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings _Add before/after screenshots of the custom provider form showing the new toggle._ ## Breaking changes - [ ] Yes - [x] No ## Related issues Link related issues and discussions. Example: Closes maximhq#123 ## Security considerations This toggle explicitly opts a custom provider into connecting to private network ranges. It defaults to `false` (blocked), preserving the existing secure-by-default behavior. Link-local addresses remain blocked unconditionally to prevent SSRF via metadata endpoints. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an "Allow Private Network" toggle option in the custom provider creation form, enabling users to control private network access settings when setting up custom providers. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…est provider/model routing (maximhq#4175) ## Summary Introduces a new `PreRequestHook` phase to the `LLMPlugin` interface. This hook runs exactly once per top-level request — after `HTTPTransportPreHook` and before `PreLLMHook` — and is the canonical place for plugins to resolve provider, model, and fallback routing decisions. Previously, routing logic had to be shoehorned into `PreLLMHook`, which runs on every fallback attempt and whose mutations have incidental cross-fallback visibility. `PreRequestHook` mutations are committed to the shared `*BifrostRequest` before any fan-out and are observed by every subsequent plugin, every `PreLLMHook` invocation, the provider call, and every fallback. As part of this change, the `filterProvidersByContext` helper (used in `ListAllModels`) is removed, and request validation is moved to after `PreRequestHook` runs so that plugins have the opportunity to populate provider/model before the empty-field check fires. Error messages for missing provider/model are updated to reflect that auto-resolution was attempted. ## Changes - Added `PreRequestHook(ctx *BifrostContext, req *BifrostRequest) error` to the `LLMPlugin` interface with non-blocking error semantics (logged as warning, pipeline continues). - Added `RunPreRequestHooks` to `PluginPipeline`, executing the hook in registration order once per request with tracing and plugin-scope isolation. - Added `RunPreRequestHooks` as a public method on `Bifrost` for callers (e.g., realtime WebSocket handlers) that bypass the normal inference path. - Moved `validateRequest` to after `PreRequestHook` execution in both `handleRequest` and `handleStreamRequest`, renamed to `validateRequestAfterPreRequestHooks` with updated error messages. - Added primary-provider error logging to `handleStreamRequest` to match `handleRequest` behavior. - Removed `filterProvidersByContext` and its tests from `ListAllModels`. - Updated `DynamicPlugin` (shared-object loader) to optionally load `PreRequestHook` from `.so` plugins; legacy plugins without the export get a no-op passthrough, preserving backward compatibility. - Updated `AsLLMPlugin` to recognize `preRequestHook` as sufficient to qualify a `DynamicPlugin` as an `LLMPlugin`. - Added no-op `PreRequestHook` implementations to all existing plugins (`compat`, `governance`, `jsonparser`, `logging`, `maxim`, `mocker`, `prompts`, `semanticcache`, `telemetry`) and all example/test plugins to satisfy the updated interface. - Updated plugin execution-order documentation in `plugin.go` to describe per-request vs. per-attempt semantics. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Validate that: - A plugin implementing `PreRequestHook` can mutate `req.Provider` and `req.Model` before the provider call, and those mutations are visible to subsequent plugins and fallback attempts. - A plugin that returns a non-nil error from `PreRequestHook` does not abort the request; the pipeline continues to the next plugin and a warning is logged. - Existing plugins with no-op `PreRequestHook` implementations behave identically to before. - Legacy `.so` plugins that do not export `PreRequestHook` load and function correctly with the no-op passthrough. - Requests with no provider set (and no plugin resolving one) return the updated error message: `"could not auto resolve a provider for the request, please specify a provider explicitly"`. ## Breaking changes - [x] Yes - [ ] No The `LLMPlugin` interface gains a new required method `PreRequestHook`. Any external plugin implementing `LLMPlugin` must add a `PreRequestHook` method. Plugins that do not participate in routing should return `nil`. Shared-object (`.so`) plugins are exempt — the loader treats `PreRequestHook` as optional and provides a no-op default. ## Security considerations `PreRequestHook` runs with `BlockRestrictedWrites` active on the context (same as `RunLLMPreHooks`), preventing plugins from writing to restricted context keys during the hook. Plugins cannot abort or gate requests via error return from this hook; authorization and content-policy enforcement must remain in `HTTPTransportPreHook` or via a short-circuit in `PreLLMHook`. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added `PreRequestHook` plugin phase enabling plugins to perform per-request routing decisions before provider and model validation. * **Refactor** * Request validation now occurs after plugin hooks execute, allowing automatic resolution of routing parameters. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…PreRequestHook` with `BifrostRequest`-native mutations (maximhq#4176) ## Summary Governance routing logic has been migrated from `HTTPTransportPreHook` to `PreRequestHook`, operating directly on `BifrostRequest` structs rather than raw HTTP bodies. This eliminates the need to unmarshal/marshal JSON, parse multipart forms, or extract models from URL path parameters inside the governance plugin. Integration-specific normalization (Gemini, Bedrock, etc.) now happens upstream before the request reaches governance, so the plugin sees a clean `provider/model` pair regardless of the originating integration. ## Changes - `HTTPTransportPreHook` is now a no-op stub retained only to satisfy the `HTTPTransportPlugin` interface; all routing flows through `PreRequestHook` - `loadBalanceProvider` and `applyRoutingRules` now accept `*schemas.BifrostRequest` instead of `map[string]any` + `*schemas.HTTPRequest`, mutating `Provider`/`Model`/`Fallbacks` directly via typed setters - `addMCPIncludeTools` (header mutation) replaced by `computeMCPIncludeTools` (returns `[]string`); result is stored on context via `MCPContextKeyIncludeTools` rather than written into HTTP headers - `validateRequiredHeaders` and `stampGovernanceCtxFromVK` moved to `utils.go`; `stampGovernanceCtxFromVK` also fixes a bug where `Team.CustomerID`/`Team.Customer` were not propagated when the VK had a team association - `governLargePayload` and `governRealtimeQueryParam` removed; large-payload routing now runs through `runPreRequestRouting` (a thin wrapper that builds a synthetic `BifrostRequest` and calls the same helpers), and realtime WebSocket upgrades are routed via an explicit `RunPreRequestHooks` call in `wsrealtime.go` before the upgrade completes - `BifrostContextKeyRequestQuery` context key added; query params (lowercased) are now populated in `ConvertToBifrostContext` for normal HTTP requests and explicitly in the WS upgrade path, making them available to governance CEL routing rules via `params["..."]` - Fallbacks produced by `loadBalanceProvider` are now `[]schemas.Fallback` (typed) instead of `[]string` - Tests that exercised the old `HTTPTransportPreHook` body-parsing path are skipped with a note to rewrite them as `PreRequestHook` tests in Phase 3 ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./plugins/governance/... go test ./transports/bifrost-http/... ``` Verify that: - Governance routing rules (CEL expressions reading `headers[...]` and `params[...]`) resolve correctly for both normal HTTP and WebSocket realtime upgrade requests - Virtual key load balancing selects a provider and populates typed fallbacks on the `BifrostRequest` - Large-payload streaming requests route via `LargePayloadMetadata.Model` and the rewritten model is visible to the upstream provider's body rewriter - Realtime WebSocket upgrades pick up the governance-routed `provider`/`model` before the connection is established ## Breaking changes - [x] Yes - [ ] No `applyRoutingRules` and `loadBalanceProvider` signatures have changed from `(ctx, *HTTPRequest, map[string]any, *TableVirtualKey)` to `(ctx, *BifrostRequest, *TableVirtualKey)`. Any code calling these methods directly (outside the governance plugin itself) must be updated. The `addMCPIncludeTools` method has been removed; callers should use `computeMCPIncludeTools` and store the result on context. ## Related issues Closes maximhq#2516 ## Security considerations No new auth surfaces introduced. Query params are now stored on context with lowercased keys, consistent with how request headers are already handled. No secrets or PII are added to context beyond what was already present. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Governance routing and load balancing now support WebSocket realtime connections. * Request query parameters are now accessible for governance routing rules. * **Improvements** * Enhanced governance routing by moving rule evaluation to an earlier request processing phase for improved handling of large payloads. * Added required header validation in governance rules. * **Tests** * Migrated governance pre-hook tests; rewrite pending in Phase 3. * **Chores** * Updated indirect dependency version. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…RequestHook plugin (maximhq#4177) ## Summary Provider resolution for unprefixed model strings (e.g. `gpt-4o` without a `provider/` prefix) was previously scattered across every integration router, every provider's `ToBifrost*` converter, and the `CheckAndSetDefaultProvider` utility — each doing its own inline catalog lookup or context-key dance. This PR consolidates all of that into a single, dedicated `modelcatalogresolver` built-in plugin that runs as the last `PreRequestHook` before the request reaches the LLM layer. ## Changes - **New `plugins/modelcatalogresolver` plugin**: A `PreRequestHook` that fills `req.Provider` from the model catalog when no provider was specified and no earlier routing plugin (governance routing rules, governance VK load balancing, enterprise LB) already set one. It also promotes remaining catalog candidates to fallbacks automatically when the caller didn't configure any. An integration-type hint (`BifrostContextKeyIntegrationType`) biases the pick toward the integration's canonical provider; Azure user-agent detection is handled here instead of inline in the OpenAI converters. - **Removed `CheckAndSetDefaultProvider`**: The `providerUtils.CheckAndSetDefaultProvider` helper and its `BifrostContextKeyAvailableProviders` / `BifrostContextKeyResolvedProvider` context keys are deleted. All `ToBifrost*` converters across Anthropic, Bedrock, Cohere, Gemini, OpenAI, and Vertex now pass `""` as the default provider to `ParseModelString`, deferring resolution entirely to the plugin layer. - **Removed `GetRequestModel` / `GetProvidersForModel` from integration routers**: The per-route `RequestModelGetter` callbacks and the inline catalog-lookup block in `GenericRouter.createHandler` are removed. The `HandlerStore` interface no longer requires `GetProvidersForModel`, and `Config.GetProvidersForModel` is deleted. The governance plugin no longer sets `BifrostContextKeyAvailableProviders`. - **Simplified `resolveModelAndProvider` and `resolveRealtimeTarget`**: These functions in the inference and WebSocket realtime handlers no longer do inline catalog lookups. The realtime handlers (`webrtc_realtime.go`, `realtime_client_secrets.go`) that couldn't go through `PreRequestHook` now call the exported `modelcatalogresolver.ResolveProviderFromCatalog` directly. - **WebSocket realtime empty-provider guard**: `wsrealtime.go`'s `handleUpgrade` now returns a clear WebSocket error when no routing layer could resolve a provider, mirroring the empty-provider validation in `handleRequest`/`handleStreamRequest`. - **Observability consolidation**: `EmitModelCatalogRoutingLog` is extracted into `lib/ctx.go` so all paths (normal HTTP, WebRTC, realtime client secrets) emit routing engine logs in the same shape. The `snapshotRealtimeMiddlewareValues` function no longer duplicates this logic. - **Governance plugin cleanup**: Removed the `BifrostContextKeyAvailableProviders` writes and the fallback-filtering logic from `extractAndParseFallbacks` that depended on them. - **`BifrostContextKeySkipModelCatalogProviderSelection` removed**: The context key and all references are deleted; the resolver plugin's position as the last hook makes the skip flag unnecessary. ## Type of change - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/... go test ./transports/... go test ./plugins/governance/... go test ./plugins/modelcatalogresolver/... ``` Verify that requests with unprefixed model strings (e.g. `{"model":"gpt-4o"}` on `/v1/chat/completions`) still resolve to the correct provider when a model catalog is configured. Verify that requests with explicit prefixes (e.g. `{"model":"anthropic/claude-opus-4"}`) are unaffected. Verify that WebSocket realtime connections with unresolvable models receive a `400 invalid_request_error` frame instead of hanging. ## Breaking changes - [x] Yes - [ ] No `HandlerStore.GetProvidersForModel` is removed from the interface — any custom `HandlerStore` implementations must drop this method. `BifrostContextKeyAvailableProviders`, `BifrostContextKeyResolvedProvider`, and `BifrostContextKeySkipModelCatalogProviderSelection` context keys are removed from `schemas`; any code reading or writing these keys must be updated. `CheckAndSetDefaultProvider` is removed from `core/providers/utils`. ## Related issues ## Security considerations No auth, secrets, or PII changes. Provider resolution is now centralised in a single plugin rather than distributed across converters, reducing the surface area for routing bypasses. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Introduced model catalog resolver for intelligent provider selection based on model availability and integration type. * Added integration-aware provider preferences (e.g., Azure-OpenAI pairing). * **Bug Fixes** * Improved model-to-provider resolution to eliminate context-dependent provider defaults. * **Refactor** * Streamlined provider resolution logic across all AI provider integrations. * Simplified request routing configuration by removing model getter dependencies. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…encing docs (maximhq#4178) ## Summary Documents the `PreRequestHook` interface introduced in v1.6.x — a new per-request routing phase that fires exactly once before any provider call, distinct from `PreLLMHook` which runs per provider attempt. This clarifies where routing decisions (provider, model, fallbacks) should be made and how they propagate through the fallback chain. ## Changes - Added `PreRequestHook` to the plugin lifecycle state diagram, sequence diagrams, and execution order descriptions across the architecture and getting-started docs, making clear it runs once per request while `PreLLMHook`/`PostLLMHook` run per attempt - Added a routing layer order table in `sequencing.mdx` documenting the built-in plugin execution order within `PreRequestHook`: governance (order 4) → enterprise load balancer → model-catalog-resolver (order 9, final fallback) - Added a full `PreRequestHook` reference section in `writing-go-plugin.mdx` with a comparison table against `PreLLMHook`, a routing example using `SetProvider`/`SetModel`, and notes on the two routing observability helpers (`AppendRoutingEngineLog`, `AppendToContextList`) - Updated `provider-routing.mdx` to reflect that all three routing layers (governance, enterprise LB Level 1, model-catalog-resolver) now execute inside the `PreRequestHook` phase rather than across separate middleware stages, and updated the flowcharts and execution order lists accordingly - Clarified that `model-catalog-resolver` now prefers the integration's canonical provider (OpenAI/Anthropic/GenAI/Bedrock/Cohere) when the request arrived via an integration route, rather than always selecting the first catalog candidate - Updated the log message example from `selecting first:` to `selected:` to match the new resolver behavior ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered docs for accuracy against the v1.6.x plugin interface. Verify that: - The `PreRequestHook` signature (`func PreRequestHook(ctx *schemas.BifrostContext, req *schemas.BifrostRequest) error`) matches the SDK - The execution order table in `sequencing.mdx` matches the registered plugin orders in the codebase - The flowcharts in `provider-routing.mdx` correctly reflect that all Level 1 routing now happens in `PreRequestHook` ## Breaking changes - [ ] Yes - [x] No ## Related issues Documents the `PreRequestHook` routing phase introduced alongside the model-catalog-resolver and routing engine changes in v1.6.x. ## Security considerations None. Documentation-only change. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated plugin lifecycle documentation to clarify `PreRequestHook` execution (v1.6.x+) as a once-per-request routing phase separate from per-attempt hook phases. * Enhanced plugin sequencing and execution order documentation with clearer per-request vs. per-attempt semantics. * Expanded provider routing documentation with updated default resolution order and governance/load-balancing interaction details. * Added comprehensive `PreRequestHook` guidance for plugin developers with routing examples and helpers. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…AllowedProviders` (maximhq#4179) ## Summary Introduces a two-level routing allowlist enforcement mechanism that prevents any plugin or user-specified provider prefix from bypassing provider restrictions defined by a Virtual Key's `provider_configs`. Previously, governance could fail to pick a provider but a downstream routing layer could still select a non-permitted provider. This change closes that gap by publishing the VK's allowed-provider set to the request context and enforcing it both cooperatively (in the model catalog resolver) and as a hard guarantee (in core, after all `PreRequestHook` plugins have run). ## Changes - Adds `BifrostContextKeyRoutingAllowedProviders` context key (`[]ModelProvider`) that plugins can set to constrain which providers are valid for a request. An empty slice means "no provider is permitted" (fail-closed → HTTP 400). - Adds `enforceRoutingAllowlist` and `filterFallbacksByAllowlist` helpers in `core/bifrost.go`. After all pre-request hooks complete, `handleRequest` and `handleStreamRequest` validate the resolved provider against the allowlist and prune fallbacks to only allowed providers. A non-allowed primary provider returns HTTP 400. - The governance plugin now publishes the VK's `provider_configs` providers to `BifrostContextKeyRoutingAllowedProviders` during `PreRequestHook`, covering the case where governance cannot pick a provider itself but still needs to constrain downstream layers. - The model catalog resolver intersects its catalog candidates with the allowlist (when set) before selecting a provider, emitting routing-engine observability logs that explain which candidates were excluded and why. Returns `("", nil)` when the allowlist excludes all candidates. - Documents the two-level enforcement model, context key semantics, and custom plugin usage in `docs/providers/provider-routing.mdx`. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test ```sh go test ./... ``` - Configure a Virtual Key with `provider_configs` restricted to a specific provider (e.g., `openai`). - Send a request with an explicit `provider/model` prefix targeting a non-allowed provider (e.g., `anthropic/claude-3`). Expect HTTP 400 with a message indicating the provider is not permitted. - Send a request with a model resolvable by the catalog to multiple providers where some are excluded by the VK allowlist. Verify routing-engine logs show the excluded candidates and the request routes only to an allowed provider. - Set `BifrostContextKeyRoutingAllowedProviders` to an empty slice from a custom plugin and confirm the request fails closed with HTTP 400. - Verify fallbacks targeting non-allowed providers are silently pruned and do not appear in the fallback chain. ## Breaking changes - [x] Yes - [ ] No Requests that previously succeeded by specifying an explicit `provider/model` prefix that bypassed a VK's `provider_configs` restrictions will now be rejected with HTTP 400. Any fallbacks targeting providers outside the VK's allowed set will be silently removed from the fallback chain. ## Security considerations This change strengthens provider-level access control enforced by Virtual Keys. Without this, a user or plugin could bypass governance-imposed provider restrictions by specifying an explicit provider prefix or by relying on a downstream routing layer to select a non-permitted provider. The hard enforcement in core ensures the allowlist is a guarantee rather than a best-effort constraint. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## New Features * Added provider routing allowlist enforcement: configured allowed providers now restrict request routing across primary and fallback options; requests using non-permitted primary providers return HTTP 400, and fallback providers are automatically filtered to only permitted options. ## Documentation * Added routing allowlist enforcement documentation covering enforcement mechanisms and fail-closed behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…ig` with legacy wire-shape compatibility (maximhq#4180) ## Summary `KeyAliases` previously mapped user-facing model names to plain strings (`map[string]string`). This PR promotes the value type to a rich `AliasConfig` struct, enabling per-alias metadata (`ModelName`, `ModelFamily`) and provider-specific overrides (`AzureAliasCfg`, `VertexAliasCfg`, `BedrockAliasCfg`, `ReplicateAliasCfg`) to be expressed directly on an alias entry rather than inferred from the wire model ID or duplicated at the key level. ## Changes - `KeyAliases` is now `map[string]AliasConfig` instead of `map[string]string`. `AliasConfig` carries `ModelID` (the wire identifier), optional `ModelName`, `ModelFamily` (a typed enum for routing decisions), `Description`, `Region`, and embedded provider sub-configs for Azure, Vertex, Bedrock, and Replicate. - `AliasConfig.MarshalJSON` emits the legacy `{"k":"v"}` string-valued wire shape when only `ModelID` is set, preserving byte-for-byte JSON compatibility with pre-refactor consumers and keeping `config_hash` stable for unenriched entries. - `KeyAliases.UnmarshalJSON` transparently accepts both the legacy string shape and the new object shape, promoting legacy string values to `AliasConfig{ModelID: <string>}`. - `KeyAliases.Resolve` is preserved for backward compatibility. A new `ResolveConfig` method returns the full `AliasConfig` for callers that need more than the wire model string. - `KeyAliases.Validate` is extended to check `ModelName` whitespace and `ModelFamily` validity. - `ModelFamily` is introduced as a typed enum (`anthropic`, `openai`, `mistral`, `cohere`, `gemini`, `nova`, `titan`) with an `IsValid` method, enabling provider routing decisions without substring-sniffing the wire model ID. - All provider `ToBifrostListModelsResponse` and `ListModelsPipeline` call sites are updated to use `schemas.KeyAliases` and access `.ModelID` from `AliasConfig` values. - The JSON schema (`config.schema.json`) is updated so the `aliases` property accepts either the legacy string shape or the new object shape via `oneOf`. - New tests cover legacy/rich/mixed unmarshal, round-trip marshal stability, `Resolve`/`ResolveConfig` behavior, `Validate` error cases, `ModelFamily.IsValid`, DB persistence of both wire shapes, and hash stability guarantees. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/schemas/... go test ./core/providers/... go test ./framework/configstore/... go test ./transports/bifrost-http/lib/... ``` Existing configs using the legacy `"alias": "model-id"` string shape require no changes — they deserialize and re-serialize identically. To opt into the rich shape, update an alias entry to the object form: ```json "aliases": { "my-model": { "model_id": "azure-deployment-xyz", "model_family": "anthropic", "model_name": "claude-3-5-sonnet", "api_version": "2024-08-01-preview" } } ``` ## Breaking changes - [x] Yes - [ ] No Any code that directly reads `KeyAliases` values as `string` (e.g. `aliases["key"]` expecting a `string`) must be updated to access `.ModelID` on the returned `AliasConfig`. The JSON wire format for unenriched aliases is unchanged. The `Resolve(model string) string` method signature is unchanged. ## Related issues ## Security considerations No new secrets or auth surfaces are introduced. Provider sub-config fields that accept `EnvVar` values follow the existing env-var resolution and encryption patterns already in place for key-level configs. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…n overrides with context-aware `ResolveFamily` (maximhq#4181) ## Summary This PR introduces per-alias Azure overrides for endpoint, API version, and Anthropic version, and adds a `ResolvedAlias` context value that providers can read to determine model family routing. Previously, model family detection relied solely on substring matching against the wire model ID, which broke when Azure deployment IDs were opaque strings (e.g., `12345-azure-deployment`) even if the alias key or config clearly indicated the model family (e.g., `best-claude`). This change fixes that by walking a precedence chain — explicit `ModelFamily` field → `ModelName` → `ModelID` → alias key — before falling back to substring matching. ## Changes - Introduced `ResolvedAlias` struct and `BifrostContextKeyResolvedAlias` context key; `bifrost.go` now stashes the full `AliasConfig` (and the user-facing alias key) into context after each key-level alias resolution, for both streaming and non-streaming paths. - Added `ResolveFamily`, `IsAnthropicModelFamily`, and `GetResolvedAlias` helpers in `schemas/account.go` that walk the alias precedence chain for model family detection. All Azure provider call sites that previously called `schemas.IsAnthropicModel(model)` now call `schemas.IsAnthropicModelFamily(ctx, model)` or `schemas.ResolveFamily(ctx, model)`. - Added `AzureAliasCfg` fields (`APIVersion`, `AnthropicVersion`, `Endpoint`) and three resolver helpers (`resolveAzureEndpoint`, `resolveAPIVersion`, `resolveAnthropicVersion`) in `core/providers/azure/utils.go`. All Azure provider methods now use these helpers instead of reading `key.AzureKeyConfig.Endpoint.GetValue()` directly, enabling per-alias endpoint and version overrides. - `buildPassthroughURL` now returns an error when the endpoint is empty and accepts a `*BifrostContext` to apply alias-level `api-version` overrides on passthrough routes. - `buildContainerURL` now accepts a `*BifrostContext` for the same reason. - `KeyAliases.UnmarshalJSON` switched from `encoding/json` to `sonic` for consistency with the rest of the codebase. - Added `KeyAliases.ResolveConfig` (returns `*AliasConfig`) used by `bifrost.go` to populate `ResolvedAlias`; the existing `Resolve` (returns string) is preserved for backward compatibility. ## Type of change - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/... ./core/providers/azure/... ./core/schemas/... ``` Key scenarios to validate: 1. **Opaque Azure deployment alias** — configure an alias like `best-claude → { model_id: "12345-deployment", azure_alias_cfg: {} }` with alias key containing "claude"; verify the request is routed through the Anthropic path (correct URL, `anthropic-version` header set). 2. **Alias-level endpoint override** — configure two aliases pointing to different Azure cognitive-services resources under the same key; verify each request hits the correct endpoint. 3. **Alias-level `api_version` override** — configure an alias with `azure_alias_cfg.api_version: "2024-10-21"`; verify the `api-version` query parameter on `/openai/deployments/` and `/openai/v1/responses` routes uses the override rather than the route default. 4. **Caller-supplied `api-version` wins** — pass `api-version` explicitly in a passthrough `rawQuery`; verify the alias override does not overwrite it. 5. **No alias matched** — verify existing substring-based family detection is unchanged. ## Breaking changes - [ ] Yes - [x] No `buildPassthroughURL` now returns `(string, error)` instead of `string`. This is an internal method on `AzureProvider` and is not part of any exported interface. ## Related issues ## Security considerations The `ResolvedAlias` value stored in `BifrostContext` is set exclusively by the core request worker and is documented as read-only for plugins. Alias-level endpoint overrides are resolved from `EnvVar` (supporting environment variable indirection), consistent with how key-level endpoints are handled, so secrets are not inlined in config. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…mily resolution for Bedrock (maximhq#4182) ## Summary Model-family detection on the Bedrock provider previously relied on substring matching against the raw model string. This meant that aliases pointing to opaque Bedrock deployments (e.g., inference profiles or cross-region ARNs) could not be correctly routed to the right request/response shape. This PR threads `*BifrostContext` through all family-detection call sites so that the resolved alias — including its explicit `ModelFamily`, `Region`, and `BedrockAliasCfg.InferenceProfileARN` — takes precedence over substring heuristics. ## Changes - Replaced all calls to `schemas.IsAnthropicModel`, `IsMistralModel`, `IsNovaModel`, `IsLlamaModel`, `IsCohereModel` (bare string matchers) with new context-aware variants: `IsAnthropicModelFamily`, `IsMistralModelFamily`, `IsNovaModelFamily`, `IsLlamaModelFamily`, `IsCohereModelFamily`, `IsTitanModelFamily`. These consult `ResolveFamily(ctx, model)` first, which reads the alias family tag before falling back to substring detection. - Added `ModelFamilyLlama`, `ModelFamilyTitan`, and `ModelFamilyCohere` to the `ModelFamily` enum and wired them into `ResolveFamily` so alias-tagged models route correctly. - Added `IsCohereModel` and `IsTitanModel` substring helpers used as the final fallback inside `ResolveFamily`. - Introduced `resolveBedrockARN(ctx, key)` to resolve the inference-profile ARN with priority: alias-level `BedrockAliasCfg.InferenceProfileARN` > key-level `BedrockKeyConfig.ARN`. Removed the inline ARN lookup from `getModelPathAndRegion`. - Updated `resolveBedrockRegion` and `getModelPathAndRegion` to accept `*BifrostContext` and honor the alias-level `Region` override between the model-string prefix (highest) and the key-level region (lower). - Propagated `*BifrostContext` into `DetermineEmbeddingModelType` and `ToBedrockEmbeddingInvokeResponse` so embedding model routing uses the same family resolution path. - Changed `convertToolConfigFromFiltered` to accept `*schemas.BifrostContext` instead of `context.Context` so family gates inside it can read the resolved alias. - Added nil-guard checks on `BifrostContext` in `bedrockAliasToolName` and `bedrockRestoreToolName`. - Added tests covering alias-level `Region` and `InferenceProfileARN` override priority in `region_test.go`. ## Type of change - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/bedrock/... ./core/schemas/... ``` To validate alias-level overrides end-to-end, configure an alias with an explicit `ModelFamily`, `Region`, or `BedrockAliasCfg.InferenceProfileARN` and confirm that: - Requests are signed and routed to the alias-specified region rather than the key-level region. - The ARN is prepended to the model path when `InferenceProfileARN` is set on the alias. - Embedding requests to an alias tagged `cohere` or `titan` use the correct request/response envelope regardless of the wire model string. ## Breaking changes - [x] Yes - [ ] No `ToBedrockEmbeddingInvokeResponse` now requires a `*schemas.BifrostContext` as its first argument. Any external callers of this function must be updated to pass the context. The `convertToolConfigFromFiltered` signature changed from `context.Context` to `*schemas.BifrostContext`. ## Related issues ## Security considerations No new secrets or auth surfaces introduced. ARN and region values continue to flow through the existing `EnvVar`/`GetValue()` resolution path. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… model family resolution (maximhq#4183) ## Summary Introduces per-alias overrides for Vertex AI's `project_id`, `project_number`, and `region` configuration values, allowing a single Vertex credential to serve requests across multiple GCP projects and regions. This is particularly useful when different model families (e.g., Anthropic Claude on `us-east5`, Gemini on `us-central1`) are deployed in separate GCP projects or regions. ## Changes - Added `resolveVertexProjectID`, `resolveVertexProjectNumber`, and `resolveVertexRegion` helper functions in `utils.go` that check for alias-level overrides in `BifrostContext` before falling back to key-level configuration values. - Replaced all direct `key.VertexKeyConfig.*` field accesses throughout the Vertex provider with calls to these resolver functions, covering all operations: chat completion, streaming, embeddings, responses, cached content, image/video generation, reranking, token counting, and passthrough. - Replaced model-family detection calls (`IsAnthropicModel`, `IsGeminiModel`, `IsGemmaModel`, `IsImagenModel`, `IsVeoModel`, `IsMistralModel`) with context-aware variants (`IsAnthropicModelFamily`, `IsGeminiModelFamily`, etc.) so that alias-level `ModelFamily` overrides are respected when routing requests to the correct Vertex endpoint and request format. - Added `ModelFamilyGemma`, `ModelFamilyImagen`, and `ModelFamilyVeo` as first-class `ModelFamily` constants and registered them in `ResolveFamily`, with Imagen and Veo checked before Gemini to avoid substring-match conflicts. - Added context-aware `IsGeminiModelFamily`, `IsGemmaModelFamily`, `IsImagenModelFamily`, and `IsVeoModelFamily` helpers to `account.go`. - Added unit tests covering alias override precedence, empty-alias fallthrough to key-level values, and nil-context handling for all three resolver functions. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/vertex/... -run TestResolveVertex go test ./core/... ``` Configure a Vertex key alias with `VertexAliasCfg.ProjectID` or `AliasConfig.Region` set to a value different from the key-level config and verify that requests are routed to the alias-specified project/region rather than the key-level defaults. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations No new secrets or auth mechanisms are introduced. Alias-level project/region values follow the same `EnvVar` resolution path as key-level values, so secrets can still be sourced from environment variables rather than being hardcoded. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Extended support for additional Google Vertex AI model families. * Added flexible configuration resolution with alias-level parameter overrides for the Vertex provider. * **Tests** * Added comprehensive tests validating configuration override behavior for Vertex parameters. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…de with provider-scoped alias sub-config validation (maximhq#4184) ## Summary Adds per-alias control over whether Replicate requests are routed to the deployments endpoint or the predictions endpoint. Previously this was only configurable at the key level via `ReplicateKeyConfig.UseDeploymentsEndpoint`. This change introduces a `ReplicateAliasCfg.UseDeploymentsEndpoint` field that, when set, takes precedence over the key-level setting — allowing a single Replicate API token to route some aliases through the deployments endpoint (e.g. production-pinned models) while others use the predictions endpoint (e.g. experimental versioned models). Additionally, `KeyAliases.Validate` now accepts the owning key's provider and rejects provider-specific sub-configs (Azure, Vertex, Bedrock, Replicate) when attached to a key belonging to a different provider. ## Changes - `useDeploymentsEndpoint` now accepts a `*schemas.BifrostContext` and checks for a `ReplicateAliasCfg.UseDeploymentsEndpoint` override on the resolved alias before falling back to the key-level config. All call sites updated accordingly. - `KeyAliases.Validate` signature changed from `Validate()` to `Validate(providerKey ModelProvider)`. It now returns an error if a provider-specific alias sub-config (e.g. `AzureAliasCfg`) is attached to a key that does not belong to that provider. - Alias validation in `processProvider`, `processAuthoritativeProvider`, and the HTTP handler create/update paths now resolves the effective base provider (accounting for custom provider configs) before calling `Validate`. - The redundant `Validate` call in the GORM `BeforeSave` hook was removed since validation is enforced at the handler layer. - Tests added for `useDeploymentsEndpoint` covering nil context, key-level fallback, alias override true/false, and alias present but without a `ReplicateAliasCfg`. Existing `TestKeyAliasesValidate` extended with provider-mismatch cases for all four provider sub-configs. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/replicate/... go test ./core/schemas/... go test ./transports/bifrost-http/... ``` To exercise the alias override end-to-end, configure a Replicate key with `use_deployments_endpoint: false` at the key level, then define an alias with `replicate_alias_cfg.use_deployments_endpoint: true`. Requests routed through that alias should target the deployments endpoint while requests without the alias continue to use the predictions endpoint. To verify provider-mismatch validation, attach a `replicate_alias_cfg` to an Azure key in the config file or via the HTTP API and confirm a `400 Bad Request` is returned with a descriptive error. ## Breaking changes - [x] Yes `KeyAliases.Validate()` now requires a `ModelProvider` argument. Any code calling `Validate()` directly must be updated to pass the owning key's provider. ## Security considerations None beyond standard input validation. The new provider-mismatch check prevents misconfigured aliases from silently routing requests to unintended endpoints. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ing per-deployment model family, canonical name, and provider overrides (maximhq#4185) ## Summary Replaces the flat "Aliases" key-value table (mapping request model name → string deployment ID) with a richer "Deployments" table that supports per-deployment metadata and provider-specific overrides. This enables cost/pricing logs and family-based routing to work correctly for custom deployments. ## Changes - Introduced a new `DeploymentsTable` component that renders each deployment as a collapsible row. Expanding a row exposes fields for canonical model name, model family, description, and provider-specific overrides (Azure API version, endpoint, Anthropic version; Vertex project ID/number/region; Bedrock region and inference profile ARN; Replicate deployments endpoint toggle). - Replaced the `normalizeAliasesValue` helper and `HeadersTable`-based aliases editor in `apiKeysFormFragment.tsx` with the new `DeploymentsTable`. The form label and description were updated from "Aliases" to "Deployments" to reflect the richer semantics. - Added `AliasConfig` and `ModelFamily` types to `config.ts`, mirroring the Go `schemas.AliasConfig` struct (with embedded provider sub-configs flattened to top-level fields on the wire). - Added `aliasConfigSchema` and `modelFamilySchema` Zod schemas to `schemas.ts`. The alias schema uses `z.preprocess` to accept the legacy `string` wire shape emitted by the Go server for simple aliases, coercing it to `{ model_id: string }` so hydrated state passes validation without a migration. - Updated `KeySchema` in `providerForm.ts` to use `z.record(z.string(), aliasConfigSchema)` and updated the validation error message to reflect the new requirement. - Rewrote `isValidAliases` in `validation.ts` to validate the rich `Record<string, { model_id?: string }>` shape, checking that every entry has a non-empty deployment name and a non-empty `model_id`. - Updated `ModelProviderKey` in `config.ts` to type `aliases` as `Record<string, AliasConfig>` instead of `Record<string, string>`. - The `DeploymentsTable` includes a draft row at the bottom for adding new entries. The draft is committed automatically when both the deployment name and model ID are filled. Rename collision detection is case-insensitive and stable row IDs are used to preserve expanded/pending state across renames. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i pnpm build ``` 1. Navigate to a provider key configuration form. 2. Verify the "Deployments" table renders in place of the old "Aliases" table. 3. Add a new deployment by filling in the deployment name and model ID in the draft row — confirm it commits automatically when both fields are populated. 4. Expand a committed row and verify the canonical model name, model family, description, and provider-specific override fields are visible and editable. 5. For Azure, Vertex, Bedrock, and Replicate providers, confirm the correct provider-specific section appears in the expanded panel. 6. Rename a deployment to an existing name and confirm the inline collision error appears and the row is not committed. 7. Load an existing config that uses the legacy `Record<string, string>` alias format and confirm it hydrates correctly into the new table without validation errors. ## Screenshots/Recordings Before: A simple two-column key/value table labeled "Aliases" with a plain text input for the deployment ID. After: A collapsible table labeled "Deployments" where each row can be expanded to reveal canonical model name, model family, description, and provider-specific override fields. ## Breaking changes - [x] Yes - [ ] No The `aliases` field type changes from `Record<string, string>` to `Record<string, AliasConfig>` in the UI type system and form schema. Existing configs using the legacy string format are handled transparently via the `aliasConfigSchema` preprocessor and the `normalize` function in `DeploymentsTable`, so no manual migration is required for stored configs. Any code outside this diff that directly constructs or reads `ModelProviderKey.aliases` as `Record<string, string>` will need to be updated. ## Related issues ## Security considerations No new secrets or auth surfaces introduced. Provider-specific override fields (endpoint, credentials) use the existing `EnvVarInput` component, which supports environment variable references and redaction consistent with the rest of the form. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…and alias resolution context (maximhq#4186) ## Summary Introduces a structured `RoutingInfo` field on both `BifrostResponseExtraFields` and `BifrostErrorExtraFields` that exposes per-attempt routing context (provider, model, key used, resolved alias, and fallback signals) in a single, well-typed struct. The existing `Provider`, `OriginalModelRequested`, and `ResolvedModelUsed` fields are preserved for backward compatibility but deprecated in favour of `RoutingInfo`. ## Changes - Added `RoutingInfo` and `ResolvedKeyAlias` schema types capturing the provider, model, key name, resolved alias metadata, fallback flag, and primary provider/model when a fallback occurred. - Added `BuildRoutingInfo` helper in `account.go` that constructs a per-attempt `RoutingInfo` from the current context, provider, model, and key, including any resolved alias config. - Added `PopulateRoutingInfo` on `BifrostResponse` and `BifrostError` to stamp `RoutingInfo` onto responses/errors and keep the deprecated triplet in sync via the shared `syncDeprecatedFromRoutingInfo` helper. - Added `SetFallbackRoutingInfo` on both types, called by the orchestrator (`handleRequest` / `handleStreamRequest`) to layer on `IsFallback`, `PrimaryProvider`, and `PrimaryModel` after a fallback attempt completes. These signals are intentionally set at the orchestrator scope rather than inside per-attempt code. - `requestWorker` now seeds `attemptRoutingInfo` with the known provider/model before the retry loop so that early failures (e.g. key selection errors) still produce a populated `RoutingInfo` on the error. Each retry iteration snapshots a `perAttemptRoutingInfo` to avoid races in async streaming closures. - `PopulateRoutingInfo` is called alongside `PopulateExtraFields` both before and after `RunPostLLMHooks`, ensuring plugin modifications cannot corrupt routing metadata. - `ProcessedStreamResponse` gains a `RoutingInfo` field to carry routing context through the streaming pipeline. - The deprecated `Provider`, `OriginalModelRequested`, and `ResolvedModelUsed` fields are annotated with deprecation notices and derivation rules pointing consumers to the equivalent `RoutingInfo` paths. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go version go test ./... ``` Verify that responses and errors include a populated `routing_info` object in their JSON output. For fallback scenarios, confirm `is_fallback` is `true` and `primary_provider`/`primary_model` reflect the original attempt. Confirm that `provider`, `original_model_requested`, and `resolved_model_used` continue to be populated with the same values as before. ## Breaking changes - [ ] Yes - [x] No Existing fields are preserved. `RoutingInfo` is additive. ## Related issues ## Security considerations None. No auth, secrets, or PII are introduced. `RoutingInfo` surfaces key names already present in existing fields. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…dd backward-compat fallback for legacy `ExtraFields` (maximhq#4187) ## Summary Replaces the flat `(provider, originalModel, resolvedModel)` triplet passed into `resolvePricing` with a structured `schemas.RoutingInfo` value. This aligns pricing lookups with the routing context that `core.bifrost` already populates on every response, and introduces a well-defined lookup precedence: `AliasModelName → AliasModelID → ModelName`, with overrides keyed by the wire model identifier. ## Changes - `resolvePricing` now accepts a single `schemas.RoutingInfo` argument instead of three separate string parameters. The lookup iterates over `[AliasModelName, AliasModelID, ModelName]`, stopping at the first catalog hit, and applies overrides keyed by the wire model (`AliasModelID` when an alias matched, otherwise `ModelName`). - `calculateBaseCost` reads `RoutingInfo` directly from `ExtraFields`. A backward-compatibility fallback synthesises a `RoutingInfo` from the deprecated `Provider`/`OriginalModelRequested`/`ResolvedModelUsed` triplet only when `RoutingInfo` is fully unset (zero `Provider`, zero `Model`, nil `ResolvedKeyAlias`). Partial population is trusted as-is to prevent false-positive fallbacks. - `computeCacheEmbeddingCost` constructs a minimal `RoutingInfo` from the cache-debug fields, since no alias resolution context exists for cache-replayed requests. - Container pricing overrides build a synthetic `RoutingInfo` that pins both model fields to the container identifier, preserving per-container override addressability. - All call sites in tests are updated to pass `schemas.RoutingInfo` structs directly. - New backward-compat tests cover: legacy-fields-only (no alias), legacy-fields-only (with alias/resolved model), `RoutingInfo` winning over legacy fields when both are set, both empty returning zero cost, and partial `RoutingInfo` suppressing the fallback. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/... ``` Key scenarios to verify: - Pricing resolves correctly for a non-aliased request via `RoutingInfo.Model`. - Pricing resolves via `AliasModelID` when an alias was matched and the wire model differs from the caller-facing name. - Pricing resolves via `AliasModelName` when the admin tagged a canonical name on the alias. - Legacy callers with only `Provider`/`OriginalModelRequested`/`ResolvedModelUsed` populated still receive correct costs. - When both `RoutingInfo` and the deprecated triplet are set, `RoutingInfo` wins. - Partial `RoutingInfo` (e.g. `Model` set but `Provider` empty) does not trigger the legacy fallback. ## Breaking changes - [ ] Yes - [x] No The `resolvePricing` method is unexported. The public `CalculateCost` API is unchanged. The backward-compat fallback ensures existing callers writing only the deprecated `ExtraFields` triplet continue to receive correct cost calculations. ## Related issues ## Security considerations None. This change affects cost accounting logic only; no auth, secrets, or PII are involved. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…pricing lookup precedence (maximhq#4188) ## Summary Documents the rich object form for alias values, the new `routing_info` response block, and the pricing lookup precedence that uses canonical `model_name` to resolve opaque deployment IDs against the catalog. ## Changes - Expanded the alias schema documentation to cover the object form alongside the existing plain-string shorthand. The object form accepts `model_id`, `model_name`, `model_family`, `description`, `region`, and provider-specific overrides (`api_version`, `anthropic_version`, `endpoint` for Azure; `project_id`, `project_number` for Vertex; `inference_profile_arn` for Bedrock; `use_deployments_endpoint` for Replicate). - Added a validation rule documenting that provider-specific sub-config fields are rejected when the owning key belongs to a different provider. - Replaced the flat `extra_fields` response fields (`original_model_requested`, `resolved_model_used`, `provider`) with the new `routing_info` block. The old fields are noted as deprecated but still populated for backward compatibility. - Added a `routing_info` field reference table covering `provider`, `model`, `key`, `resolved_key_alias`, `is_fallback`, `primary_provider`, and `primary_model`. - Added a pricing lookup precedence section explaining the three-candidate resolution order (`model_name` → `model_id` → caller-sent model) and how it solves the opaque deployment ID problem for cost attribution. - Added a cross-reference note in the Azure provider config page pointing readers to the full alias object schema. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered docs for: - Correct table formatting for the alias object schema and provider-specific overrides - The `routing_info` JSON example rendering properly - The pricing lookup precedence section appearing between the wildcard patterns section and the request type filtering section - The deprecation `<Note>` rendering in the aliasing-models page ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…/block/alias views (maximhq#4189) ## Summary Introduces a new `keyconfig` package under `framework/modelcatalog` that provides a thread-safe, in-memory store for per-key configuration (allowed models, blacklisted models, and aliases) across all configured providers. This centralizes the aggregation logic previously scattered in the load balancer plugin, making it available as a pure transformation layer for routing-time queries. ## Changes - Added `framework/modelcatalog/keyconfig/store.go` implementing a `Store` type that: - Maintains an immutable `providerState` snapshot per provider, swapped atomically under a write lock so readers never observe torn state - Aggregates allowed models as the union of enabled keys' `Models` fields minus per-key blacklisted entries, collapsing to `["*"]` when any enabled key is unrestricted - Computes the provider-level blacklist as the intersection across enabled keys (a model is only provider-blocked when every enabled key blacklists it) - Builds a case-insensitive alias index keyed by lowercase alias name, with last-enabled-key-wins collision resolution and a debug log on collision - Treats keyless non-standard (custom) providers as unrestricted (`["*"]`) to support ambient/IAM auth flows - Drops providers from the store entirely when they have no routable keys, keeping the store focused on routing-time queries rather than full config inspection - Exposes `Replace` (full atomic resync), `SetProvider` (single-provider update), `RemoveProvider`, `EntriesFor`, `EntryFor`, `AllowedFor`, `BlacklistedFor`, `IsAllowed`, `ResolveAlias`, `Providers`, and `KeysAllowingModel` - Added `framework/modelcatalog/keyconfig/store_test.go` with comprehensive behavioral tests covering: wildcard and explicit allow lists, blacklist intersection, disabled keys, block-all keys, alias ownership and collision, case-insensitive blacklist and alias normalization, atomic snapshot correctness under concurrent reads, and defensive copy guarantees ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/keyconfig/... ``` The test suite covers all aggregation semantics, concurrency safety (atomic snapshot test with 200 Replace cycles and a concurrent reader), and edge cases including nil logger, keyless non-standard providers, and case-insensitive alias collision detection. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations The store holds API key IDs and model allow/block lists in memory. No secrets (key values) are stored — only key IDs and routing metadata. The alias index is keyed by lowercase model name; alias configs may contain region or deployment override fields that are treated as read-only by callers. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ntry support (maximhq#4190) ## Summary Introduces a thread-safe, in-memory cache (`live.Store`) for provider model catalog responses. The store holds per-`(provider, keyID, unfiltered)` entries and is intentionally passive — it never initiates network calls. Callers are responsible for fetching and pushing results via `Upsert`, keeping the cache decoupled from transport concerns. ## Changes - Added `framework/modelcatalog/live/store.go` with a `Store` type that caches `/v1/models` responses keyed by provider, key ID, and a filtered/unfiltered flag. - Filtered entries are pre-gated by the provider's `ListModelsPipeline` at write time; callers reading filtered entries must not reapply that gate to avoid dropping alias-backfill rows. - `ModelsForProvider` and `UnfilteredModelsForProvider` return the sorted, deduplicated union across all matching keys for a provider. - `Invalidate` drops both filtered and unfiltered entries for a single key (e.g., on credential rotation or key deletion). `InvalidateProvider` drops all entries for a provider (e.g., on provider deletion). - `Snapshot` returns a full defensive copy of the store for diagnostics. - Both `Upsert` and `Snapshot` copy slices to prevent external mutation of cached state. - Keyless providers (Vertex workload identity, Bedrock IAM, etc.) are supported via an empty `KeyID`. - Added `framework/modelcatalog/live/store_test.go` covering union across keys, filtered/unfiltered isolation, invalidation behavior, defensive copying, overwrite semantics, and keyless provider handling. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/live/... ``` All tests should pass. Key scenarios covered: - Filtered and unfiltered entries for the same key do not bleed into each other. - Union across multiple keys for the same provider is deduplicated and sorted. - `Invalidate` removes both filtered and unfiltered entries for the target key while leaving other keys intact. - `InvalidateProvider` removes all entries for the target provider while leaving other providers intact. - Mutating the input slice after `Upsert`, or mutating a `Snapshot`, does not affect store state. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations Cached model lists may include key IDs as cache discriminators. The store holds no credential values — only the key ID string used as a lookup discriminator. No PII or secrets are stored. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes maximhq#123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…erKeysByID` helper (maximhq#4193) ## Summary Adds a `KeyID` field to `BifrostListModelsRequest` that scopes a `ListModels` call to a single key matched by `Key.ID`. This allows callers such as the catalog composer to cache list-models output per-key for fine-grained invalidation without requiring an extra round-trip or having the provider aggregate results across every configured key. ## Changes - Added `KeyID *string` to `BifrostListModelsRequest` (tagged `json:"-"` so it is never forwarded to providers). When set, the request worker filters the already-validated key set down to the single matching key before dispatching. If no key matches, a `BifrostError` is returned immediately. - Added `filterKeysByID` helper that returns a new slice containing only keys whose `ID` equals the target, leaving the input slice unmodified. - Added `TestFilterKeysByID` covering: a successful match, a missing key, an empty target string, and input-slice immutability. Note: the lookup runs against the already-filtered set of supported keys (disabled or invalid keys are excluded before the match), so a `KeyID` pointing to a disabled key produces the same "no key found" error as a non-existent `KeyID`. Callers that need to distinguish these cases must inspect the raw account configuration directly. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go version go test ./... ``` The new `TestFilterKeysByID` test directly exercises the helper and the key-scoping logic. To validate end-to-end, issue a `ListModels` request with `KeyID` set to a valid key ID and confirm only that key's models are returned. Issue one with an unknown `KeyID` and confirm a "no key found" error is returned. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations `KeyID` is tagged `json:"-"` and is never serialised or forwarded to any external provider. No secrets or PII are introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…cache fanout via `OnKeyAdded/Updated/Deleted` (maximhq#4194) ## Summary Replaces the coarse provider-level model discovery (one aggregated live entry per provider) with per-key live cache entries, so adding, updating, or deleting a single key triggers at most 2 list-models calls for that key rather than 2×N calls across all keys. Removes the deprecated compatibility shims (`UpsertModelDataForProvider`, `UpsertUnfilteredModelDataForProvider`, `DeleteModelDataForProvider`) and replaces them with `UpsertLiveFromResponse`, `OnKeyAdded`, `OnKeyUpdated`, and `OnKeyDeleted`. ## Changes - **`modelcatalog/pool.go`**: Added `UpsertLiveFromResponse`, which extracts and deduplicates model IDs from a `BifrostListModelsResponse` before writing to the live cache. A nil response is a no-op, preventing accidental cache eviction. - **`modelcatalog/shims.go`**: Deleted. The three deprecated shim methods are gone; call sites now use the per-key API directly. - **`modelcatalog/pool_test.go`**: New test file covering `UpsertLiveFromResponse` (nil no-op, happy path), `extractModelIDs` (prefix stripping, gateway nested prefixes, foreign prefix filtering, nil input, deduplication), `InvalidateLive`, and `InvalidateLiveProvider`. - **`server/server.go`**: Replaced `populateModelPoolWithListModels` (one aggregated entry per provider) with `RefreshLiveModelsForProvider` (fans out per key in parallel) and `FetchAndStoreLiveForKey` (issues filtered + unfiltered list-models for a single key). `ReloadProvider` now reads keys from the in-memory store, calls `SetKeyConfigForProvider` + `InvalidateLiveProvider`, then delegates to `RefreshLiveModelsForProvider`. `ForceReloadPricing` and `ReloadPricingFromDBAndPopulateModelPool` no longer trigger a full model pool refresh — pricing reload is now pricing-only. `RemoveProvider` calls `InvalidateLiveProvider` + `RemoveKeyConfigForProvider` instead of the deleted shim. Added `OnKeyAdded`, `OnKeyUpdated`, `OnKeyDeleted` to `ServerCallbacks` and implemented them on `BifrostHTTPServer`. - **`handlers/provider_keys.go`**: Key create/update/delete handlers now call `modelsManager.OnKeyAdded/OnKeyUpdated/OnKeyDeleted` instead of `attemptModelDiscovery`. Keyless providers skip the add/update path. - **`handlers/providers.go`**: Extended `ModelsManager` interface with `OnKeyAdded`, `OnKeyUpdated`, `OnKeyDeleted`. - **`handlers/providers_test.go`** and **`governance/httptransportprehook_test.go`**: Updated to use `UpsertLiveFromResponse` and `NewTestCatalog(nil)` instead of the removed shims and bare struct literals. - **`governance/resolver_test.go`**: Removed two tests that depended on the shim API; equivalent coverage exists in the catalog-level tests. - **`modelcatalog/models.go`**: Removed stale comment referencing the pre-refactor file. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/... go test ./transports/bifrost-http/... go test ./plugins/governance/... go test ./... ``` After adding a provider key via `POST /api/providers/{provider}/keys`, verify that only the new key's models are fetched (2 list-models calls) rather than a full provider refresh. After deleting a key, confirm the deleted key's live entries are evicted while other keys' entries remain intact. ## Screenshots/Recordings N/A ## Breaking changes - [x] Yes - [ ] No `UpsertModelDataForProvider`, `UpsertUnfilteredModelDataForProvider`, and `DeleteModelDataForProvider` are removed from `ModelCatalog`. Any external code calling these methods must migrate to `UpsertLiveFromResponse` / `UpsertLive` / `InvalidateLiveProvider`. `ServerCallbacks` now requires `OnKeyAdded`, `OnKeyUpdated`, and `OnKeyDeleted` — implementors must add these three methods. ## Related issues N/A ## Security considerations No new auth surfaces. Key validation (`BifrostContextKeyValidateKeys`) is preserved in `FetchAndStoreLiveForKey`, maintaining the same key-validation behavior at boot, after key add, and after provider reload. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ansitions (maximhq#4195) ## Summary Adds a `core` routing engine to the per-request audit trail so that fallback transitions and retry transitions made by the Bifrost orchestrator itself are visible alongside decisions made by plugins like `governance`, `loadbalancing`, `routing-rule`, and `model-catalog`. Previously, the routing engine log trail went silent after a plugin selected the primary upstream — there was no record of why core advanced through the fallback chain or how many retries were attempted before succeeding or giving up. ## Changes - Introduces `RoutingEngineCore = "core"` as a named routing engine constant, emitted by `handleRequest`, `handleStreamRequest`, and `executeRequestWithRetries` at each fallback and retry transition. - `handleRequest` and `handleStreamRequest` now log: primary failure entering the fallback loop, each fallback attempt (with the triggering error), skipped fallbacks (missing provider config), successful fallback, short-circuit halts, and full fallback exhaustion. - `executeRequestWithRetries` uses a named return + `defer` to guarantee a terminal log entry on every return path (including early exits from key-selection failures). Each retry transition records whether the key was rotated or reused, and keyless providers omit the key segment entirely. - Adds `routingErrorSummary()` — a sanitized formatter that surfaces only the error type and HTTP status code, deliberately excluding the upstream provider message to prevent API keys, tokens, or user input from leaking into log storage or the UI. - `AppendToContextList` is tightened from `any` to `comparable` and gains a deduplication check, so `core` (and any other engine) appears at most once in `routing_engines_used` even when both the retry and fallback orchestrators fire on the same request. - Governance plugin gains two additional log entries: one when load balancing is skipped because the model is already provider-prefixed, and one when a fallback provider is skipped due to model refinement failure. - UI adds `core` as a recognized routing engine with a sky-blue color scheme and a `Workflow` icon. The `loadbalancing` engine color is changed from red to orange. Badge rendering is updated to pass icon size props correctly. - Prometheus and telemetry documentation updated to include `core` in the `routing_engine_used` label description. - Retries and fallbacks documentation gains a new "Auditing retry and fallback decisions" section with a full table of log entry shapes and a note on the intentional omission of upstream provider messages. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [x] Docs ## How to test ```sh # Core/Transports go version go test ./... ``` Configure a provider with retries and a fallback chain. Trigger a primary failure (e.g. an invalid API key or a rate-limited endpoint) and inspect the routing engine log trail on the resulting request log. Verify: 1. `core` appears in `routing_engines_used` exactly once. 2. The log trail contains entries for the primary failure, each fallback attempt, and the terminal outcome (success or exhaustion). 3. No upstream provider error message text appears in any `core` log entry — only error type and HTTP status code. 4. For retry scenarios, each retry entry notes whether the key was rotated or reused. ```sh # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` In the log detail view, confirm that requests involving fallbacks or retries show a `Core` badge in sky-blue alongside any plugin engine badges, and that the `Loadbalancing` badge now renders in orange rather than red. ## Breaking changes - [ ] Yes - [x] No ## Security considerations `routingErrorSummary()` is specifically designed to exclude upstream provider messages from the routing log trail, since providers can echo back API keys, tokens, or user-supplied content. Only the error type (e.g. `rate_limit_error`) and HTTP status code are recorded. Key rotation notes surface the user-set key **name**, not the secret value. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…terprise fallback (maximhq#4196) ## Summary Adds a dedicated **Settings** sub-route (`/workspace/adaptive-routing/settings`) to the Adaptive Routing section, giving it a tabbed sidebar structure similar to other sections like Custom Pricing. On OSS builds, the settings page reuses the existing enterprise upsell fallback from the adaptive routing dashboard rather than introducing a duplicate. ## Changes - Added `/workspace/adaptive-routing/settings` as a child route with its own layout and page component, rendering `LoadBalancerSettingsView` from the enterprise layer. - Updated the adaptive routing layout to use `useChildMatches` and `<Outlet />` so the dashboard renders at the base path while child routes (e.g. `/settings`) render independently. - Added `Dashboard` and `Settings` sub-items to the Adaptive Routing sidebar entry, mirroring the tab pattern used elsewhere. - Extended the `isRouteMatch` exact-match logic in the sidebar to include `/workspace/adaptive-routing`, preventing the Dashboard tab from remaining highlighted when the Settings tab is active. - Added an OSS fallback for `loadBalancerSettingsView` that re-exports the existing `adaptiveRoutingView` upsell component. - Registered `LoadBalancerConfig` as a tag in the base API for cache invalidation. - Updated the sidebar description from "Manage adaptive load balancer" to "Manage adaptive routing". ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to the Adaptive Routing section in the sidebar. 2. Confirm the sidebar now shows **Dashboard** and **Settings** sub-items. 3. Click **Dashboard** — verify it renders the adaptive routing dashboard and the Dashboard tab is highlighted. 4. Click **Settings** — verify it renders the settings view and the Settings tab is highlighted (Dashboard tab should not remain highlighted). 5. On an OSS build, verify the Settings page displays the same enterprise upsell as the Dashboard page. ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` ## Screenshots/Recordings _Add before/after screenshots showing the new sidebar sub-items and the settings page._ ## Breaking changes - [x] No ## Related issues ## Security considerations No new auth surfaces introduced. The existing RBAC check (`RbacResource.AdaptiveRouter`) in the layout guards both the dashboard and the new settings route. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Adaptive Routing Settings page with dedicated navigation and dashboard access. * Enhanced sidebar navigation with sub-items for Adaptive Routing Dashboard and Settings. * Integrated load balancer settings into the Adaptive Routing interface with enterprise fallback support. * Improved plugin execution order to ensure provider selection occurs after routing components. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
… field reference, troubleshooting, and updated screenshots (maximhq#4206) ## Summary Rewrites the semantic caching documentation to accurately reflect how the feature works, including the two distinct lookup paths (direct hash matching and semantic similarity), the mandatory cache key requirement, and the asynchronous write behavior. Also adds a new log-details screenshot and updates the existing config screenshot. ## Changes - Replaced the overview with a clearer description of the two caching paths (direct and semantic) and a Mermaid flow diagram showing the full lookup sequence. - Added a "How it works" section that calls out the four most common first-time pitfalls: missing cache key, direct-before-semantic ordering, async writes, and persistence across restarts. - Consolidated the configuration reference into a single field table covering all options (`provider`, `embedding_model`, `dimension`, `ttl`, `threshold`, `conversation_history_threshold`, `exclude_system_prompt`, `cache_by_model`, `cache_by_provider`, `vector_store_namespace`, `default_cache_key`). - Restructured configuration tabs to lead with the Web UI, then API, then `config.json`, then Go SDK — matching the most common usage order. - Added an API tab showing `POST /api/plugins` and `PUT /api/plugins/semantic_cache` examples for enabling, updating, and disabling the plugin without a restart. - Replaced the separate "Direct Hash Mode" section with a comparison table and folded the setup instructions into the main configuration section. - Expanded the `cache_debug` metadata table to include all fields (`cache_hit`, `cache_id`, `hit_type`, `threshold`, `similarity`, `provider_used`, `model_used`, `input_tokens`) with accurate presence conditions. - Added documentation for cache visibility in the Logs UI: hit-type badges, the Cache row with copyable `cache_id`, the Caching Details block, and the Local Caching filter. - Replaced the "Cache Lifecycle & Cleanup" prose with a concise bullet list and clarified that entries persist across restarts (previous docs implied a restart would clear the cache). - Added a Troubleshooting accordion section covering the six most common failure modes. - Added a "Next steps" section linking to vector store setup, plugins overview, and providers docs. - Updated `ui-semantic-cache-config.png` and added `ui-semantic-cache-log-details.png`. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered docs to confirm: 1. The Mermaid diagram renders correctly. 2. All tab groups (`config-method`, `direct-hash-setup`, `cache-triggering`, `per-request-overrides`, `cache-clear`) display the correct content per tab. 3. The `ui-semantic-cache-log-details.png` image renders in the Cache Management section. 4. The Troubleshooting accordions expand and collapse correctly. 5. Internal anchor links (`#prerequisites`, `#configuration-reference`, `#cache-lifecycle--cleanup`, `#cache-management`) resolve without 404s. ## Screenshots/Recordings Updated `ui-semantic-cache-config.png` reflects the revised UI layout. New `ui-semantic-cache-log-details.png` shows the log detail sheet with the Semantic Cache badge and Caching Details block. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. Documentation-only change; no secrets, auth flows, or PII handling modified. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…heckRequest` context marker (maximhq#4207) ## Summary Introduces a new context key, `BifrostContextKeyMCPHealthCheckRequest`, that marks MCP ping and `list_tools` requests as internally generated by Bifrost's health monitor. This allows plugins, hooks, and other middleware to distinguish these internal probes from caller-initiated requests. ## Changes - Added `BifrostContextKeyMCPHealthCheckRequest` context key (`"bifrost-mcp-health-check-request"`) to the set of reserved Bifrost context keys. - In `performHealthCheck`, the timeout context is now wrapped in a `BifrostContext` with the new key set to `true` before being passed to `runPingWithHooks` / `runListToolsWithHooks`, ensuring the marker propagates through the entire health check call chain. - The key is reserved (added to `reservedKeys`) so it cannot be overridden externally — the comment explicitly notes it should not be set manually. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Write a plugin or hook that inspects the incoming context for `BifrostContextKeyMCPHealthCheckRequest`. Trigger a health check cycle and verify the key is present and set to `true` for ping/list_tools probes, while being absent for normal caller-initiated requests. ```sh go test ./... ``` ## Breaking changes - [ ] Yes - [x] No ## Security considerations The new context key is reserved and explicitly documented as not to be set manually, preventing external callers from spoofing health check requests to bypass plugin logic that gates on this marker. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io> Co-authored-by: stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>
## Summary `PluginSpanFilter` and its associated logic (`ShouldExportSpan`, `BuildReparentMap`, `PluginNameFromSpan`) were previously defined and implemented inside the OTEL plugin package. This PR promotes them to `core/schemas` so they can be shared across all observability connectors (OTEL, Datadog, BigQuery) without duplicating the span-name contract or reparenting behavior. The OTEL package re-exports the types and constants as aliases to preserve existing import paths. The `PluginTracingSheet` UI component is also generalized to accept a `pluginName` and `destination` prop, and is relocated from the plugins page to the OTEL observability view where it belongs. ## Changes - Introduced `core/schemas/span_filter.go` with `PluginSpanFilter`, `PluginSpanFilterMode`, `PluginNameFromSpan`, `ShouldExportSpan`, and `BuildReparentMap`, along with full unit test coverage in `span_filter_test.go`. - Removed the duplicate `shouldExportSpan` and `buildReparentMap` methods from `plugins/otel/converter.go`; call sites now delegate to the shared schema methods. - `PluginSpanFilter`, `PluginSpanFilterMode`, and the include/exclude constants in `plugins/otel/main.go` are replaced with type aliases and const aliases pointing to `core/schemas`, keeping the OTEL package's public API unchanged. - Validation in `otel.Init` is replaced with a call to `config.PluginSpanFilter.Validate()`. - `PluginTracingSheet` is moved from `ui/app/workspace/plugins/sheets/` to `ui/app/workspace/observability/sheets/` and now accepts `pluginName` and `destination` props, making it connector-agnostic. - The "Configure Plugin Tracing" button and `PluginTracingSheet` are removed from the plugins page and plugins empty state, and are instead surfaced directly in `OtelView`. - Added a warning notice to the `ent-v1.4.7` changelog about a known `/virtual-key/quota` issue fixed in v1.4.8. - Improved the `v1.5.11` changelog rollback section with a warning callout and collapsible `AccordionGroup` sections for single-node and multi-node rollback SQL. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [x] Docs ## How to test ```sh # Core/Transports go test ./core/schemas/... ./plugins/otel/... # UI cd ui pnpm i pnpm build ``` - Open the Observability → OTEL view and confirm the "Configure Plugin Tracing" button appears and opens the sheet correctly. - Verify the sheet reads and writes `plugin_span_filter` only for the `otel` plugin. - Confirm the Plugins page no longer shows a "Configure Plugin Tracing" button or sheet. - Confirm the plugins empty state no longer renders the tracing button. ## Screenshots/Recordings N/A — functional behavior is unchanged; only the location of the tracing button has moved. ## Breaking changes - [ ] Yes - [x] No The OTEL package re-exports all renamed types and constants as aliases, so existing config parsing and external consumers are unaffected. ## Related issues N/A ## Security considerations No new auth, secrets, PII handling, or sandboxing changes introduced. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added configurable span filtering for plugin observability exports with include/exclude modes to control which plugins' spans are exported to observability connectors. * Extended plugin tracing configuration to support any backend plugin destination, not limited to a single connector. * **Refactor** * Consolidated span filtering logic for improved reusability and consistency across observability integrations. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…q#4201) ## Summary Extends `plugin_span_filter` support to the Datadog observability connector and introduces a new BigQuery traces connector, while renaming the shared `otelPluginSpanFilter` / `otel_plugin_span_filter` schema definition to the more generic `pluginSpanFilter` / `plugin_span_filter` so it can be reused across all observability connectors. ## Changes - Added `plugin_span_filter` support to the Datadog plugin in both the Helm chart template (`_helpers.tpl`) and its schema/values definitions, matching the existing pattern used by OTEL connectors. - Renamed the `otelPluginSpanFilter` / `otel_plugin_span_filter` schema `$defs` entry to `pluginSpanFilter` / `plugin_span_filter` in both `values.schema.json` and `transports/config.schema.json`, and updated all `$ref` usages accordingly. The description was also updated to clarify that the filter applies to any observability connector, not just OTEL. - Added a full JSON schema definition for a new `bigquery` observability connector in `transports/config.schema.json`, including fields for `project_id`, `dataset_id`, `table_id`, `location`, `service_account_key` (with ADC fallback), `flush_interval_seconds`, `buffer_size`, `custom_labels`, `disable_content_logging`, `request_headers`, and `plugin_span_filter`. - Added the `bigquery` plugin to the Helm chart (`values.yaml`, `values.schema.json`, and `_helpers.tpl`) with the same `version` validation guard used by other built-in plugins. - Added commented-out `plugin_span_filter` examples to `values.yaml` for both the Datadog and BigQuery plugins to aid discoverability. ## Type of change - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test 1. Deploy the Helm chart with a Datadog plugin config that includes `plugin_span_filter`: ```yaml bifrost: plugins: datadog: config: plugin_span_filter: mode: "exclude" plugins: ["logging"] ``` Verify the rendered manifest includes `plugin_span_filter` in the Datadog plugin config. 2. Deploy the Helm chart with the BigQuery plugin enabled: ```yaml bifrost: plugins: bigquery: enabled: true version: 1 config: project_id: "my-gcp-project" dataset_id: "bifrost_traces" table_id: "traces" ``` Verify the rendered manifest includes the BigQuery plugin config with the expected fields. 3. Validate `transports/config.schema.json` against a BigQuery connector config: ```json { "name": "bigquery", "config": { "project_id": "my-gcp-project", "plugin_span_filter": { "mode": "include", "plugins": ["auth"] } } } ``` 4. Confirm that no dangling `$ref` entries referencing the old `otelPluginSpanFilter` / `otel_plugin_span_filter` names remain in either schema file. ## Breaking changes - [x] Yes - [ ] No The `otelPluginSpanFilter` / `otel_plugin_span_filter` `$defs` keys have been renamed to `pluginSpanFilter` / `plugin_span_filter`. Any external tooling or configs that reference these definition names directly will need to be updated. ## Related issues ## Security considerations The BigQuery connector schema supports `service_account_key` via an environment variable reference (`env.MY_VAR`) or Application Default Credentials, avoiding the need to embed raw credentials in config files. Care should be taken to ensure service account keys are not logged or exposed through the `custom_labels` or `request_headers` fields. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Documents the `plugin_span_filter` configuration option for the Datadog connector, and corrects the UI navigation path for the OTEL connector's plugin span filtering instructions.
## Changes
- Added a new **Plugin Span Filtering** section to the Datadog connector docs, covering `exclude`/`include` filter modes, config.json usage, UI configuration, built-in plugin names, child span re-parenting behavior, and a note on per-connector filter independence and config versioning precedence.
- Updated the OTEL connector docs to correct the UI navigation path from the generic "Plugins page" to the specific **Observability** page → **Open Telemetry** connector flow, matching the Datadog connector's updated instructions.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Review the rendered documentation for:
- The new **Plugin Span Filtering** section appearing correctly in the Datadog connector page, including the config.json example, filter mode table, built-in plugin name list, and the `<Note>` callout.
- The OTEL connector page showing the corrected UI navigation path ("Open the **Observability** page, select the **Open Telemetry** connector...").
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
None.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **Documentation**
* Added a Plugin Span Filtering guide for Datadog APM explaining how to include/exclude plugin execution spans, supported filter modes (include/exclude), built-in plugin names, example configuration, and how filtering re-parents child spans in traces.
* Clarified OpenTelemetry connector docs to use the Observability page for plugin tracing and that connector config can override UI-saved filters by version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Adds support for multimodal function responses (images and files returned by tools) in the Gemini provider. Previously, when a tool returned image or file content alongside text, the media was either dropped or serialized as a raw JSON fallback. This PR preserves those media blocks by routing them through `FunctionResponse.Parts` (a Gemini 3+ feature), with correct provider-specific behavior for the Gemini Developer API vs. Vertex AI.
## Changes
- **`FunctionResponse.Parts` field added** to the `FunctionResponse` type so images/files returned by tools can be attached as sibling parts alongside the structured response object.
- **Forward conversion (`convertResponsesMessagesToGeminiContents`)** now accepts `model` and `provider` arguments. For Gemini 3+ models, image/file content blocks from `ResponsesFunctionToolCallOutputBlocks` are converted to `inlineData`/`fileData` parts and attached to `FunctionResponse.Parts`. For older models (e.g. `gemini-2.5-flash`), media is silently dropped to avoid a hard upstream 400. Vertex AI emits a `{"$ref": "<displayName>"}` entry in the response object (as documented); the Gemini Developer API does not (the `$ref` form triggers an upstream bug).
- **Reverse conversion (`convertGeminiContentsToResponsesMessages`)** reconstructs multimodal function responses back into `ResponsesFunctionToolCallOutputBlocks` (text + image blocks), preserving media on the Bifrost side instead of collapsing everything to a plain string.
- **`Part.UnmarshalJSON`** now handles snake_case fallbacks (`inline_data`, `file_data`) emitted by the google-genai SDK inside `functionResponse.parts`.
- **`Blob.UnmarshalJSON`** now handles snake_case fallbacks (`mime_type`, `display_name`) from `FunctionResponseBlob`.
- **`FileData.UnmarshalJSON`** added with snake_case fallbacks (`mime_type`, `file_uri`, `display_name`) from `FunctionResponseFileData`.
- **Unit tests** added for: image preserved on Gemini 3 (Developer API, no `$ref`), image dropped on older models, Vertex emitting `$ref`, and a full round-trip (`GeminiGenerationRequest` → `BifrostResponsesRequest` → `GeminiGenerationRequest`).
- **Integration tests** added (`test_30`, `test_30b`) covering a fabricated multimodal tool history and a real two-turn workflow, parameterized across `gemini-3-flash-preview` (image understood) and `gemini-2.5-flash` (image dropped, request still succeeds).
## Type of change
- [x] Bug fix
- [x] Feature
## Affected areas
- [x] Core (Go)
- [x] Providers/Integrations
## How to test
```sh
# Unit tests
go test ./core/providers/gemini/... -run TestResponsesAPIParallelFunctionCalling
go test ./core/providers/gemini/... -run TestMultimodalFunctionResponse_RoundTrip
# Integration tests (requires GEMINI_API_KEY)
cd tests/integrations/python
pytest tests/test_google.py::TestGoogleProvider::test_30_multimodal_function_response_image
pytest tests/test_google.py::TestGoogleProvider::test_30b_multimodal_function_response_full_workflow
```
Expected outcomes:
- `gemini-3-flash-preview`: model identifies the tool-returned image color as "red".
- `gemini-2.5-flash`: request succeeds without a 400; model produces a text reply (image was dropped by gating).
## Breaking changes
- [x] No
## Security considerations
No new auth, secrets, or PII surface. Base64 image data passes through in-memory only and is not logged or persisted.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **New Features**
* Improved multimodal function-response support: tools can return images alongside text for Gemini 3 models; other model types gracefully fall back to text-only or reference-style handling.
* **Compatibility**
* Better handling of provider/model variations so image-containing tool outputs are preserved where supported and safely downgraded otherwise.
* **Tests**
* Added end-to-end and regression tests validating multimodal function-response round‑trips and field preservation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…aximhq#4228) ## Summary Updates workspace dependency and tooling metadata required by the Skills Repository stack, including Go workspace/module version alignment, Nix tooling updates, transport Git-serving dependencies, and UI package metadata. This keeps cross-module dependency churn isolated at the bottom of the Graphite stack so later feature PRs are easier to review. ## Changes - Aligned Go workspace/module metadata across CLI, core, framework, plugins, tests, and transports for the Skills Repository stack. - Added transport-layer Git repository construction dependencies used by later Git-backed marketplace serving work. - Updated Nix flake tooling for the workspace Go toolchain. - Updated UI package/build metadata needed by the dashboard work. - Kept this as a chore-only base PR so subsequent Skills Repository PRs can focus on product/code changes. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test Validated in the Skills Repository stack with: ```sh # From bifrost/ direnv exec . go build ./transports/bifrost-http/... ./framework/configstore/... direnv exec . go test ./transports/bifrost-http/handlers -run 'Test.*Skill' direnv exec . go test ./framework/configstore -count=1 # From bifrost/ui direnv exec . npm run build ``` Expected result: targeted Go build succeeds, Skills HTTP handler tests pass, configstore tests pass, and the UI production build completes successfully. No new configs or environment variables are added in this PR. ## Screenshots/Recordings N/A — no direct UI behavior changes in this PR. ## Breaking changes - [ ] Yes - [x] No If yes, describe impact and migration instructions. ## Related issues N/A ## Security considerations No direct security changes. This PR only updates dependency/tooling metadata. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate (no new tests required for metadata-only dependency/tooling changes) - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated Go and third-party dependencies to latest stable versions for improved compatibility and security. * Refined build system configuration to enhance type-checking during the build process. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary The `GET /api/governance/virtual-keys/quota` endpoint previously returned only overall and per-provider budgets/rate limits. This PR extends it to also return per-model budget quotas and rate limits configured for the virtual key, and enriches each top-level budget with a breakdown of actual per-model spend (requests, tokens, cost) drawn from request logs over that budget's current cycle. ## Changes - Added a `model_configs` field to `VirtualKeyQuotaResponse` exposing per-model budgets and rate limits (specific-model configs only; wildcard `"*"` configs continue to feed the existing VK-level and provider-level governance fields). - Each entry in `budgets` now embeds a `per_model_usage` array containing actual spend per model sourced from request logs, windowed to the budget's `[last_reset, now]` cycle. The list is empty when the logging plugin is not enabled. - Replaced the per-call `GetModelConfig` lookup in the quota path with a single bulk `GetModelConfigsByScopeAndScopeIDs` query. Wildcard configs are reverse-mapped onto VK/provider governance (same hydration as before); specific-model configs are returned as the new `model_configs` list. - `NewGovernanceHandler` now accepts an optional `logging.LogManager`. When the logging plugin is active, the server wires its log manager through so the quota endpoint can query `GetModelRankings` per budget cycle. When nil, the per-model spend breakdown is omitted gracefully. - Updated OpenAPI spec (`openapi.json`, `governance.yaml`) with new schemas: `VirtualKeyBudgetUsage`, `VirtualKeyModelSpend`, and `VirtualKeyModelUsage`. - Extended unit and end-to-end tests to cover the new `model_configs` field, the per-budget `per_model_usage` breakdown, and the assertion that `GetModelRankings` is called with the correct VK scope and cycle window. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test ```sh go test ./transports/bifrost-http/handlers/... -run TestGetVirtualKeyQuota go test ./... ``` 1. Start the server with both the governance and logging plugins enabled. 2. Make requests through a virtual key that has per-model limits configured. 3. Call `GET /api/governance/virtual-keys/quota` with the virtual key in the `x-bf-vk` header. 4. Verify the response contains a `model_configs` array with the configured per-model budgets and rate limits. 5. Verify each entry in `budgets` contains a `per_model_usage` array reflecting actual spend from request logs, scoped to the budget's current cycle. 6. Repeat with the logging plugin disabled and confirm `per_model_usage` is an empty array rather than an error. ## Breaking changes - [ ] Yes - [x] No `VirtualKeyQuotaResponse` gains two new fields (`model_configs`, `per_model_usage` inside each budget). Existing consumers are unaffected as the additions are additive. `NewGovernanceHandler` gains a new `logManager` parameter; callers must pass `nil` if logging is not available. ## Related issues ## Security considerations The quota endpoint is self-service and authenticated solely by the virtual key value. The new fields expose only governance configuration and usage data scoped to that specific virtual key — no cross-key data is accessible. The `GetModelRankings` query is explicitly filtered by `VirtualKeyID` to enforce this boundary. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Quota responses now include per-model configurations and embed per-model current-cycle usage totals alongside provider-level quotas and rate limits. * **Documentation** * API docs updated to describe per-model quota/rate-limit details, required model config data in responses, current-cycle usage, and self-service access. * **Bug Fixes** * Endpoint now fails closed (500) on model-config or usage-lookup errors. * **Tests** * Added/updated tests to assert per-model usage, query scoping, cycle windowing, and fail-closed behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary Fixes a bug where `DeleteCustomer` fails with a foreign key constraint violation when attempting to `DELETE FROM governance_budgets WHERE customer_id = ?`. The root cause is that `governance_customers.budget_id` rows were left populated by a previous migration (`migrationAddCustomerBudgetsToBudgetsTable`), causing the FK check (`fk_governance_customers_budget`) to block deletion of the referenced budget rows. Since budget ownership now lives on `governance_budgets.customer_id`, the legacy `budget_id` values on `governance_customers` can be safely nulled — a `NULL` reference satisfies the FK unconditionally. ## Changes - Adds a new migration `migrationNullLegacyCustomerBudgetID` that: - Performs a defensive backfill to ensure any `governance_budgets` rows that still lack a `customer_id` (e.g. written by an older instance in a mixed-version cluster) are claimed before the legacy references are cleared. - Nulls all non-null `governance_customers.budget_id` values, resolving the FK conflict without dropping the column or constraint (deferred to a major release). - Includes a best-effort rollback that repopulates `budget_id` from `governance_budgets.customer_id`, picking the oldest budget for customers with multiple budgets. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/configstore/... ``` 1. Ensure a customer with an associated budget exists in the database (with `governance_customers.budget_id` populated). 2. Run the migration and confirm `governance_customers.budget_id` is `NULL` for all rows. 3. Confirm `governance_budgets.customer_id` is correctly populated. 4. Attempt to delete the customer and verify no FK constraint error occurs. ## Breaking changes - [ ] Yes - [x] No The `governance_customers.budget_id` column and its FK are retained; only the values are cleared. ## Related issues Closes the FK violation bug introduced by the `migrationAddCustomerBudgetsToBudgetsTable` migration leaving legacy `budget_id` references intact. ## Security considerations None. This migration only modifies internal budget ownership references and does not affect auth, secrets, or PII. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Added a database migration to clean up legacy customer→budget links: claims unowned budgets safely, refreshes affected customer configuration, and clears deprecated legacy references to improve data consistency. * **Bug Fixes** * Improved rollback behavior to best-effort restore legacy customer→budget links when possible, preferring the oldest matching budget. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary Adds `http_request_size_bytes` and `http_response_size_bytes` histogram metrics to the OTel plugin, giving operators visibility into HTTP payload sizes alongside the existing request count and duration metrics. ## Changes - Added `RecordHTTPMetrics` method to `OtelPlugin` that records request count, duration, and request/response body sizes in a single call. Non-positive sizes (e.g. `-1` when `Content-Length` is unknown in fasthttp) are skipped to avoid polluting histograms with sentinel values. - Added an OTel HTTP metrics middleware in `PrepareCommonMiddlewares` that captures request size before the handler runs, then resolves the OTel plugin instance after the response is complete. The plugin is resolved per-request rather than captured at startup to avoid recording against stale meter providers after a config reload. - Documented the two new metrics (`http_request_size_bytes`, `http_response_size_bytes`) in the OTel observability reference docs. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test Start Bifrost with an OTel plugin configured and a metrics exporter pointing at an OTLP-compatible backend (e.g. Prometheus via the OTLP receiver or an OTel Collector). Send a few HTTP requests and verify the following metrics appear with `path`, `method`, and `status` labels: ```sh # Example: query Prometheus curl -s http://localhost:9090/api/v1/query?query=http_request_size_bytes_bucket | jq . curl -s http://localhost:9090/api/v1/query?query=http_response_size_bytes_bucket | jq . ``` Confirm that requests without a `Content-Length` header (size reported as `-1` by fasthttp) do **not** produce a data point in `http_request_size_bytes`. ```sh go test ./plugins/otel/... ./transports/bifrost-http/... ``` ## Screenshots/Recordings N/A ## Breaking changes - [x] No ## Related issues N/A ## Security considerations No new auth, secrets, or PII surface. Request and response sizes are recorded as numeric values only; no body content is captured. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * HTTP request and response body size histograms are now collected and exported via OpenTelemetry with explicit bucket boundaries; sizes are recorded only when Content-Length is present. * Per-request metrics now include counts, durations, and conditional payload-size measurements to improve request observability. * **Documentation** * Observability docs updated to describe the new size histograms and their recording conditions. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary Adds `GetTeamName` and `GetCustomerName` accessors to `LocalGovernanceStore` so the enterprise layer has a reliable fallback for log stamping when its edge-driven name caches miss — specifically for teams with no user members and no associated business unit. ## Changes - Added `GetTeamName` to look up a team's display name from the in-memory store, returning `""` for unknown or empty IDs. - Added `GetCustomerName` to look up a customer's display name from the in-memory store, returning `""` for unknown or empty IDs. - Added `TestGetTeamNameAndGetCustomerName` to verify both accessors return correct names for known entities and empty strings for unknown or empty IDs. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./plugins/governance/... ``` Expected: all tests pass, including `TestGetTeamNameAndGetCustomerName`, which validates that known team/customer IDs return their display names and unknown/empty IDs return `""`. ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No auth, secrets, or PII implications. The accessors read only display names from the in-memory store, which is already trusted internal state. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added helper methods to retrieve team and customer display names by ID from cached data; unknown or invalid IDs (including empty) return an empty string as a safe fallback. * **Tests** * Added test coverage verifying correct display names are returned for known IDs and that unknown or empty IDs gracefully yield empty strings. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary Metadata is now written to object store snapshots so that consumers reading log objects directly can see custom attributes. Previously, metadata was intentionally excluded from snapshots and kept exclusively in the database. The DB row remains authoritative — metadata is still not restored from the snapshot during hydration, and `ClearPayload` does not strip it from the DB row. ## Changes - `ExtractPayload` now includes the `metadata` field in the object store snapshot payload when it is non-empty, allowing external object consumers to access custom attributes without querying the database. - `MergePayloadFromJSON` continues to intentionally skip restoring metadata from the snapshot, preserving the DB row as the authoritative source. - Tests updated to assert that metadata is present in the object store snapshot and that the round-trip payload count reflects the additional field. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/logstore/... ``` Verify that: - `TestHybrid_MetadataIsRetainedInDBAndWrittenToObjectPayload` passes, confirming metadata appears in the raw object store payload. - `TestExtractPayload_RoundTrip` passes, confirming the snapshot payload contains the metadata field. - Hydration via `FindByID` still returns metadata sourced from the DB row, not the snapshot. ## Breaking changes - [ ] Yes - [x] No ## Security considerations Metadata written to object store snapshots may contain user-identifiable or tenant-scoped attributes (e.g., `cortex-user-id`). Ensure object store access controls are appropriate for any environments where metadata sensitivity is a concern. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Snapshots now include log metadata (e.g., cortex-user-id and team) so object consumers see the same metadata while the database remains the authoritative source. * **Tests** * Updated tests to assert metadata is written into snapshots; added tests to ensure metadata is omitted when nil or empty. * **Documentation** * Clarified that snapshot metadata is for external consumers while DB row metadata remains authoritative. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary Moves the TypeScript type check (`tsc --noEmit`) to run **after** the Vite build step rather than before, for both the `build` and `build-enterprise` scripts. This allows the build artifact to be produced even when there are type errors, which is useful for unblocking deployments or debugging build output when type issues are present. ## Changes - In `build-enterprise`: `typecheck` now runs after `vite build` instead of before. - In `build`: `typecheck` now runs after `vite build` instead of before, while `copy-build` remains the final step. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui npm i npm run build npm run build-enterprise ``` Verify that the build output is generated successfully and that type checking still runs and reports any errors after the build completes. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated build script execution order. Type-checking now runs after the build process completes in both standard and enterprise builds, rather than before. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
maximhq#4235) * fix: metadata filters and pagination total_count for Postgres logstore - Always match metadata values as JSON strings in the Postgres JSONB containment query; header-sourced values are always stored as strings, so the previous numeric/boolean type detection caused zero results. - Replace `IS JSON OBJECT` (PG 15+) with `jsonb_typeof(metadata::jsonb) = 'object'` in applyFilters, GetDistinctMetadataKeys, and the GIN index partial predicate so the query guard and index predicate align and the planner uses the index. - Assign pagination.TotalCount = totalCount in SearchLogs; it was only stored in Stats.TotalRequests, so every response returned total_count: 0. * fix: align GIN index predicate check and cleanup query with jsonb_typeof ensureMetadataGINIndex now reads the stored index predicate via pg_get_expr and drops the index when it still carries the old IS JSON OBJECT guard so it gets rebuilt with the compatible jsonb_typeof predicate; without this, existing PG 15+ deployments would keep the stale index and metadata filter queries would fall back to sequential scans. cleanupInvalidLogMetadata replaces the PG 15+ IS NOT JSON OBJECT guard with jsonb_typeof(metadata::jsonb) <> 'object' for consistency with the rest of the file. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove jsonb_typeof changes --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment on lines
+209
to
+219
| func isJSONObject(raw json.RawMessage) bool { | ||
| var value map[string]json.RawMessage | ||
| return json.Unmarshal(raw, &value) == nil | ||
| } | ||
|
|
||
| func isEmptyJSONObject(raw json.RawMessage) bool { | ||
| var value map[string]json.RawMessage | ||
| if err := json.Unmarshal(raw, &value); err != nil { | ||
| return false | ||
| } | ||
| return len(value) == 0 |
Contributor
There was a problem hiding this comment.
isJSONObject / isEmptyJSONObject treat JSON null as a valid object, causing a server panic
json.Unmarshal([]byte("null"), &mapVar) returns nil error while leaving the map as nil. Both helpers therefore return true for a null literal. This feeds directly into a panic in mergeJSONObject:
- An upstream provider (e.g. OpenRouter) returns
"pricing": nullfor a model with no known pricing. UnmarshalJSONstores the raw bytes inRawModelJSON, somerged["pricing"] = json.RawMessage("null").- The catalog enrichment path in
inference.goseesmodelEntry.Pricing == niland sets a non-nil&schemas.Pricing{Prompt: …}. MarshalJSONcallsmergeJSONObject(json.RawMessage("null"), {"prompt":"…"}).- Inside,
json.Unmarshal("null", &baseMap)succeeds withbaseMap == nil. The firstbaseMap[key] = valueassignment then panics with "assignment to entry in nil map", crashing the/v1/modelshandler.
The minimal fix is to guard against a nil map in mergeJSONObject, or to treat null as a non-object in both helpers (e.g. reject it before the Unmarshal call).
akshaydeo
force-pushed
the
dev
branch
4 times, most recently
from
June 21, 2026 11:44
fa15f50 to
ca190fc
Compare
akshaydeo
force-pushed
the
dev
branch
2 times, most recently
from
July 1, 2026 12:24
ac30a53 to
7c66b20
Compare
akshaydeo
force-pushed
the
dev
branch
2 times, most recently
from
July 18, 2026 01:10
44564de to
493bff0
Compare
Contributor
|
Hi @kristiandrucker — thanks for the contribution! This PR is currently blocked because our CLA bot shows the Contributor License Agreement as not yet signed. Could you sign it here so we can move this forward: https://cla-assistant.io/maximhq/bifrost?pullRequest=4301 Let us know if you run into any issues signing. |
|
@kristiandrucker Thank you for your work on this; I would like to mention that Bifrost dropping This is a critical parameter for which a simple passthrough strategy is unfortunately insufficient, given the fragmented convention around it. Would be useful to be able to map Appreciate this goes beyond data layer and the configuration would mean UI layer impacts for Bifrost. |
Author
|
Hey @chris-hatton. My main issue I wanted to fix was model list metadata over from upstream because its missing model features, context sizes and so on from upstream openai provider |
akshaydeo
force-pushed
the
dev
branch
3 times, most recently
from
August 13, 2026 09:47
244a01d to
ce1b2a6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes metadata loss in native
/v1/modelsresponses for OpenAI-compatible providers.Today, when an upstream provider returns rich model objects, Bifrost narrows them down to a small subset of fields and drops useful metadata such as
name,description,context_length,architecture,supported_parameters,top_provider, and richer nestedpricingfields. This change preserves upstream model metadata at the top level while keeping Bifrost’s existing response envelope, normalized model IDs, and pricing enrichment behavior.Changes
schemas.Modelso unknown fields survive/v1/modelsserialization instead of being dropped.idvalues and preserving existing core fields such asid,object,created, andowned_by.OpenAI/gpt-4odo not break model matching or pricing enrichment./v1/modelsbehavior.Notable design decision:
schemas.ModelJSON preservation hook instead of introducing a separatemetadatawrapper. That keeps the response shape passthrough-compatible for model pickers and agent runtimes, at the cost of slightly widening the serialization blast radius forschemas.Model.Type of change
Affected areas
How to test
Validate the schema, provider conversion, model-catalog normalization, and runtime behavior.
Expected outcomes:
Manual validation against a real provider:
Expected outcome:
New configs/environment variables:
Screenshots/Recordings
N/A - no UI changes.
Breaking changes
If yes, describe impact and migration instructions.
Related issues
N/A
Security considerations
No new auth, secret, or PII handling changes. This change only preserves upstream model metadata already returned by configured providers.
Checklist
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Tests