test: add model catalog wiring e2e collection and generator script - #4197
Draft
Pratham-Mishra04 wants to merge 24 commits into
Draft
test: add model catalog wiring e2e collection and generator script#4197Pratham-Mishra04 wants to merge 24 commits into
Pratham-Mishra04 wants to merge 24 commits into
Conversation
## Summary Adds an "Allow Private Network" toggle to the custom provider creation form, enabling users to configure whether a custom provider can connect to private network IP ranges (e.g., `192.168.x.x`, `10.x.x.x`). Link-local addresses remain blocked regardless of this setting. ## Changes - Added `allow_private_network` as an optional boolean field to the custom provider form schema, defaulting to `false` - Wired the field value into `network_config.allow_private_network` when saving the provider - Added a labeled toggle switch in the form UI with a description clarifying which address ranges are affected and which remain blocked ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to the custom provider creation sheet in the workspace providers UI. 2. Verify the "Allow Private Network" toggle is visible and defaults to off. 3. Enable the toggle and save the provider — confirm `allow_private_network: true` is included in the saved `network_config`. 4. Disable the toggle and save — confirm `allow_private_network: false` is sent. 5. Verify the toggle is disabled when the user lacks provider create access. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings _Add before/after screenshots of the custom provider form showing the new toggle._ ## Breaking changes - [ ] Yes - [x] No ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations This toggle explicitly opts a custom provider into connecting to private network ranges. It defaults to `false` (blocked), preserving the existing secure-by-default behavior. Link-local addresses remain blocked unconditionally to prevent SSRF via metadata endpoints. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an "Allow Private Network" toggle option in the custom provider creation form, enabling users to control private network access settings when setting up custom providers. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This was referenced Jun 9, 2026
Merged
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This was referenced Jun 9, 2026
Merged
Merged
Collaborator
Author
17 tasks
|
tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Pratham-Mishra04
force-pushed
the
06-09-tests_adds_routing_test_harness
branch
from
June 9, 2026 07:36
26a4096 to
1046c21
Compare
…ing per-deployment model family, canonical name, and provider overrides (#4185) ## Summary Replaces the flat "Aliases" key-value table (mapping request model name → string deployment ID) with a richer "Deployments" table that supports per-deployment metadata and provider-specific overrides. This enables cost/pricing logs and family-based routing to work correctly for custom deployments. ## Changes - Introduced a new `DeploymentsTable` component that renders each deployment as a collapsible row. Expanding a row exposes fields for canonical model name, model family, description, and provider-specific overrides (Azure API version, endpoint, Anthropic version; Vertex project ID/number/region; Bedrock region and inference profile ARN; Replicate deployments endpoint toggle). - Replaced the `normalizeAliasesValue` helper and `HeadersTable`-based aliases editor in `apiKeysFormFragment.tsx` with the new `DeploymentsTable`. The form label and description were updated from "Aliases" to "Deployments" to reflect the richer semantics. - Added `AliasConfig` and `ModelFamily` types to `config.ts`, mirroring the Go `schemas.AliasConfig` struct (with embedded provider sub-configs flattened to top-level fields on the wire). - Added `aliasConfigSchema` and `modelFamilySchema` Zod schemas to `schemas.ts`. The alias schema uses `z.preprocess` to accept the legacy `string` wire shape emitted by the Go server for simple aliases, coercing it to `{ model_id: string }` so hydrated state passes validation without a migration. - Updated `KeySchema` in `providerForm.ts` to use `z.record(z.string(), aliasConfigSchema)` and updated the validation error message to reflect the new requirement. - Rewrote `isValidAliases` in `validation.ts` to validate the rich `Record<string, { model_id?: string }>` shape, checking that every entry has a non-empty deployment name and a non-empty `model_id`. - Updated `ModelProviderKey` in `config.ts` to type `aliases` as `Record<string, AliasConfig>` instead of `Record<string, string>`. - The `DeploymentsTable` includes a draft row at the bottom for adding new entries. The draft is committed automatically when both the deployment name and model ID are filled. Rename collision detection is case-insensitive and stable row IDs are used to preserve expanded/pending state across renames. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i pnpm build ``` 1. Navigate to a provider key configuration form. 2. Verify the "Deployments" table renders in place of the old "Aliases" table. 3. Add a new deployment by filling in the deployment name and model ID in the draft row — confirm it commits automatically when both fields are populated. 4. Expand a committed row and verify the canonical model name, model family, description, and provider-specific override fields are visible and editable. 5. For Azure, Vertex, Bedrock, and Replicate providers, confirm the correct provider-specific section appears in the expanded panel. 6. Rename a deployment to an existing name and confirm the inline collision error appears and the row is not committed. 7. Load an existing config that uses the legacy `Record<string, string>` alias format and confirm it hydrates correctly into the new table without validation errors. ## Screenshots/Recordings Before: A simple two-column key/value table labeled "Aliases" with a plain text input for the deployment ID. After: A collapsible table labeled "Deployments" where each row can be expanded to reveal canonical model name, model family, description, and provider-specific override fields. ## Breaking changes - [x] Yes - [ ] No The `aliases` field type changes from `Record<string, string>` to `Record<string, AliasConfig>` in the UI type system and form schema. Existing configs using the legacy string format are handled transparently via the `aliasConfigSchema` preprocessor and the `normalize` function in `DeploymentsTable`, so no manual migration is required for stored configs. Any code outside this diff that directly constructs or reads `ModelProviderKey.aliases` as `Record<string, string>` will need to be updated. ## Related issues ## Security considerations No new secrets or auth surfaces introduced. Provider-specific override fields (endpoint, credentials) use the existing `EnvVarInput` component, which supports environment variable references and redaction consistent with the rest of the form. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…and alias resolution context (#4186) ## Summary Introduces a structured `RoutingInfo` field on both `BifrostResponseExtraFields` and `BifrostErrorExtraFields` that exposes per-attempt routing context (provider, model, key used, resolved alias, and fallback signals) in a single, well-typed struct. The existing `Provider`, `OriginalModelRequested`, and `ResolvedModelUsed` fields are preserved for backward compatibility but deprecated in favour of `RoutingInfo`. ## Changes - Added `RoutingInfo` and `ResolvedKeyAlias` schema types capturing the provider, model, key name, resolved alias metadata, fallback flag, and primary provider/model when a fallback occurred. - Added `BuildRoutingInfo` helper in `account.go` that constructs a per-attempt `RoutingInfo` from the current context, provider, model, and key, including any resolved alias config. - Added `PopulateRoutingInfo` on `BifrostResponse` and `BifrostError` to stamp `RoutingInfo` onto responses/errors and keep the deprecated triplet in sync via the shared `syncDeprecatedFromRoutingInfo` helper. - Added `SetFallbackRoutingInfo` on both types, called by the orchestrator (`handleRequest` / `handleStreamRequest`) to layer on `IsFallback`, `PrimaryProvider`, and `PrimaryModel` after a fallback attempt completes. These signals are intentionally set at the orchestrator scope rather than inside per-attempt code. - `requestWorker` now seeds `attemptRoutingInfo` with the known provider/model before the retry loop so that early failures (e.g. key selection errors) still produce a populated `RoutingInfo` on the error. Each retry iteration snapshots a `perAttemptRoutingInfo` to avoid races in async streaming closures. - `PopulateRoutingInfo` is called alongside `PopulateExtraFields` both before and after `RunPostLLMHooks`, ensuring plugin modifications cannot corrupt routing metadata. - `ProcessedStreamResponse` gains a `RoutingInfo` field to carry routing context through the streaming pipeline. - The deprecated `Provider`, `OriginalModelRequested`, and `ResolvedModelUsed` fields are annotated with deprecation notices and derivation rules pointing consumers to the equivalent `RoutingInfo` paths. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go version go test ./... ``` Verify that responses and errors include a populated `routing_info` object in their JSON output. For fallback scenarios, confirm `is_fallback` is `true` and `primary_provider`/`primary_model` reflect the original attempt. Confirm that `provider`, `original_model_requested`, and `resolved_model_used` continue to be populated with the same values as before. ## Breaking changes - [ ] Yes - [x] No Existing fields are preserved. `RoutingInfo` is additive. ## Related issues ## Security considerations None. No auth, secrets, or PII are introduced. `RoutingInfo` surfaces key names already present in existing fields. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…dd backward-compat fallback for legacy `ExtraFields` (#4187) ## Summary Replaces the flat `(provider, originalModel, resolvedModel)` triplet passed into `resolvePricing` with a structured `schemas.RoutingInfo` value. This aligns pricing lookups with the routing context that `core.bifrost` already populates on every response, and introduces a well-defined lookup precedence: `AliasModelName → AliasModelID → ModelName`, with overrides keyed by the wire model identifier. ## Changes - `resolvePricing` now accepts a single `schemas.RoutingInfo` argument instead of three separate string parameters. The lookup iterates over `[AliasModelName, AliasModelID, ModelName]`, stopping at the first catalog hit, and applies overrides keyed by the wire model (`AliasModelID` when an alias matched, otherwise `ModelName`). - `calculateBaseCost` reads `RoutingInfo` directly from `ExtraFields`. A backward-compatibility fallback synthesises a `RoutingInfo` from the deprecated `Provider`/`OriginalModelRequested`/`ResolvedModelUsed` triplet only when `RoutingInfo` is fully unset (zero `Provider`, zero `Model`, nil `ResolvedKeyAlias`). Partial population is trusted as-is to prevent false-positive fallbacks. - `computeCacheEmbeddingCost` constructs a minimal `RoutingInfo` from the cache-debug fields, since no alias resolution context exists for cache-replayed requests. - Container pricing overrides build a synthetic `RoutingInfo` that pins both model fields to the container identifier, preserving per-container override addressability. - All call sites in tests are updated to pass `schemas.RoutingInfo` structs directly. - New backward-compat tests cover: legacy-fields-only (no alias), legacy-fields-only (with alias/resolved model), `RoutingInfo` winning over legacy fields when both are set, both empty returning zero cost, and partial `RoutingInfo` suppressing the fallback. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/... ``` Key scenarios to verify: - Pricing resolves correctly for a non-aliased request via `RoutingInfo.Model`. - Pricing resolves via `AliasModelID` when an alias was matched and the wire model differs from the caller-facing name. - Pricing resolves via `AliasModelName` when the admin tagged a canonical name on the alias. - Legacy callers with only `Provider`/`OriginalModelRequested`/`ResolvedModelUsed` populated still receive correct costs. - When both `RoutingInfo` and the deprecated triplet are set, `RoutingInfo` wins. - Partial `RoutingInfo` (e.g. `Model` set but `Provider` empty) does not trigger the legacy fallback. ## Breaking changes - [ ] Yes - [x] No The `resolvePricing` method is unexported. The public `CalculateCost` API is unchanged. The backward-compat fallback ensures existing callers writing only the deprecated `ExtraFields` triplet continue to receive correct cost calculations. ## Related issues ## Security considerations None. This change affects cost accounting logic only; no auth, secrets, or PII are involved. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…pricing lookup precedence (#4188) ## Summary Documents the rich object form for alias values, the new `routing_info` response block, and the pricing lookup precedence that uses canonical `model_name` to resolve opaque deployment IDs against the catalog. ## Changes - Expanded the alias schema documentation to cover the object form alongside the existing plain-string shorthand. The object form accepts `model_id`, `model_name`, `model_family`, `description`, `region`, and provider-specific overrides (`api_version`, `anthropic_version`, `endpoint` for Azure; `project_id`, `project_number` for Vertex; `inference_profile_arn` for Bedrock; `use_deployments_endpoint` for Replicate). - Added a validation rule documenting that provider-specific sub-config fields are rejected when the owning key belongs to a different provider. - Replaced the flat `extra_fields` response fields (`original_model_requested`, `resolved_model_used`, `provider`) with the new `routing_info` block. The old fields are noted as deprecated but still populated for backward compatibility. - Added a `routing_info` field reference table covering `provider`, `model`, `key`, `resolved_key_alias`, `is_fallback`, `primary_provider`, and `primary_model`. - Added a pricing lookup precedence section explaining the three-candidate resolution order (`model_name` → `model_id` → caller-sent model) and how it solves the opaque deployment ID problem for cost attribution. - Added a cross-reference note in the Azure provider config page pointing readers to the full alias object schema. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered docs for: - Correct table formatting for the alias object schema and provider-specific overrides - The `routing_info` JSON example rendering properly - The pricing lookup precedence section appearing between the wildcard patterns section and the request type filtering section - The deprecation `<Note>` rendering in the aliasing-models page ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…/block/alias views (#4189) ## Summary Introduces a new `keyconfig` package under `framework/modelcatalog` that provides a thread-safe, in-memory store for per-key configuration (allowed models, blacklisted models, and aliases) across all configured providers. This centralizes the aggregation logic previously scattered in the load balancer plugin, making it available as a pure transformation layer for routing-time queries. ## Changes - Added `framework/modelcatalog/keyconfig/store.go` implementing a `Store` type that: - Maintains an immutable `providerState` snapshot per provider, swapped atomically under a write lock so readers never observe torn state - Aggregates allowed models as the union of enabled keys' `Models` fields minus per-key blacklisted entries, collapsing to `["*"]` when any enabled key is unrestricted - Computes the provider-level blacklist as the intersection across enabled keys (a model is only provider-blocked when every enabled key blacklists it) - Builds a case-insensitive alias index keyed by lowercase alias name, with last-enabled-key-wins collision resolution and a debug log on collision - Treats keyless non-standard (custom) providers as unrestricted (`["*"]`) to support ambient/IAM auth flows - Drops providers from the store entirely when they have no routable keys, keeping the store focused on routing-time queries rather than full config inspection - Exposes `Replace` (full atomic resync), `SetProvider` (single-provider update), `RemoveProvider`, `EntriesFor`, `EntryFor`, `AllowedFor`, `BlacklistedFor`, `IsAllowed`, `ResolveAlias`, `Providers`, and `KeysAllowingModel` - Added `framework/modelcatalog/keyconfig/store_test.go` with comprehensive behavioral tests covering: wildcard and explicit allow lists, blacklist intersection, disabled keys, block-all keys, alias ownership and collision, case-insensitive blacklist and alias normalization, atomic snapshot correctness under concurrent reads, and defensive copy guarantees ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/keyconfig/... ``` The test suite covers all aggregation semantics, concurrency safety (atomic snapshot test with 200 Replace cycles and a concurrent reader), and edge cases including nil logger, keyless non-standard providers, and case-insensitive alias collision detection. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations The store holds API key IDs and model allow/block lists in memory. No secrets (key values) are stored — only key IDs and routing metadata. The alias index is keyed by lowercase model name; alias configs may contain region or deployment override fields that are treated as read-only by callers. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ntry support (#4190) ## Summary Introduces a thread-safe, in-memory cache (`live.Store`) for provider model catalog responses. The store holds per-`(provider, keyID, unfiltered)` entries and is intentionally passive — it never initiates network calls. Callers are responsible for fetching and pushing results via `Upsert`, keeping the cache decoupled from transport concerns. ## Changes - Added `framework/modelcatalog/live/store.go` with a `Store` type that caches `/v1/models` responses keyed by provider, key ID, and a filtered/unfiltered flag. - Filtered entries are pre-gated by the provider's `ListModelsPipeline` at write time; callers reading filtered entries must not reapply that gate to avoid dropping alias-backfill rows. - `ModelsForProvider` and `UnfilteredModelsForProvider` return the sorted, deduplicated union across all matching keys for a provider. - `Invalidate` drops both filtered and unfiltered entries for a single key (e.g., on credential rotation or key deletion). `InvalidateProvider` drops all entries for a provider (e.g., on provider deletion). - `Snapshot` returns a full defensive copy of the store for diagnostics. - Both `Upsert` and `Snapshot` copy slices to prevent external mutation of cached state. - Keyless providers (Vertex workload identity, Bedrock IAM, etc.) are supported via an empty `KeyID`. - Added `framework/modelcatalog/live/store_test.go` covering union across keys, filtered/unfiltered isolation, invalidation behavior, defensive copying, overwrite semantics, and keyless provider handling. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/live/... ``` All tests should pass. Key scenarios covered: - Filtered and unfiltered entries for the same key do not bleed into each other. - Union across multiple keys for the same provider is deduplicated and sorted. - `Invalidate` removes both filtered and unfiltered entries for the target key while leaving other keys intact. - `InvalidateProvider` removes all entries for the target provider while leaving other providers intact. - Mutating the input slice after `Upsert`, or mutating a `Snapshot`, does not affect store state. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations Cached model lists may include key IDs as cache discriminators. The store holds no credential values — only the key ID string used as a lookup discriminator. No PII or secrets are stored. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…erKeysByID` helper (#4193) ## Summary Adds a `KeyID` field to `BifrostListModelsRequest` that scopes a `ListModels` call to a single key matched by `Key.ID`. This allows callers such as the catalog composer to cache list-models output per-key for fine-grained invalidation without requiring an extra round-trip or having the provider aggregate results across every configured key. ## Changes - Added `KeyID *string` to `BifrostListModelsRequest` (tagged `json:"-"` so it is never forwarded to providers). When set, the request worker filters the already-validated key set down to the single matching key before dispatching. If no key matches, a `BifrostError` is returned immediately. - Added `filterKeysByID` helper that returns a new slice containing only keys whose `ID` equals the target, leaving the input slice unmodified. - Added `TestFilterKeysByID` covering: a successful match, a missing key, an empty target string, and input-slice immutability. Note: the lookup runs against the already-filtered set of supported keys (disabled or invalid keys are excluded before the match), so a `KeyID` pointing to a disabled key produces the same "no key found" error as a non-existent `KeyID`. Callers that need to distinguish these cases must inspect the raw account configuration directly. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go version go test ./... ``` The new `TestFilterKeysByID` test directly exercises the helper and the key-scoping logic. To validate end-to-end, issue a `ListModels` request with `KeyID` set to a valid key ID and confirm only that key's models are returned. Issue one with an unknown `KeyID` and confirm a "no key found" error is returned. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations `KeyID` is tagged `json:"-"` and is never serialised or forwarded to any external provider. No secrets or PII are introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…cache fanout via `OnKeyAdded/Updated/Deleted` (#4194) ## Summary Replaces the coarse provider-level model discovery (one aggregated live entry per provider) with per-key live cache entries, so adding, updating, or deleting a single key triggers at most 2 list-models calls for that key rather than 2×N calls across all keys. Removes the deprecated compatibility shims (`UpsertModelDataForProvider`, `UpsertUnfilteredModelDataForProvider`, `DeleteModelDataForProvider`) and replaces them with `UpsertLiveFromResponse`, `OnKeyAdded`, `OnKeyUpdated`, and `OnKeyDeleted`. ## Changes - **`modelcatalog/pool.go`**: Added `UpsertLiveFromResponse`, which extracts and deduplicates model IDs from a `BifrostListModelsResponse` before writing to the live cache. A nil response is a no-op, preventing accidental cache eviction. - **`modelcatalog/shims.go`**: Deleted. The three deprecated shim methods are gone; call sites now use the per-key API directly. - **`modelcatalog/pool_test.go`**: New test file covering `UpsertLiveFromResponse` (nil no-op, happy path), `extractModelIDs` (prefix stripping, gateway nested prefixes, foreign prefix filtering, nil input, deduplication), `InvalidateLive`, and `InvalidateLiveProvider`. - **`server/server.go`**: Replaced `populateModelPoolWithListModels` (one aggregated entry per provider) with `RefreshLiveModelsForProvider` (fans out per key in parallel) and `FetchAndStoreLiveForKey` (issues filtered + unfiltered list-models for a single key). `ReloadProvider` now reads keys from the in-memory store, calls `SetKeyConfigForProvider` + `InvalidateLiveProvider`, then delegates to `RefreshLiveModelsForProvider`. `ForceReloadPricing` and `ReloadPricingFromDBAndPopulateModelPool` no longer trigger a full model pool refresh — pricing reload is now pricing-only. `RemoveProvider` calls `InvalidateLiveProvider` + `RemoveKeyConfigForProvider` instead of the deleted shim. Added `OnKeyAdded`, `OnKeyUpdated`, `OnKeyDeleted` to `ServerCallbacks` and implemented them on `BifrostHTTPServer`. - **`handlers/provider_keys.go`**: Key create/update/delete handlers now call `modelsManager.OnKeyAdded/OnKeyUpdated/OnKeyDeleted` instead of `attemptModelDiscovery`. Keyless providers skip the add/update path. - **`handlers/providers.go`**: Extended `ModelsManager` interface with `OnKeyAdded`, `OnKeyUpdated`, `OnKeyDeleted`. - **`handlers/providers_test.go`** and **`governance/httptransportprehook_test.go`**: Updated to use `UpsertLiveFromResponse` and `NewTestCatalog(nil)` instead of the removed shims and bare struct literals. - **`governance/resolver_test.go`**: Removed two tests that depended on the shim API; equivalent coverage exists in the catalog-level tests. - **`modelcatalog/models.go`**: Removed stale comment referencing the pre-refactor file. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/modelcatalog/... go test ./transports/bifrost-http/... go test ./plugins/governance/... go test ./... ``` After adding a provider key via `POST /api/providers/{provider}/keys`, verify that only the new key's models are fetched (2 list-models calls) rather than a full provider refresh. After deleting a key, confirm the deleted key's live entries are evicted while other keys' entries remain intact. ## Screenshots/Recordings N/A ## Breaking changes - [x] Yes - [ ] No `UpsertModelDataForProvider`, `UpsertUnfilteredModelDataForProvider`, and `DeleteModelDataForProvider` are removed from `ModelCatalog`. Any external code calling these methods must migrate to `UpsertLiveFromResponse` / `UpsertLive` / `InvalidateLiveProvider`. `ServerCallbacks` now requires `OnKeyAdded`, `OnKeyUpdated`, and `OnKeyDeleted` — implementors must add these three methods. ## Related issues N/A ## Security considerations No new auth surfaces. Key validation (`BifrostContextKeyValidateKeys`) is preserved in `FetchAndStoreLiveForKey`, maintaining the same key-validation behavior at boot, after key add, and after provider reload. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ansitions (#4195) ## Summary Adds a `core` routing engine to the per-request audit trail so that fallback transitions and retry transitions made by the Bifrost orchestrator itself are visible alongside decisions made by plugins like `governance`, `loadbalancing`, `routing-rule`, and `model-catalog`. Previously, the routing engine log trail went silent after a plugin selected the primary upstream — there was no record of why core advanced through the fallback chain or how many retries were attempted before succeeding or giving up. ## Changes - Introduces `RoutingEngineCore = "core"` as a named routing engine constant, emitted by `handleRequest`, `handleStreamRequest`, and `executeRequestWithRetries` at each fallback and retry transition. - `handleRequest` and `handleStreamRequest` now log: primary failure entering the fallback loop, each fallback attempt (with the triggering error), skipped fallbacks (missing provider config), successful fallback, short-circuit halts, and full fallback exhaustion. - `executeRequestWithRetries` uses a named return + `defer` to guarantee a terminal log entry on every return path (including early exits from key-selection failures). Each retry transition records whether the key was rotated or reused, and keyless providers omit the key segment entirely. - Adds `routingErrorSummary()` — a sanitized formatter that surfaces only the error type and HTTP status code, deliberately excluding the upstream provider message to prevent API keys, tokens, or user input from leaking into log storage or the UI. - `AppendToContextList` is tightened from `any` to `comparable` and gains a deduplication check, so `core` (and any other engine) appears at most once in `routing_engines_used` even when both the retry and fallback orchestrators fire on the same request. - Governance plugin gains two additional log entries: one when load balancing is skipped because the model is already provider-prefixed, and one when a fallback provider is skipped due to model refinement failure. - UI adds `core` as a recognized routing engine with a sky-blue color scheme and a `Workflow` icon. The `loadbalancing` engine color is changed from red to orange. Badge rendering is updated to pass icon size props correctly. - Prometheus and telemetry documentation updated to include `core` in the `routing_engine_used` label description. - Retries and fallbacks documentation gains a new "Auditing retry and fallback decisions" section with a full table of log entry shapes and a note on the intentional omission of upstream provider messages. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [x] Docs ## How to test ```sh # Core/Transports go version go test ./... ``` Configure a provider with retries and a fallback chain. Trigger a primary failure (e.g. an invalid API key or a rate-limited endpoint) and inspect the routing engine log trail on the resulting request log. Verify: 1. `core` appears in `routing_engines_used` exactly once. 2. The log trail contains entries for the primary failure, each fallback attempt, and the terminal outcome (success or exhaustion). 3. No upstream provider error message text appears in any `core` log entry — only error type and HTTP status code. 4. For retry scenarios, each retry entry notes whether the key was rotated or reused. ```sh # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` In the log detail view, confirm that requests involving fallbacks or retries show a `Core` badge in sky-blue alongside any plugin engine badges, and that the `Loadbalancing` badge now renders in orange rather than red. ## Breaking changes - [ ] Yes - [x] No ## Security considerations `routingErrorSummary()` is specifically designed to exclude upstream provider messages from the routing log trail, since providers can echo back API keys, tokens, or user-supplied content. Only the error type (e.g. `rate_limit_error`) and HTTP status code are recorded. Key rotation notes surface the user-set key **name**, not the secret value. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Pratham-Mishra04
changed the base branch from
feat_add_lb_ui_stubs
to
graphite-base/4197
June 9, 2026 11:03
…terprise fallback (#4196) ## Summary Adds a dedicated **Settings** sub-route (`/workspace/adaptive-routing/settings`) to the Adaptive Routing section, giving it a tabbed sidebar structure similar to other sections like Custom Pricing. On OSS builds, the settings page reuses the existing enterprise upsell fallback from the adaptive routing dashboard rather than introducing a duplicate. ## Changes - Added `/workspace/adaptive-routing/settings` as a child route with its own layout and page component, rendering `LoadBalancerSettingsView` from the enterprise layer. - Updated the adaptive routing layout to use `useChildMatches` and `<Outlet />` so the dashboard renders at the base path while child routes (e.g. `/settings`) render independently. - Added `Dashboard` and `Settings` sub-items to the Adaptive Routing sidebar entry, mirroring the tab pattern used elsewhere. - Extended the `isRouteMatch` exact-match logic in the sidebar to include `/workspace/adaptive-routing`, preventing the Dashboard tab from remaining highlighted when the Settings tab is active. - Added an OSS fallback for `loadBalancerSettingsView` that re-exports the existing `adaptiveRoutingView` upsell component. - Registered `LoadBalancerConfig` as a tag in the base API for cache invalidation. - Updated the sidebar description from "Manage adaptive load balancer" to "Manage adaptive routing". ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to the Adaptive Routing section in the sidebar. 2. Confirm the sidebar now shows **Dashboard** and **Settings** sub-items. 3. Click **Dashboard** — verify it renders the adaptive routing dashboard and the Dashboard tab is highlighted. 4. Click **Settings** — verify it renders the settings view and the Settings tab is highlighted (Dashboard tab should not remain highlighted). 5. On an OSS build, verify the Settings page displays the same enterprise upsell as the Dashboard page. ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` ## Screenshots/Recordings _Add before/after screenshots showing the new sidebar sub-items and the settings page._ ## Breaking changes - [x] No ## Related issues ## Security considerations No new auth surfaces introduced. The existing RBAC check (`RbacResource.AdaptiveRouter`) in the layout guards both the dashboard and the new settings route. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added Adaptive Routing Settings page with dedicated navigation and dashboard access. * Enhanced sidebar navigation with sub-items for Adaptive Routing Dashboard and Settings. * Integrated load balancer settings into the Adaptive Routing interface with enterprise fallback support. * Improved plugin execution order to ensure provider selection occurs after routing components. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Pratham-Mishra04
force-pushed
the
06-09-tests_adds_routing_test_harness
branch
from
June 9, 2026 11:06
74d1049 to
5ad8298
Compare
Pratham-Mishra04
force-pushed
the
graphite-base/4197
branch
from
June 9, 2026 11:06
accf1e2 to
23a767c
Compare
Pratham-Mishra04
force-pushed
the
06-09-tests_adds_routing_test_harness
branch
from
June 9, 2026 11:07
5ad8298 to
45ceff4
Compare
akshaydeo
force-pushed
the
dev
branch
4 times, most recently
from
June 19, 2026 07:23
7f86f4e to
2f96e3a
Compare
akshaydeo
force-pushed
the
dev
branch
2 times, most recently
from
July 1, 2026 12:24
ac30a53 to
7c66b20
Compare
akshaydeo
force-pushed
the
dev
branch
2 times, most recently
from
July 18, 2026 01:10
44564de to
493bff0
Compare
akshaydeo
force-pushed
the
dev
branch
3 times, most recently
from
August 13, 2026 09:47
244a01d to
ce1b2a6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new end-to-end test suite that verifies the wiring between the management API and the model catalog — specifically that mutations to providers and keys (add, update, delete, toggle, alias) propagate correctly into the
/api/modelsand/api/models/detailsread endpoints. Also fixes a plugin ordering bug where the model catalog resolver plugin was placed at a fixed order that could run before enterprise post-builtin plugins like the load balancer.Changes
New Postman collection (
bifrost-model-catalog-wiring.postman_collection.json): A machine-generated collection covering six wiring contracts — adding a provider/key surfaces models, updating a key's allow-list re-gates the catalog, disabling/re-enabling a key drops and restores models, deleting one of two keys leaves the sibling's models intact, deleting a provider removes all its models, and alias resolution routes inference to the underlying wire model. Each scenario uses an isolated run-namespaced custom provider backed by real OpenAI so parallel runs never collide.Collection generator (
runners/build-model-catalog-wiring-collection.py): A Python script that holds the scenario spec as the source of truth and emits the Postman JSON. To extend or modify scenarios, edit this script and re-run it; the JSON is not hand-edited.Test runner (
runners/individual/run-newman-model-catalog-wiring-tests.sh): A Bash 4+ script that loads credentials from the seed env file or shell environment, forwards the full per-provider credential set, and runs the collection via Newman with exponential-backoff-aware timeouts.Plugin ordering fix (
transports/bifrost-http/server/plugins.go): Changed the model catalog resolver plugin's placement from a fixed order of9withinbuiltinPlacementtomath.MaxIntwithinpost_builtin. This ensures it runs after all other post-builtin plugins, including the enterprise load balancer, so those plugins get a chance to select the provider before the catalog resolver runs.README updates: Documents the new test suite, its scenarios, requirements, run instructions, and how to regenerate the collection.
ui/package-lock.json: Added trailing newline.Type of change
Affected areas
How to test
Ensure Bifrost is running locally against a clean config store (no pre-existing
catwiring-*providers), then run:To regenerate the collection after modifying scenarios:
Commit both the script and the regenerated JSON.
Requirements:
npm install -g newman newman-reporter-htmlextra)http://localhost:8080openai_api_keyavailable in the seed env file (generated/seed.envor$BIFROST_E2E_SEED_ENV) or exported in the shellbrew install bash)Scenarios whose required credentials are absent skip themselves rather than fail.
Breaking changes
Related issues
Security considerations
API keys are read from environment variables or a seed env file and forwarded to Newman as
--env-varflags. The seed env file is parsed without being sourced to prevent execution of embedded command substitutions.Checklist
docs/contributing/README.mdand followed the guidelines