feat: add PreRequestHook to LLMPlugin interface for once-per-request provider/model routing - #4175
Merged
Merged
Conversation
|
|
Collaborator
Author
This was referenced Jun 9, 2026
Merged
Contributor
|
Warning Review limit reached
More reviews will be available in 21 minutes and 17 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (23)
📝 WalkthroughWalkthroughThis PR introduces ChangesPreRequestHook Routing Phase
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This was referenced Jun 9, 2026
Merged
Contributor
Confidence Score: 4/5Safe to merge once the previously-flagged BaseGovernancePlugin interface gap is resolved; the new hook phase and pipeline wiring are correct. The core RunPreRequestHooks logic, pool acquisition/release, context blocking, and fallback re-read are all correct. The two open threads from prior reviews — the BaseGovernancePlugin interface missing PreRequestHook (which can silently drop the governance plugin from the LLM pipeline for custom implementations) and the filterProvidersByContext removal (which drops governance-scoped provider filtering from ListAllModels) — remain unaddressed in this diff. plugins/governance/main.go — BaseGovernancePlugin interface; core/bifrost.go — filterProvidersByContext removal from ListAllModels Important Files Changed
Reviews (2): Last reviewed commit: "feat: add RoutingHook for plugins" | Re-trigger Greptile |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/bifrost.go`:
- Around line 4178-4189: After running pipeline.RunPreRequestHooks(ctx, req)
ensure plugin-scoped logs are flushed or cleared: call
bifrost.drainAndAttachPluginLogs(ctx) and if that would be a no-op when no trace
metadata is present, also call the context/log clear function (e.g.
bifrost.clearPluginLogs(ctx) or the existing method that removes buffered plugin
logs on BifrostContext) so logs are not carried to later requests; apply the
same change pattern to the other PreRequestHook-only exit sites noted (around
lines 4655-4657 and 4757-4759) referencing RunPreRequestHooks and the
drainAndAttachPluginLogs/clear plugin-log helper methods.
- Around line 4645-4647: The PreRequestHook contract must be tightened: ensure
plugin code invoked in PreRequestHook only mutates routing metadata (use
UpdateProvider, UpdateModel, UpdateAPIKey, UpdateProviderBaseURL,
UpdateBaseProviderType) and must not modify content-bearing
slices/maps/pointers; update the hook handling in PreRequestHook and in
prepareFallbackRequest to either (a) enforce/validate that only routing fields
were changed or (b) perform a deep copy of the mutable request payload (body,
slices, maps, pointers) before invoking any plugin hooks so fallback attempts
are isolated; locate and change the code paths around PreRequestHook and
prepareFallbackRequest to implement the enforcement or deep-copy strategy.
In `@core/schemas/plugin.go`:
- Around line 267-284: The new PreRequestHook should not be added as a required
method on LLMPlugin; instead revert that change and introduce a new optional
interface type PreRequestPlugin (with the method PreRequestHook(ctx
*BifrostContext, req *BifrostRequest) error) following the MCPConnectionPlugin
pattern, then update the pipeline caller to detect and invoke PreRequestHook via
a type assertion on plugins (e.g., if p, ok := plugin.(PreRequestPlugin); ok {
p.PreRequestHook(...) }) so existing source-built plugins that don't implement
routing remain compatible and LLMPlugin stays small and stable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: d455ef2e-cd6a-4380-af6b-619ba699b9e0
📒 Files selected for processing (23)
core/bifrost.gocore/bifrost_test.gocore/schemas/plugin.gocore/utils.goexamples/plugins/hello-world/main.goexamples/plugins/llm-only/main.goexamples/plugins/multi-interface/main.goframework/plugins/main.goframework/plugins/soloader.goframework/plugins/soplugin.goframework/tracing/tracer_test.goplugins/compat/main.goplugins/governance/main.goplugins/jsonparser/main.goplugins/logging/main.goplugins/maxim/main.goplugins/mocker/main.goplugins/prompts/main.goplugins/semanticcache/main.goplugins/semanticcache/plugin_no_mutation_test.goplugins/telemetry/main.gotransports/bifrost-http/handlers/realtime_client_secrets_test.gotransports/bifrost-http/lib/config_test.go
💤 Files with no reviewable changes (1)
- core/bifrost_test.go
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Pratham-Mishra04
force-pushed
the
feat_adds_routing_hooks_to_plugins
branch
from
June 9, 2026 08:10
cb5f525 to
2e715eb
Compare
Collaborator
Author
Merge activity
|
akshaydeo
pushed a commit
that referenced
this pull request
Jun 12, 2026
…est provider/model routing (#4175) ## Summary Introduces a new `PreRequestHook` phase to the `LLMPlugin` interface. This hook runs exactly once per top-level request — after `HTTPTransportPreHook` and before `PreLLMHook` — and is the canonical place for plugins to resolve provider, model, and fallback routing decisions. Previously, routing logic had to be shoehorned into `PreLLMHook`, which runs on every fallback attempt and whose mutations have incidental cross-fallback visibility. `PreRequestHook` mutations are committed to the shared `*BifrostRequest` before any fan-out and are observed by every subsequent plugin, every `PreLLMHook` invocation, the provider call, and every fallback. As part of this change, the `filterProvidersByContext` helper (used in `ListAllModels`) is removed, and request validation is moved to after `PreRequestHook` runs so that plugins have the opportunity to populate provider/model before the empty-field check fires. Error messages for missing provider/model are updated to reflect that auto-resolution was attempted. ## Changes - Added `PreRequestHook(ctx *BifrostContext, req *BifrostRequest) error` to the `LLMPlugin` interface with non-blocking error semantics (logged as warning, pipeline continues). - Added `RunPreRequestHooks` to `PluginPipeline`, executing the hook in registration order once per request with tracing and plugin-scope isolation. - Added `RunPreRequestHooks` as a public method on `Bifrost` for callers (e.g., realtime WebSocket handlers) that bypass the normal inference path. - Moved `validateRequest` to after `PreRequestHook` execution in both `handleRequest` and `handleStreamRequest`, renamed to `validateRequestAfterPreRequestHooks` with updated error messages. - Added primary-provider error logging to `handleStreamRequest` to match `handleRequest` behavior. - Removed `filterProvidersByContext` and its tests from `ListAllModels`. - Updated `DynamicPlugin` (shared-object loader) to optionally load `PreRequestHook` from `.so` plugins; legacy plugins without the export get a no-op passthrough, preserving backward compatibility. - Updated `AsLLMPlugin` to recognize `preRequestHook` as sufficient to qualify a `DynamicPlugin` as an `LLMPlugin`. - Added no-op `PreRequestHook` implementations to all existing plugins (`compat`, `governance`, `jsonparser`, `logging`, `maxim`, `mocker`, `prompts`, `semanticcache`, `telemetry`) and all example/test plugins to satisfy the updated interface. - Updated plugin execution-order documentation in `plugin.go` to describe per-request vs. per-attempt semantics. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Validate that: - A plugin implementing `PreRequestHook` can mutate `req.Provider` and `req.Model` before the provider call, and those mutations are visible to subsequent plugins and fallback attempts. - A plugin that returns a non-nil error from `PreRequestHook` does not abort the request; the pipeline continues to the next plugin and a warning is logged. - Existing plugins with no-op `PreRequestHook` implementations behave identically to before. - Legacy `.so` plugins that do not export `PreRequestHook` load and function correctly with the no-op passthrough. - Requests with no provider set (and no plugin resolving one) return the updated error message: `"could not auto resolve a provider for the request, please specify a provider explicitly"`. ## Breaking changes - [x] Yes - [ ] No The `LLMPlugin` interface gains a new required method `PreRequestHook`. Any external plugin implementing `LLMPlugin` must add a `PreRequestHook` method. Plugins that do not participate in routing should return `nil`. Shared-object (`.so`) plugins are exempt — the loader treats `PreRequestHook` as optional and provides a no-op default. ## Security considerations `PreRequestHook` runs with `BlockRestrictedWrites` active on the context (same as `RunLLMPreHooks`), preventing plugins from writing to restricted context keys during the hook. Plugins cannot abort or gate requests via error return from this hook; authorization and content-policy enforcement must remain in `HTTPTransportPreHook` or via a short-circuit in `PreLLMHook`. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Added `PreRequestHook` plugin phase enabling plugins to perform per-request routing decisions before provider and model validation. * **Refactor** * Request validation now occurs after plugin hooks execute, allowing automatic resolution of routing parameters. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
added a commit
that referenced
this pull request
Jun 12, 2026
* fix: private network toggle in custom provider form
* feat: add `PreRequestHook` to `LLMPlugin` interface for once-per-request provider/model routing (#4175)
## Summary
Introduces a new `PreRequestHook` phase to the `LLMPlugin` interface. This hook runs exactly once per top-level request — after `HTTPTransportPreHook` and before `PreLLMHook` — and is the canonical place for plugins to resolve provider, model, and fallback routing decisions. Previously, routing logic had to be shoehorned into `PreLLMHook`, which runs on every fallback attempt and whose mutations have incidental cross-fallback visibility. `PreRequestHook` mutations are committed to the shared `*BifrostRequest` before any fan-out and are observed by every subsequent plugin, every `PreLLMHook` invocation, the provider call, and every fallback.
As part of this change, the `filterProvidersByContext` helper (used in `ListAllModels`) is removed, and request validation is moved to after `PreRequestHook` runs so that plugins have the opportunity to populate provider/model before the empty-field check fires. Error messages for missing provider/model are updated to reflect that auto-resolution was attempted.
## Changes
- Added `PreRequestHook(ctx *BifrostContext, req *BifrostRequest) error` to the `LLMPlugin` interface with non-blocking error semantics (logged as warning, pipeline continues).
- Added `RunPreRequestHooks` to `PluginPipeline`, executing the hook in registration order once per request with tracing and plugin-scope isolation.
- Added `RunPreRequestHooks` as a public method on `Bifrost` for callers (e.g., realtime WebSocket handlers) that bypass the normal inference path.
- Moved `validateRequest` to after `PreRequestHook` execution in both `handleRequest` and `handleStreamRequest`, renamed to `validateRequestAfterPreRequestHooks` with updated error messages.
- Added primary-provider error logging to `handleStreamRequest` to match `handleRequest` behavior.
- Removed `filterProvidersByContext` and its tests from `ListAllModels`.
- Updated `DynamicPlugin` (shared-object loader) to optionally load `PreRequestHook` from `.so` plugins; legacy plugins without the export get a no-op passthrough, preserving backward compatibility.
- Updated `AsLLMPlugin` to recognize `preRequestHook` as sufficient to qualify a `DynamicPlugin` as an `LLMPlugin`.
- Added no-op `PreRequestHook` implementations to all existing plugins (`compat`, `governance`, `jsonparser`, `logging`, `maxim`, `mocker`, `prompts`, `semanticcache`, `telemetry`) and all example/test plugins to satisfy the updated interface.
- Updated plugin execution-order documentation in `plugin.go` to describe per-request vs. per-attempt semantics.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./...
```
Validate that:
- A plugin implementing `PreRequestHook` can mutate `req.Provider` and `req.Model` before the provider call, and those mutations are visible to subsequent plugins and fallback attempts.
- A plugin that returns a non-nil error from `PreRequestHook` does not abort the request; the pipeline continues to the next plugin and a warning is logged.
- Existing plugins with no-op `PreRequestHook` implementations behave identically to before.
- Legacy `.so` plugins that do not export `PreRequestHook` load and function correctly with the no-op passthrough.
- Requests with no provider set (and no plugin resolving one) return the updated error message: `"could not auto resolve a provider for the request, please specify a provider explicitly"`.
## Breaking changes
- [x] Yes
- [ ] No
The `LLMPlugin` interface gains a new required method `PreRequestHook`. Any external plugin implementing `LLMPlugin` must add a `PreRequestHook` method. Plugins that do not participate in routing should return `nil`. Shared-object (`.so`) plugins are exempt — the loader treats `PreRequestHook` as optional and provides a no-op default.
## Security considerations
`PreRequestHook` runs with `BlockRestrictedWrites` active on the context (same as `RunLLMPreHooks`), preventing plugins from writing to restricted context keys during the hook. Plugins cannot abort or gate requests via error return from this hook; authorization and content-policy enforcement must remain in `HTTPTransportPreHook` or via a short-circuit in `PreLLMHook`.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added `PreRequestHook` plugin phase enabling plugins to perform per-request routing decisions before provider and model validation.
* **Refactor**
* Request validation now occurs after plugin hooks execute, allowing automatic resolution of routing parameters.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* refactor: migrate governance routing from `HTTPTransportPreHook` to `PreRequestHook` with `BifrostRequest`-native mutations (#4176)
## Summary
Governance routing logic has been migrated from `HTTPTransportPreHook` to `PreRequestHook`, operating directly on `BifrostRequest` structs rather than raw HTTP bodies. This eliminates the need to unmarshal/marshal JSON, parse multipart forms, or extract models from URL path parameters inside the governance plugin. Integration-specific normalization (Gemini, Bedrock, etc.) now happens upstream before the request reaches governance, so the plugin sees a clean `provider/model` pair regardless of the originating integration.
## Changes
- `HTTPTransportPreHook` is now a no-op stub retained only to satisfy the `HTTPTransportPlugin` interface; all routing flows through `PreRequestHook`
- `loadBalanceProvider` and `applyRoutingRules` now accept `*schemas.BifrostRequest` instead of `map[string]any` + `*schemas.HTTPRequest`, mutating `Provider`/`Model`/`Fallbacks` directly via typed setters
- `addMCPIncludeTools` (header mutation) replaced by `computeMCPIncludeTools` (returns `[]string`); result is stored on context via `MCPContextKeyIncludeTools` rather than written into HTTP headers
- `validateRequiredHeaders` and `stampGovernanceCtxFromVK` moved to `utils.go`; `stampGovernanceCtxFromVK` also fixes a bug where `Team.CustomerID`/`Team.Customer` were not propagated when the VK had a team association
- `governLargePayload` and `governRealtimeQueryParam` removed; large-payload routing now runs through `runPreRequestRouting` (a thin wrapper that builds a synthetic `BifrostRequest` and calls the same helpers), and realtime WebSocket upgrades are routed via an explicit `RunPreRequestHooks` call in `wsrealtime.go` before the upgrade completes
- `BifrostContextKeyRequestQuery` context key added; query params (lowercased) are now populated in `ConvertToBifrostContext` for normal HTTP requests and explicitly in the WS upgrade path, making them available to governance CEL routing rules via `params["..."]`
- Fallbacks produced by `loadBalanceProvider` are now `[]schemas.Fallback` (typed) instead of `[]string`
- Tests that exercised the old `HTTPTransportPreHook` body-parsing path are skipped with a note to rewrite them as `PreRequestHook` tests in Phase 3
## Type of change
- [ ] Bug fix
- [ ] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./plugins/governance/...
go test ./transports/bifrost-http/...
```
Verify that:
- Governance routing rules (CEL expressions reading `headers[...]` and `params[...]`) resolve correctly for both normal HTTP and WebSocket realtime upgrade requests
- Virtual key load balancing selects a provider and populates typed fallbacks on the `BifrostRequest`
- Large-payload streaming requests route via `LargePayloadMetadata.Model` and the rewritten model is visible to the upstream provider's body rewriter
- Realtime WebSocket upgrades pick up the governance-routed `provider`/`model` before the connection is established
## Breaking changes
- [x] Yes
- [ ] No
`applyRoutingRules` and `loadBalanceProvider` signatures have changed from `(ctx, *HTTPRequest, map[string]any, *TableVirtualKey)` to `(ctx, *BifrostRequest, *TableVirtualKey)`. Any code calling these methods directly (outside the governance plugin itself) must be updated. The `addMCPIncludeTools` method has been removed; callers should use `computeMCPIncludeTools` and store the result on context.
## Related issues
Closes #2516
## Security considerations
No new auth surfaces introduced. Query params are now stored on context with lowercased keys, consistent with how request headers are already handled. No secrets or PII are added to context beyond what was already present.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **New Features**
* Governance routing and load balancing now support WebSocket realtime connections.
* Request query parameters are now accessible for governance routing rules.
* **Improvements**
* Enhanced governance routing by moving rule evaluation to an earlier request processing phase for improved handling of large payloads.
* Added required header validation in governance rules.
* **Tests**
* Migrated governance pre-hook tests; rewrite pending in Phase 3.
* **Chores**
* Updated indirect dependency version.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* refactor: extract provider resolution into `modelcatalogresolver` PreRequestHook plugin (#4177)
## Summary
Provider resolution for unprefixed model strings (e.g. `gpt-4o` without a `provider/` prefix) was previously scattered across every integration router, every provider's `ToBifrost*` converter, and the `CheckAndSetDefaultProvider` utility — each doing its own inline catalog lookup or context-key dance. This PR consolidates all of that into a single, dedicated `modelcatalogresolver` built-in plugin that runs as the last `PreRequestHook` before the request reaches the LLM layer.
## Changes
- **New `plugins/modelcatalogresolver` plugin**: A `PreRequestHook` that fills `req.Provider` from the model catalog when no provider was specified and no earlier routing plugin (governance routing rules, governance VK load balancing, enterprise LB) already set one. It also promotes remaining catalog candidates to fallbacks automatically when the caller didn't configure any. An integration-type hint (`BifrostContextKeyIntegrationType`) biases the pick toward the integration's canonical provider; Azure user-agent detection is handled here instead of inline in the OpenAI converters.
- **Removed `CheckAndSetDefaultProvider`**: The `providerUtils.CheckAndSetDefaultProvider` helper and its `BifrostContextKeyAvailableProviders` / `BifrostContextKeyResolvedProvider` context keys are deleted. All `ToBifrost*` converters across Anthropic, Bedrock, Cohere, Gemini, OpenAI, and Vertex now pass `""` as the default provider to `ParseModelString`, deferring resolution entirely to the plugin layer.
- **Removed `GetRequestModel` / `GetProvidersForModel` from integration routers**: The per-route `RequestModelGetter` callbacks and the inline catalog-lookup block in `GenericRouter.createHandler` are removed. The `HandlerStore` interface no longer requires `GetProvidersForModel`, and `Config.GetProvidersForModel` is deleted. The governance plugin no longer sets `BifrostContextKeyAvailableProviders`.
- **Simplified `resolveModelAndProvider` and `resolveRealtimeTarget`**: These functions in the inference and WebSocket realtime handlers no longer do inline catalog lookups. The realtime handlers (`webrtc_realtime.go`, `realtime_client_secrets.go`) that couldn't go through `PreRequestHook` now call the exported `modelcatalogresolver.ResolveProviderFromCatalog` directly.
- **WebSocket realtime empty-provider guard**: `wsrealtime.go`'s `handleUpgrade` now returns a clear WebSocket error when no routing layer could resolve a provider, mirroring the empty-provider validation in `handleRequest`/`handleStreamRequest`.
- **Observability consolidation**: `EmitModelCatalogRoutingLog` is extracted into `lib/ctx.go` so all paths (normal HTTP, WebRTC, realtime client secrets) emit routing engine logs in the same shape. The `snapshotRealtimeMiddlewareValues` function no longer duplicates this logic.
- **Governance plugin cleanup**: Removed the `BifrostContextKeyAvailableProviders` writes and the fallback-filtering logic from `extractAndParseFallbacks` that depended on them.
- **`BifrostContextKeySkipModelCatalogProviderSelection` removed**: The context key and all references are deleted; the resolver plugin's position as the last hook makes the skip flag unnecessary.
## Type of change
- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/...
go test ./transports/...
go test ./plugins/governance/...
go test ./plugins/modelcatalogresolver/...
```
Verify that requests with unprefixed model strings (e.g. `{"model":"gpt-4o"}` on `/v1/chat/completions`) still resolve to the correct provider when a model catalog is configured. Verify that requests with explicit prefixes (e.g. `{"model":"anthropic/claude-opus-4"}`) are unaffected. Verify that WebSocket realtime connections with unresolvable models receive a `400 invalid_request_error` frame instead of hanging.
## Breaking changes
- [x] Yes
- [ ] No
`HandlerStore.GetProvidersForModel` is removed from the interface — any custom `HandlerStore` implementations must drop this method. `BifrostContextKeyAvailableProviders`, `BifrostContextKeyResolvedProvider`, and `BifrostContextKeySkipModelCatalogProviderSelection` context keys are removed from `schemas`; any code reading or writing these keys must be updated. `CheckAndSetDefaultProvider` is removed from `core/providers/utils`.
## Related issues
## Security considerations
No auth, secrets, or PII changes. Provider resolution is now centralised in a single plugin rather than distributed across converters, reducing the surface area for routing bypasses.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Introduced model catalog resolver for intelligent provider selection based on model availability and integration type.
* Added integration-aware provider preferences (e.g., Azure-OpenAI pairing).
* **Bug Fixes**
* Improved model-to-provider resolution to eliminate context-dependent provider defaults.
* **Refactor**
* Streamlined provider resolution logic across all AI provider integrations.
* Simplified request routing configuration by removing model getter dependencies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* docs: add `PreRequestHook` routing phase to plugin lifecycle and sequencing docs (#4178)
## Summary
Documents the `PreRequestHook` interface introduced in v1.6.x — a new per-request routing phase that fires exactly once before any provider call, distinct from `PreLLMHook` which runs per provider attempt. This clarifies where routing decisions (provider, model, fallbacks) should be made and how they propagate through the fallback chain.
## Changes
- Added `PreRequestHook` to the plugin lifecycle state diagram, sequence diagrams, and execution order descriptions across the architecture and getting-started docs, making clear it runs once per request while `PreLLMHook`/`PostLLMHook` run per attempt
- Added a routing layer order table in `sequencing.mdx` documenting the built-in plugin execution order within `PreRequestHook`: governance (order 4) → enterprise load balancer → model-catalog-resolver (order 9, final fallback)
- Added a full `PreRequestHook` reference section in `writing-go-plugin.mdx` with a comparison table against `PreLLMHook`, a routing example using `SetProvider`/`SetModel`, and notes on the two routing observability helpers (`AppendRoutingEngineLog`, `AppendToContextList`)
- Updated `provider-routing.mdx` to reflect that all three routing layers (governance, enterprise LB Level 1, model-catalog-resolver) now execute inside the `PreRequestHook` phase rather than across separate middleware stages, and updated the flowcharts and execution order lists accordingly
- Clarified that `model-catalog-resolver` now prefers the integration's canonical provider (OpenAI/Anthropic/GenAI/Bedrock/Cohere) when the request arrived via an integration route, rather than always selecting the first catalog candidate
- Updated the log message example from `selecting first:` to `selected:` to match the new resolver behavior
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Review the rendered docs for accuracy against the v1.6.x plugin interface. Verify that:
- The `PreRequestHook` signature (`func PreRequestHook(ctx *schemas.BifrostContext, req *schemas.BifrostRequest) error`) matches the SDK
- The execution order table in `sequencing.mdx` matches the registered plugin orders in the codebase
- The flowcharts in `provider-routing.mdx` correctly reflect that all Level 1 routing now happens in `PreRequestHook`
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
Documents the `PreRequestHook` routing phase introduced alongside the model-catalog-resolver and routing engine changes in v1.6.x.
## Security considerations
None. Documentation-only change.
## Checklist
- [x] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **Documentation**
* Updated plugin lifecycle documentation to clarify `PreRequestHook` execution (v1.6.x+) as a once-per-request routing phase separate from per-attempt hook phases.
* Enhanced plugin sequencing and execution order documentation with clearer per-request vs. per-attempt semantics.
* Expanded provider routing documentation with updated default resolution order and governance/load-balancing interaction details.
* Added comprehensive `PreRequestHook` guidance for plugin developers with routing examples and helpers.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* feat: add routing allowlist enforcement via `BifrostContextKeyRoutingAllowedProviders` (#4179)
## Summary
Introduces a two-level routing allowlist enforcement mechanism that prevents any plugin or user-specified provider prefix from bypassing provider restrictions defined by a Virtual Key's `provider_configs`. Previously, governance could fail to pick a provider but a downstream routing layer could still select a non-permitted provider. This change closes that gap by publishing the VK's allowed-provider set to the request context and enforcing it both cooperatively (in the model catalog resolver) and as a hard guarantee (in core, after all `PreRequestHook` plugins have run).
## Changes
- Adds `BifrostContextKeyRoutingAllowedProviders` context key (`[]ModelProvider`) that plugins can set to constrain which providers are valid for a request. An empty slice means "no provider is permitted" (fail-closed → HTTP 400).
- Adds `enforceRoutingAllowlist` and `filterFallbacksByAllowlist` helpers in `core/bifrost.go`. After all pre-request hooks complete, `handleRequest` and `handleStreamRequest` validate the resolved provider against the allowlist and prune fallbacks to only allowed providers. A non-allowed primary provider returns HTTP 400.
- The governance plugin now publishes the VK's `provider_configs` providers to `BifrostContextKeyRoutingAllowedProviders` during `PreRequestHook`, covering the case where governance cannot pick a provider itself but still needs to constrain downstream layers.
- The model catalog resolver intersects its catalog candidates with the allowlist (when set) before selecting a provider, emitting routing-engine observability logs that explain which candidates were excluded and why. Returns `("", nil)` when the allowlist excludes all candidates.
- Documents the two-level enforcement model, context key semantics, and custom plugin usage in `docs/providers/provider-routing.mdx`.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
```sh
go test ./...
```
- Configure a Virtual Key with `provider_configs` restricted to a specific provider (e.g., `openai`).
- Send a request with an explicit `provider/model` prefix targeting a non-allowed provider (e.g., `anthropic/claude-3`). Expect HTTP 400 with a message indicating the provider is not permitted.
- Send a request with a model resolvable by the catalog to multiple providers where some are excluded by the VK allowlist. Verify routing-engine logs show the excluded candidates and the request routes only to an allowed provider.
- Set `BifrostContextKeyRoutingAllowedProviders` to an empty slice from a custom plugin and confirm the request fails closed with HTTP 400.
- Verify fallbacks targeting non-allowed providers are silently pruned and do not appear in the fallback chain.
## Breaking changes
- [x] Yes
- [ ] No
Requests that previously succeeded by specifying an explicit `provider/model` prefix that bypassed a VK's `provider_configs` restrictions will now be rejected with HTTP 400. Any fallbacks targeting providers outside the VK's allowed set will be silently removed from the fallback chain.
## Security considerations
This change strengthens provider-level access control enforced by Virtual Keys. Without this, a user or plugin could bypass governance-imposed provider restrictions by specifying an explicit provider prefix or by relying on a downstream routing layer to select a non-permitted provider. The hard enforcement in core ensures the allowlist is a guarantee rather than a best-effort constraint.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
## New Features
* Added provider routing allowlist enforcement: configured allowed providers now restrict request routing across primary and fallback options; requests using non-permitted primary providers return HTTP 400, and fallback providers are automatically filtered to only permitted options.
## Documentation
* Added routing allowlist enforcement documentation covering enforcement mechanisms and fail-closed behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* refactor: promote `KeyAliases` value type from `string` to `AliasConfig` with legacy wire-shape compatibility (#4180)
## Summary
`KeyAliases` previously mapped user-facing model names to plain strings (`map[string]string`). This PR promotes the value type to a rich `AliasConfig` struct, enabling per-alias metadata (`ModelName`, `ModelFamily`) and provider-specific overrides (`AzureAliasCfg`, `VertexAliasCfg`, `BedrockAliasCfg`, `ReplicateAliasCfg`) to be expressed directly on an alias entry rather than inferred from the wire model ID or duplicated at the key level.
## Changes
- `KeyAliases` is now `map[string]AliasConfig` instead of `map[string]string`. `AliasConfig` carries `ModelID` (the wire identifier), optional `ModelName`, `ModelFamily` (a typed enum for routing decisions), `Description`, `Region`, and embedded provider sub-configs for Azure, Vertex, Bedrock, and Replicate.
- `AliasConfig.MarshalJSON` emits the legacy `{"k":"v"}` string-valued wire shape when only `ModelID` is set, preserving byte-for-byte JSON compatibility with pre-refactor consumers and keeping `config_hash` stable for unenriched entries.
- `KeyAliases.UnmarshalJSON` transparently accepts both the legacy string shape and the new object shape, promoting legacy string values to `AliasConfig{ModelID: <string>}`.
- `KeyAliases.Resolve` is preserved for backward compatibility. A new `ResolveConfig` method returns the full `AliasConfig` for callers that need more than the wire model string.
- `KeyAliases.Validate` is extended to check `ModelName` whitespace and `ModelFamily` validity.
- `ModelFamily` is introduced as a typed enum (`anthropic`, `openai`, `mistral`, `cohere`, `gemini`, `nova`, `titan`) with an `IsValid` method, enabling provider routing decisions without substring-sniffing the wire model ID.
- All provider `ToBifrostListModelsResponse` and `ListModelsPipeline` call sites are updated to use `schemas.KeyAliases` and access `.ModelID` from `AliasConfig` values.
- The JSON schema (`config.schema.json`) is updated so the `aliases` property accepts either the legacy string shape or the new object shape via `oneOf`.
- New tests cover legacy/rich/mixed unmarshal, round-trip marshal stability, `Resolve`/`ResolveConfig` behavior, `Validate` error cases, `ModelFamily.IsValid`, DB persistence of both wire shapes, and hash stability guarantees.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/schemas/...
go test ./core/providers/...
go test ./framework/configstore/...
go test ./transports/bifrost-http/lib/...
```
Existing configs using the legacy `"alias": "model-id"` string shape require no changes — they deserialize and re-serialize identically. To opt into the rich shape, update an alias entry to the object form:
```json
"aliases": {
"my-model": {
"model_id": "azure-deployment-xyz",
"model_family": "anthropic",
"model_name": "claude-3-5-sonnet",
"api_version": "2024-08-01-preview"
}
}
```
## Breaking changes
- [x] Yes
- [ ] No
Any code that directly reads `KeyAliases` values as `string` (e.g. `aliases["key"]` expecting a `string`) must be updated to access `.ModelID` on the returned `AliasConfig`. The JSON wire format for unenriched aliases is unchanged. The `Resolve(model string) string` method signature is unchanged.
## Related issues
## Security considerations
No new secrets or auth surfaces are introduced. Provider sub-config fields that accept `EnvVar` values follow the existing env-var resolution and encryption patterns already in place for key-level configs.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: add per-alias Azure endpoint, API version, and Anthropic version overrides with context-aware `ResolveFamily` (#4181)
## Summary
This PR introduces per-alias Azure overrides for endpoint, API version, and Anthropic version, and adds a `ResolvedAlias` context value that providers can read to determine model family routing. Previously, model family detection relied solely on substring matching against the wire model ID, which broke when Azure deployment IDs were opaque strings (e.g., `12345-azure-deployment`) even if the alias key or config clearly indicated the model family (e.g., `best-claude`). This change fixes that by walking a precedence chain — explicit `ModelFamily` field → `ModelName` → `ModelID` → alias key — before falling back to substring matching.
## Changes
- Introduced `ResolvedAlias` struct and `BifrostContextKeyResolvedAlias` context key; `bifrost.go` now stashes the full `AliasConfig` (and the user-facing alias key) into context after each key-level alias resolution, for both streaming and non-streaming paths.
- Added `ResolveFamily`, `IsAnthropicModelFamily`, and `GetResolvedAlias` helpers in `schemas/account.go` that walk the alias precedence chain for model family detection. All Azure provider call sites that previously called `schemas.IsAnthropicModel(model)` now call `schemas.IsAnthropicModelFamily(ctx, model)` or `schemas.ResolveFamily(ctx, model)`.
- Added `AzureAliasCfg` fields (`APIVersion`, `AnthropicVersion`, `Endpoint`) and three resolver helpers (`resolveAzureEndpoint`, `resolveAPIVersion`, `resolveAnthropicVersion`) in `core/providers/azure/utils.go`. All Azure provider methods now use these helpers instead of reading `key.AzureKeyConfig.Endpoint.GetValue()` directly, enabling per-alias endpoint and version overrides.
- `buildPassthroughURL` now returns an error when the endpoint is empty and accepts a `*BifrostContext` to apply alias-level `api-version` overrides on passthrough routes.
- `buildContainerURL` now accepts a `*BifrostContext` for the same reason.
- `KeyAliases.UnmarshalJSON` switched from `encoding/json` to `sonic` for consistency with the rest of the codebase.
- Added `KeyAliases.ResolveConfig` (returns `*AliasConfig`) used by `bifrost.go` to populate `ResolvedAlias`; the existing `Resolve` (returns string) is preserved for backward compatibility.
## Type of change
- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/... ./core/providers/azure/... ./core/schemas/...
```
Key scenarios to validate:
1. **Opaque Azure deployment alias** — configure an alias like `best-claude → { model_id: "12345-deployment", azure_alias_cfg: {} }` with alias key containing "claude"; verify the request is routed through the Anthropic path (correct URL, `anthropic-version` header set).
2. **Alias-level endpoint override** — configure two aliases pointing to different Azure cognitive-services resources under the same key; verify each request hits the correct endpoint.
3. **Alias-level `api_version` override** — configure an alias with `azure_alias_cfg.api_version: "2024-10-21"`; verify the `api-version` query parameter on `/openai/deployments/` and `/openai/v1/responses` routes uses the override rather than the route default.
4. **Caller-supplied `api-version` wins** — pass `api-version` explicitly in a passthrough `rawQuery`; verify the alias override does not overwrite it.
5. **No alias matched** — verify existing substring-based family detection is unchanged.
## Breaking changes
- [ ] Yes
- [x] No
`buildPassthroughURL` now returns `(string, error)` instead of `string`. This is an internal method on `AzureProvider` and is not part of any exported interface.
## Related issues
## Security considerations
The `ResolvedAlias` value stored in `BifrostContext` is set exclusively by the core request worker and is documented as read-only for plugins. Alias-level endpoint overrides are resolved from `EnvVar` (supporting environment variable indirection), consistent with how key-level endpoints are handled, so secrets are not inlined in config.
## Checklist
- [x] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: add alias-level region/ARN overrides and context-aware model family resolution for Bedrock (#4182)
## Summary
Model-family detection on the Bedrock provider previously relied on substring matching against the raw model string. This meant that aliases pointing to opaque Bedrock deployments (e.g., inference profiles or cross-region ARNs) could not be correctly routed to the right request/response shape. This PR threads `*BifrostContext` through all family-detection call sites so that the resolved alias — including its explicit `ModelFamily`, `Region`, and `BedrockAliasCfg.InferenceProfileARN` — takes precedence over substring heuristics.
## Changes
- Replaced all calls to `schemas.IsAnthropicModel`, `IsMistralModel`, `IsNovaModel`, `IsLlamaModel`, `IsCohereModel` (bare string matchers) with new context-aware variants: `IsAnthropicModelFamily`, `IsMistralModelFamily`, `IsNovaModelFamily`, `IsLlamaModelFamily`, `IsCohereModelFamily`, `IsTitanModelFamily`. These consult `ResolveFamily(ctx, model)` first, which reads the alias family tag before falling back to substring detection.
- Added `ModelFamilyLlama`, `ModelFamilyTitan`, and `ModelFamilyCohere` to the `ModelFamily` enum and wired them into `ResolveFamily` so alias-tagged models route correctly.
- Added `IsCohereModel` and `IsTitanModel` substring helpers used as the final fallback inside `ResolveFamily`.
- Introduced `resolveBedrockARN(ctx, key)` to resolve the inference-profile ARN with priority: alias-level `BedrockAliasCfg.InferenceProfileARN` > key-level `BedrockKeyConfig.ARN`. Removed the inline ARN lookup from `getModelPathAndRegion`.
- Updated `resolveBedrockRegion` and `getModelPathAndRegion` to accept `*BifrostContext` and honor the alias-level `Region` override between the model-string prefix (highest) and the key-level region (lower).
- Propagated `*BifrostContext` into `DetermineEmbeddingModelType` and `ToBedrockEmbeddingInvokeResponse` so embedding model routing uses the same family resolution path.
- Changed `convertToolConfigFromFiltered` to accept `*schemas.BifrostContext` instead of `context.Context` so family gates inside it can read the resolved alias.
- Added nil-guard checks on `BifrostContext` in `bedrockAliasToolName` and `bedrockRestoreToolName`.
- Added tests covering alias-level `Region` and `InferenceProfileARN` override priority in `region_test.go`.
## Type of change
- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/providers/bedrock/... ./core/schemas/...
```
To validate alias-level overrides end-to-end, configure an alias with an explicit `ModelFamily`, `Region`, or `BedrockAliasCfg.InferenceProfileARN` and confirm that:
- Requests are signed and routed to the alias-specified region rather than the key-level region.
- The ARN is prepended to the model path when `InferenceProfileARN` is set on the alias.
- Embedding requests to an alias tagged `cohere` or `titan` use the correct request/response envelope regardless of the wire model string.
## Breaking changes
- [x] Yes
- [ ] No
`ToBedrockEmbeddingInvokeResponse` now requires a `*schemas.BifrostContext` as its first argument. Any external callers of this function must be updated to pass the context. The `convertToolConfigFromFiltered` signature changed from `context.Context` to `*schemas.BifrostContext`.
## Related issues
## Security considerations
No new secrets or auth surfaces introduced. ARN and region values continue to flow through the existing `EnvVar`/`GetValue()` resolution path.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: add per-alias Vertex project/region overrides and context-aware model family resolution (#4183)
## Summary
Introduces per-alias overrides for Vertex AI's `project_id`, `project_number`, and `region` configuration values, allowing a single Vertex credential to serve requests across multiple GCP projects and regions. This is particularly useful when different model families (e.g., Anthropic Claude on `us-east5`, Gemini on `us-central1`) are deployed in separate GCP projects or regions.
## Changes
- Added `resolveVertexProjectID`, `resolveVertexProjectNumber`, and `resolveVertexRegion` helper functions in `utils.go` that check for alias-level overrides in `BifrostContext` before falling back to key-level configuration values.
- Replaced all direct `key.VertexKeyConfig.*` field accesses throughout the Vertex provider with calls to these resolver functions, covering all operations: chat completion, streaming, embeddings, responses, cached content, image/video generation, reranking, token counting, and passthrough.
- Replaced model-family detection calls (`IsAnthropicModel`, `IsGeminiModel`, `IsGemmaModel`, `IsImagenModel`, `IsVeoModel`, `IsMistralModel`) with context-aware variants (`IsAnthropicModelFamily`, `IsGeminiModelFamily`, etc.) so that alias-level `ModelFamily` overrides are respected when routing requests to the correct Vertex endpoint and request format.
- Added `ModelFamilyGemma`, `ModelFamilyImagen`, and `ModelFamilyVeo` as first-class `ModelFamily` constants and registered them in `ResolveFamily`, with Imagen and Veo checked before Gemini to avoid substring-match conflicts.
- Added context-aware `IsGeminiModelFamily`, `IsGemmaModelFamily`, `IsImagenModelFamily`, and `IsVeoModelFamily` helpers to `account.go`.
- Added unit tests covering alias override precedence, empty-alias fallthrough to key-level values, and nil-context handling for all three resolver functions.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/providers/vertex/... -run TestResolveVertex
go test ./core/...
```
Configure a Vertex key alias with `VertexAliasCfg.ProjectID` or `AliasConfig.Region` set to a value different from the key-level config and verify that requests are routed to the alias-specified project/region rather than the key-level defaults.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
No new secrets or auth mechanisms are introduced. Alias-level project/region values follow the same `EnvVar` resolution path as key-level values, so secrets can still be sourced from environment variables rather than being hardcoded.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **New Features**
* Extended support for additional Google Vertex AI model families.
* Added flexible configuration resolution with alias-level parameter overrides for the Vertex provider.
* **Tests**
* Added comprehensive tests validating configuration override behavior for Vertex parameters.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* feat: add per-alias `ReplicateAliasCfg.UseDeploymentsEndpoint` override with provider-scoped alias sub-config validation (#4184)
## Summary
Adds per-alias control over whether Replicate requests are routed to the deployments endpoint or the predictions endpoint. Previously this was only configurable at the key level via `ReplicateKeyConfig.UseDeploymentsEndpoint`. This change introduces a `ReplicateAliasCfg.UseDeploymentsEndpoint` field that, when set, takes precedence over the key-level setting — allowing a single Replicate API token to route some aliases through the deployments endpoint (e.g. production-pinned models) while others use the predictions endpoint (e.g. experimental versioned models).
Additionally, `KeyAliases.Validate` now accepts the owning key's provider and rejects provider-specific sub-configs (Azure, Vertex, Bedrock, Replicate) when attached to a key belonging to a different provider.
## Changes
- `useDeploymentsEndpoint` now accepts a `*schemas.BifrostContext` and checks for a `ReplicateAliasCfg.UseDeploymentsEndpoint` override on the resolved alias before falling back to the key-level config. All call sites updated accordingly.
- `KeyAliases.Validate` signature changed from `Validate()` to `Validate(providerKey ModelProvider)`. It now returns an error if a provider-specific alias sub-config (e.g. `AzureAliasCfg`) is attached to a key that does not belong to that provider.
- Alias validation in `processProvider`, `processAuthoritativeProvider`, and the HTTP handler create/update paths now resolves the effective base provider (accounting for custom provider configs) before calling `Validate`.
- The redundant `Validate` call in the GORM `BeforeSave` hook was removed since validation is enforced at the handler layer.
- Tests added for `useDeploymentsEndpoint` covering nil context, key-level fallback, alias override true/false, and alias present but without a `ReplicateAliasCfg`. Existing `TestKeyAliasesValidate` extended with provider-mismatch cases for all four provider sub-configs.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/providers/replicate/...
go test ./core/schemas/...
go test ./transports/bifrost-http/...
```
To exercise the alias override end-to-end, configure a Replicate key with `use_deployments_endpoint: false` at the key level, then define an alias with `replicate_alias_cfg.use_deployments_endpoint: true`. Requests routed through that alias should target the deployments endpoint while requests without the alias continue to use the predictions endpoint.
To verify provider-mismatch validation, attach a `replicate_alias_cfg` to an Azure key in the config file or via the HTTP API and confirm a `400 Bad Request` is returned with a descriptive error.
## Breaking changes
- [x] Yes
`KeyAliases.Validate()` now requires a `ModelProvider` argument. Any code calling `Validate()` directly must be updated to pass the owning key's provider.
## Security considerations
None beyond standard input validation. The new provider-mismatch check prevents misconfigured aliases from silently routing requests to unintended endpoints.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: replace flat aliases table with rich `DeploymentsTable` supporting per-deployment model family, canonical name, and provider overrides (#4185)
## Summary
Replaces the flat "Aliases" key-value table (mapping request model name → string deployment ID) with a richer "Deployments" table that supports per-deployment metadata and provider-specific overrides. This enables cost/pricing logs and family-based routing to work correctly for custom deployments.
## Changes
- Introduced a new `DeploymentsTable` component that renders each deployment as a collapsible row. Expanding a row exposes fields for canonical model name, model family, description, and provider-specific overrides (Azure API version, endpoint, Anthropic version; Vertex project ID/number/region; Bedrock region and inference profile ARN; Replicate deployments endpoint toggle).
- Replaced the `normalizeAliasesValue` helper and `HeadersTable`-based aliases editor in `apiKeysFormFragment.tsx` with the new `DeploymentsTable`. The form label and description were updated from "Aliases" to "Deployments" to reflect the richer semantics.
- Added `AliasConfig` and `ModelFamily` types to `config.ts`, mirroring the Go `schemas.AliasConfig` struct (with embedded provider sub-configs flattened to top-level fields on the wire).
- Added `aliasConfigSchema` and `modelFamilySchema` Zod schemas to `schemas.ts`. The alias schema uses `z.preprocess` to accept the legacy `string` wire shape emitted by the Go server for simple aliases, coercing it to `{ model_id: string }` so hydrated state passes validation without a migration.
- Updated `KeySchema` in `providerForm.ts` to use `z.record(z.string(), aliasConfigSchema)` and updated the validation error message to reflect the new requirement.
- Rewrote `isValidAliases` in `validation.ts` to validate the rich `Record<string, { model_id?: string }>` shape, checking that every entry has a non-empty deployment name and a non-empty `model_id`.
- Updated `ModelProviderKey` in `config.ts` to type `aliases` as `Record<string, AliasConfig>` instead of `Record<string, string>`.
- The `DeploymentsTable` includes a draft row at the bottom for adding new entries. The draft is committed automatically when both the deployment name and model ID are filled. Rename collision detection is case-insensitive and stable row IDs are used to preserve expanded/pending state across renames.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [x] UI (React)
- [ ] Docs
## How to test
```sh
cd ui
pnpm i
pnpm build
```
1. Navigate to a provider key configuration form.
2. Verify the "Deployments" table renders in place of the old "Aliases" table.
3. Add a new deployment by filling in the deployment name and model ID in the draft row — confirm it commits automatically when both fields are populated.
4. Expand a committed row and verify the canonical model name, model family, description, and provider-specific override fields are visible and editable.
5. For Azure, Vertex, Bedrock, and Replicate providers, confirm the correct provider-specific section appears in the expanded panel.
6. Rename a deployment to an existing name and confirm the inline collision error appears and the row is not committed.
7. Load an existing config that uses the legacy `Record<string, string>` alias format and confirm it hydrates correctly into the new table without validation errors.
## Screenshots/Recordings
Before: A simple two-column key/value table labeled "Aliases" with a plain text input for the deployment ID.
After: A collapsible table labeled "Deployments" where each row can be expanded to reveal canonical model name, model family, description, and provider-specific override fields.
## Breaking changes
- [x] Yes
- [ ] No
The `aliases` field type changes from `Record<string, string>` to `Record<string, AliasConfig>` in the UI type system and form schema. Existing configs using the legacy string format are handled transparently via the `aliasConfigSchema` preprocessor and the `normalize` function in `DeploymentsTable`, so no manual migration is required for stored configs. Any code outside this diff that directly constructs or reads `ModelProviderKey.aliases` as `Record<string, string>` will need to be updated.
## Related issues
## Security considerations
No new secrets or auth surfaces introduced. Provider-specific override fields (endpoint, credentials) use the existing `EnvVarInput` component, which supports environment variable references and redaction consistent with the rest of the form.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: add `RoutingInfo` to response/error extra fields with fallback and alias resolution context (#4186)
## Summary
Introduces a structured `RoutingInfo` field on both `BifrostResponseExtraFields` and `BifrostErrorExtraFields` that exposes per-attempt routing context (provider, model, key used, resolved alias, and fallback signals) in a single, well-typed struct. The existing `Provider`, `OriginalModelRequested`, and `ResolvedModelUsed` fields are preserved for backward compatibility but deprecated in favour of `RoutingInfo`.
## Changes
- Added `RoutingInfo` and `ResolvedKeyAlias` schema types capturing the provider, model, key name, resolved alias metadata, fallback flag, and primary provider/model when a fallback occurred.
- Added `BuildRoutingInfo` helper in `account.go` that constructs a per-attempt `RoutingInfo` from the current context, provider, model, and key, including any resolved alias config.
- Added `PopulateRoutingInfo` on `BifrostResponse` and `BifrostError` to stamp `RoutingInfo` onto responses/errors and keep the deprecated triplet in sync via the shared `syncDeprecatedFromRoutingInfo` helper.
- Added `SetFallbackRoutingInfo` on both types, called by the orchestrator (`handleRequest` / `handleStreamRequest`) to layer on `IsFallback`, `PrimaryProvider`, and `PrimaryModel` after a fallback attempt completes. These signals are intentionally set at the orchestrator scope rather than inside per-attempt code.
- `requestWorker` now seeds `attemptRoutingInfo` with the known provider/model before the retry loop so that early failures (e.g. key selection errors) still produce a populated `RoutingInfo` on the error. Each retry iteration snapshots a `perAttemptRoutingInfo` to avoid races in async streaming closures.
- `PopulateRoutingInfo` is called alongside `PopulateExtraFields` both before and after `RunPostLLMHooks`, ensuring plugin modifications cannot corrupt routing metadata.
- `ProcessedStreamResponse` gains a `RoutingInfo` field to carry routing context through the streaming pipeline.
- The deprecated `Provider`, `OriginalModelRequested`, and `ResolvedModelUsed` fields are annotated with deprecation notices and derivation rules pointing consumers to the equivalent `RoutingInfo` paths.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go version
go test ./...
```
Verify that responses and errors include a populated `routing_info` object in their JSON output. For fallback scenarios, confirm `is_fallback` is `true` and `primary_provider`/`primary_model` reflect the original attempt. Confirm that `provider`, `original_model_requested`, and `resolved_model_used` continue to be populated with the same values as before.
## Breaking changes
- [ ] Yes
- [x] No
Existing fields are preserved. `RoutingInfo` is additive.
## Related issues
## Security considerations
None. No auth, secrets, or PII are introduced. `RoutingInfo` surfaces key names already present in existing fields.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* refactor: replace `resolvePricing` flat args with `RoutingInfo` and add backward-compat fallback for legacy `ExtraFields` (#4187)
## Summary
Replaces the flat `(provider, originalModel, resolvedModel)` triplet passed into `resolvePricing` with a structured `schemas.RoutingInfo` value. This aligns pricing lookups with the routing context that `core.bifrost` already populates on every response, and introduces a well-defined lookup precedence: `AliasModelName → AliasModelID → ModelName`, with overrides keyed by the wire model identifier.
## Changes
- `resolvePricing` now accepts a single `schemas.RoutingInfo` argument instead of three separate string parameters. The lookup iterates over `[AliasModelName, AliasModelID, ModelName]`, stopping at the first catalog hit, and applies overrides keyed by the wire model (`AliasModelID` when an alias matched, otherwise `ModelName`).
- `calculateBaseCost` reads `RoutingInfo` directly from `ExtraFields`. A backward-compatibility fallback synthesises a `RoutingInfo` from the deprecated `Provider`/`OriginalModelRequested`/`ResolvedModelUsed` triplet only when `RoutingInfo` is fully unset (zero `Provider`, zero `Model`, nil `ResolvedKeyAlias`). Partial population is trusted as-is to prevent false-positive fallbacks.
- `computeCacheEmbeddingCost` constructs a minimal `RoutingInfo` from the cache-debug fields, since no alias resolution context exists for cache-replayed requests.
- Container pricing overrides build a synthetic `RoutingInfo` that pins both model fields to the container identifier, preserving per-container override addressability.
- All call sites in tests are updated to pass `schemas.RoutingInfo` structs directly.
- New backward-compat tests cover: legacy-fields-only (no alias), legacy-fields-only (with alias/resolved model), `RoutingInfo` winning over legacy fields when both are set, both empty returning zero cost, and partial `RoutingInfo` suppressing the fallback.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./framework/modelcatalog/...
```
Key scenarios to verify:
- Pricing resolves correctly for a non-aliased request via `RoutingInfo.Model`.
- Pricing resolves via `AliasModelID` when an alias was matched and the wire model differs from the caller-facing name.
- Pricing resolves via `AliasModelName` when the admin tagged a canonical name on the alias.
- Legacy callers with only `Provider`/`OriginalModelRequested`/`ResolvedModelUsed` populated still receive correct costs.
- When both `RoutingInfo` and the deprecated triplet are set, `RoutingInfo` wins.
- Partial `RoutingInfo` (e.g. `Model` set but `Provider` empty) does not trigger the legacy fallback.
## Breaking changes
- [ ] Yes
- [x] No
The `resolvePricing` method is unexported. The public `CalculateCost` API is unchanged. The backward-compat fallback ensures existing callers writing only the deprecated `ExtraFields` triplet continue to receive correct cost calculations.
## Related issues
## Security considerations
None. This change affects cost accounting logic only; no auth, secrets, or PII are involved.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* docs: expand alias object schema, `routing_info` response shape, and pricing lookup precedence (#4188)
## Summary
Documents the rich object form for alias values, the new `routing_info` response block, and the pricing lookup precedence that uses canonical `model_name` to resolve opaque deployment IDs against the catalog.
## Changes
- Expanded the alias schema documentation to cover the object form alongside the existing plain-string shorthand. The object form accepts `model_id`, `model_name`, `model_family`, `description`, `region`, and provider-specific overrides (`api_version`, `anthropic_version`, `endpoint` for Azure; `project_id`, `project_number` for Vertex; `inference_profile_arn` for Bedrock; `use_deployments_endpoint` for Replicate).
- Added a validation rule documenting that provider-specific sub-config fields are rejected when the owning key belongs to a different provider.
- Replaced the flat `extra_fields` response fields (`original_model_requested`, `resolved_model_used`, `provider`) with the new `routing_info` block. The old fields are noted as deprecated but still populated for backward compatibility.
- Added a `routing_info` field reference table covering `provider`, `model`, `key`, `resolved_key_alias`, `is_fallback`, `primary_provider`, and `primary_model`.
- Added a pricing lookup precedence section explaining the three-candidate resolution order (`model_name` → `model_id` → caller-sent model) and how it solves the opaque deployment ID problem for cost attribution.
- Added a cross-reference note in the Azure provider config page pointing readers to the full alias object schema.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Review the rendered docs for:
- Correct table formatting for the alias object schema and provider-specific overrides
- The `routing_info` JSON example rendering properly
- The pricing lookup precedence section appearing between the wildcard patterns section and the request type filtering section
- The deprecation `<Note>` rendering in the aliasing-models page
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
None.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: add `keyconfig.Store` for per-key config aggregation with allow/block/alias views (#4189)
## Summary
Introduces a new `keyconfig` package under `framework/modelcatalog` that provides a thread-safe, in-memory store for per-key configuration (allowed models, blacklisted models, and aliases) across all configured providers. This centralizes the aggregation logic previously scattered in the load balancer plugin, making it available as a pure transformation layer for routing-time queries.
## Changes
- Added `framework/modelcatalog/keyconfig/store.go` implementing a `Store` type that:
- Maintains an immutable `providerState` snapshot per provider, swapped atomically under a write lock so readers never observe torn state
- Aggregates allowed models as the union of enabled keys' `Models` fields minus per-key blacklisted entries, collapsing to `["*"]` when any enabled key is unrestricted
- Computes the provider-level blacklist as the intersection across enabled keys (a model is only provider-blocked when every enabled key blacklists it)
- Builds a case-insensitive alias index keyed by lowercase alias name, with last-enabled-key-wins collision resolution and a debug log on collision
- Treats keyless non-standard (custom) providers as unrestricted (`["*"]`) to support ambient/IAM auth flows
- Drops providers from the store entirely when they have no routable keys, keeping the store focused on routing-time queries rather than full config inspection
- Exposes `Replace` (full atomic resync), `SetProvider` (single-provider update), `RemoveProvider`, `EntriesFor`, `EntryFor`, `AllowedFor`, `BlacklistedFor`, `IsAllowed`, `ResolveAlias`, `Providers`, and `KeysAllowingModel`
- Added `framework/modelcatalog/keyconfig/store_test.go` with comprehensive behavioral tests covering: wildcard and explicit allow lists, blacklist intersection, disabled keys, block-all keys, alias ownership and collision, case-insensitive blacklist and alias normalization, atomic snapshot correctness under concurrent reads, and defensive copy guarantees
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./framework/modelcatalog/keyconfig/...
```
The test suite covers all aggregation semantics, concurrency safety (atomic snapshot test with 200 Replace cycles and a concurrent reader), and edge cases including nil logger, keyless non-standard providers, and case-insensitive alias collision detection.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
The store holds API key IDs and model allow/block lists in memory. No secrets (key values) are stored — only key IDs and routing metadata. The alias index is keyed by lowercase model name; alias configs may contain region or deployment override fields that are treated as read-only by callers.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: add `live` model catalog cache store with filtered/unfiltered entry support (#4190)
## Summary
Introduces a thread-safe, in-memory cache (`live.Store`) for provider model catalog responses. The store holds per-`(provider, keyID, unfiltered)` entries and is intentionally passive — it never initiates network calls. Callers are responsible for fetching and pushing results via `Upsert`, keeping the cache decoupled from transport concerns.
## Changes
- Added `framework/modelcatalog/live/store.go` with a `Store` type that caches `/v1/models` responses keyed by provider, key ID, and a filtered/unfiltered flag.
- Filtered entries are pre-gated by the provider's `ListModelsPipeline` at write time; callers reading filtered entries must not reapply that gate to avoid dropping alias-backfill rows.
- `ModelsForProvider` and `UnfilteredModelsForProvider` return the sorted, deduplicated union across all matching keys for a provider.
- `Invalidate` drops both filtered and unfiltered entries for a single key (e.g., on credential rotation or key deletion). `InvalidateProvider` drops all entries for a provider (e.g., on provider deletion).
- `Snapshot` returns a full defensive copy of the store for diagnostics.
- Both `Upsert` and `Snapshot` copy slices to prevent external mutation of cached state.
- Keyless providers (Vertex workload identity, Bedrock IAM, etc.) are supported via an empty `KeyID`.
- Added `framework/modelcatalog/live/store_test.go` covering union across keys, filtered/unfiltered isolation, invalidation behavior, defensive copying, overwrite semantics, and keyless provider handling.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./framework/modelcatalog/live/...
```
All tests should pass. Key scenarios covered:
- Filtered and unfiltered entries for the same key do not bleed into each other.
- Union across multiple keys for the same provider is deduplicated and sorted.
- `Invalidate` removes both filtered and unfiltered entries for the target key while leaving other keys intact.
- `InvalidateProvider` removes all entries for the target provider while leaving other providers intact.
- Mutating the input slice after `Upsert`, or mutating a `Snapshot`, does not affect store state.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
Cached model lists may include key IDs as cache discriminators. The store holds no credential values — only the key ID string used as a lookup discriminator. No PII or secrets are stored.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
* feat: adds datasheet store to modelcatalog (#4191)
## Summary
Briefly explain the purpose of this PR and the problem it solves.
## Changes
- What was changed and why
- Any notable design decisions or trade-offs
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## …
This was referenced Jun 13, 2026
Merged
Open
18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a new
PreRequestHookphase to theLLMPlugininterface. This hook runs exactly once per top-level request — afterHTTPTransportPreHookand beforePreLLMHook— and is the canonical place for plugins to resolve provider, model, and fallback routing decisions. Previously, routing logic had to be shoehorned intoPreLLMHook, which runs on every fallback attempt and whose mutations have incidental cross-fallback visibility.PreRequestHookmutations are committed to the shared*BifrostRequestbefore any fan-out and are observed by every subsequent plugin, everyPreLLMHookinvocation, the provider call, and every fallback.As part of this change, the
filterProvidersByContexthelper (used inListAllModels) is removed, and request validation is moved to afterPreRequestHookruns so that plugins have the opportunity to populate provider/model before the empty-field check fires. Error messages for missing provider/model are updated to reflect that auto-resolution was attempted.Changes
PreRequestHook(ctx *BifrostContext, req *BifrostRequest) errorto theLLMPlugininterface with non-blocking error semantics (logged as warning, pipeline continues).RunPreRequestHookstoPluginPipeline, executing the hook in registration order once per request with tracing and plugin-scope isolation.RunPreRequestHooksas a public method onBifrostfor callers (e.g., realtime WebSocket handlers) that bypass the normal inference path.validateRequestto afterPreRequestHookexecution in bothhandleRequestandhandleStreamRequest, renamed tovalidateRequestAfterPreRequestHookswith updated error messages.handleStreamRequestto matchhandleRequestbehavior.filterProvidersByContextand its tests fromListAllModels.DynamicPlugin(shared-object loader) to optionally loadPreRequestHookfrom.soplugins; legacy plugins without the export get a no-op passthrough, preserving backward compatibility.AsLLMPluginto recognizepreRequestHookas sufficient to qualify aDynamicPluginas anLLMPlugin.PreRequestHookimplementations to all existing plugins (compat,governance,jsonparser,logging,maxim,mocker,prompts,semanticcache,telemetry) and all example/test plugins to satisfy the updated interface.plugin.goto describe per-request vs. per-attempt semantics.Type of change
Affected areas
How to test
go test ./...Validate that:
PreRequestHookcan mutatereq.Providerandreq.Modelbefore the provider call, and those mutations are visible to subsequent plugins and fallback attempts.PreRequestHookdoes not abort the request; the pipeline continues to the next plugin and a warning is logged.PreRequestHookimplementations behave identically to before..soplugins that do not exportPreRequestHookload and function correctly with the no-op passthrough."could not auto resolve a provider for the request, please specify a provider explicitly".Breaking changes
The
LLMPlugininterface gains a new required methodPreRequestHook. Any external plugin implementingLLMPluginmust add aPreRequestHookmethod. Plugins that do not participate in routing should returnnil. Shared-object (.so) plugins are exempt — the loader treatsPreRequestHookas optional and provides a no-op default.Security considerations
PreRequestHookruns withBlockRestrictedWritesactive on the context (same asRunLLMPreHooks), preventing plugins from writing to restricted context keys during the hook. Plugins cannot abort or gate requests via error return from this hook; authorization and content-policy enforcement must remain inHTTPTransportPreHookor via a short-circuit inPreLLMHook.Checklist
docs/contributing/README.mdand followed the guidelinesSummary by CodeRabbit
Release Notes
New Features
PreRequestHookplugin phase enabling plugins to perform per-request routing decisions before provider and model validation.Refactor