feat(pricing): implement scoped pricing overrides#1825
Merged
Pratham-Mishra04 merged 2 commits intov1.5.0from Mar 22, 2026
Merged
feat(pricing): implement scoped pricing overrides#1825Pratham-Mishra04 merged 2 commits intov1.5.0from
Pratham-Mishra04 merged 2 commits intov1.5.0from
Conversation
Contributor
|
Caution Review failedThe pull request is closed. Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (3)
📒 Files selected for processing (61)
📝 WalkthroughSummary by CodeRabbit
WalkthroughMoves provider-scoped pricing overrides into a governance-scoped model with CRUD APIs, a new Changes
Sequence Diagram(s)sequenceDiagram
rect rgba(220,240,220,0.5)
participant Client
end
rect rgba(200,220,240,0.5)
participant GovernanceHandler
participant ConfigStore
end
rect rgba(240,220,220,0.5)
participant ModelCatalog
end
Client->>GovernanceHandler: POST /api/governance/pricing-overrides (body)
GovernanceHandler->>GovernanceHandler: validate & normalize payload
GovernanceHandler->>ConfigStore: CreatePricingOverride(override)
ConfigStore-->>GovernanceHandler: created override
GovernanceHandler->>ModelCatalog: UpsertPricingOverrides(override)
ModelCatalog-->>ModelCatalog: update rawOverrides and rebuild lookup
GovernanceHandler-->>Client: 201 Created
sequenceDiagram
rect rgba(220,240,220,0.5)
participant Requestor
end
rect rgba(200,220,240,0.5)
participant PricingManager
participant ModelCatalog
end
rect rgba(240,220,220,0.5)
participant Storage
end
Requestor->>PricingManager: CalculateCost(response, scopes)
PricingManager->>ModelCatalog: resolvePricing(model, provider, deployment, reqType, scopes)
ModelCatalog->>ModelCatalog: lookup override by scope priority & pattern
alt match found
ModelCatalog-->>PricingManager: patched pricing
else no match
ModelCatalog-->>PricingManager: base pricing (or override-only row)
end
PricingManager-->>Requestor: computed cost
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
12 tasks
jerkeyray
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
transports/bifrost-http/handlers/governance.go (1)
52-63:⚠️ Potential issue | 🟠 MajorFail fast if
modelCatalogis missing.
modelCatalogis now a constructor dependency for governance pricing override flows; allowing nil here risks runtime panics later. Add an explicit constructor guard.💡 Suggested patch
func NewGovernanceHandler(manager GovernanceManager, configStore configstore.ConfigStore, modelCatalog *modelcatalog.ModelCatalog) (*GovernanceHandler, error) { if manager == nil { return nil, fmt.Errorf("governance manager is required") } if configStore == nil { return nil, fmt.Errorf("config store is required") } + if modelCatalog == nil { + return nil, fmt.Errorf("model catalog is required") + } return &GovernanceHandler{ governanceManager: manager, configStore: configStore, modelCatalog: modelCatalog, }, nil }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 52 - 63, NewGovernanceHandler currently allows a nil modelCatalog which can cause runtime panics; update the constructor (NewGovernanceHandler) to validate modelCatalog and return an error if it's nil (similar to existing checks for manager and configStore) so GovernanceHandler is never created with a nil modelCatalog. Ensure the error message is descriptive (e.g., "model catalog is required") and keep the returned type and structure unchanged.
🧹 Nitpick comments (6)
transports/bifrost-http/handlers/providers.go (1)
187-187: Optional: deduplicate the repeated rejection message constant.The same error string appears in both handlers; moving it to a shared const will prevent drift.
Also applies to: 324-324
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/providers.go` at line 187, The rejection message "pricing_overrides is not a supported provider field; use /api/governance/pricing-overrides" is duplicated; introduce a shared constant (e.g., ErrPricingOverridesNotSupported) in transports/bifrost-http/handlers/providers.go and replace the literal strings in both handler functions that call SendError (the occurrences around the existing callers at the previously noted spots) with that constant to avoid drift.ui/components/sidebar.tsx (1)
195-198: Consider centralizing the custom-pricing route matcher.The same special-case matcher is duplicated in multiple places. Extracting one shared helper will reduce drift risk.
♻️ Proposed refactor
+const matchWorkspaceRoute = (pathname: string, url: string) => { + if (url === "/workspace/custom-pricing") return pathname === url; + return pathname.startsWith(url); +}; ... -const isRouteMatch = (url: string) => { - if (url === "/workspace/custom-pricing") return pathname === url; - return pathname.startsWith(url); -}; +const isRouteMatch = (url: string) => matchWorkspaceRoute(pathname, url); ... -const isRouteMatch = (url: string) => { - if (url === "/workspace/custom-pricing") return pathname === url; - return pathname.startsWith(url); -}; +const isRouteMatch = (url: string) => matchWorkspaceRoute(pathname, url);Also applies to: 773-776
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/components/sidebar.tsx` around lines 195 - 198, The route matcher special-casing "/workspace/custom-pricing" is duplicated; extract a single shared helper (e.g., export function isWorkspaceRouteMatch(url: string, pathname: string)) and replace local isRouteMatch implementations with calls to that helper; update usages that reference the literal "/workspace/custom-pricing" and the local pathname variable (including the other duplicated locations around the second occurrence) so all route checks use the shared function to avoid drift.ui/lib/types/governance.ts (1)
391-419: Consider tighteningrequest_typesfromstring[]to a constrained request-type union.Using
string[]here allows invalid values through the type system and pushes failures to API-time 4xx.💡 Suggested refinement
+export type PricingOverrideRequestType = + | "chat_completion" + | "text_completion" + | "embedding" + | "rerank" + | "responses" + | "speech" + | "transcription" + | "image_generation" + | "video_generation"; export interface PricingOverride { ... - request_types?: string[]; + request_types?: PricingOverrideRequestType[]; ... } export interface CreatePricingOverrideRequest { ... - request_types?: string[]; + request_types?: PricingOverrideRequestType[]; ... } export interface PatchPricingOverrideRequest { ... - request_types?: string[]; + request_types?: PricingOverrideRequestType[]; ... }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/types/governance.ts` around lines 391 - 419, Tighten the request_types fields by replacing the broad string[] with a constrained union type (e.g., type RequestType = "chat" | "completion" | "embeddings" | ...) and use RequestType[] for both CreatePricingOverrideRequest.request_types and PatchPricingOverrideRequest.request_types; add or reuse an existing RequestType union/enum in this module and update any code that constructs or validates these requests to use the new type so invalid values are caught at compile time (refer to the CreatePricingOverrideRequest and PatchPricingOverrideRequest interfaces and the request_types property).ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
22-23: Use relativeui/libimports in this TSX file to match repository conventions.These imports currently use alias paths where the guideline requests relative imports from
ui/libfor utilities/types.As per coding guidelines: `ui/**/*.{ts,tsx}` should use relative imports from the `ui/lib` directory for constants, utilities, and types.♻️ Proposed fix
} from "@/lib/store"; -import { PricingOverride, PricingOverrideScopeKind } from "@/lib/types/governance"; +} from "../../../../lib/store"; +import { PricingOverride, PricingOverrideScopeKind } from "../../../../lib/types/governance";🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 22 - 23, The imports in scopedPricingOverridesView.tsx use alias paths "@/lib/store" and "@/lib/types/governance" but should use relative ui/lib imports per repo convention; update the import statements that bring in store utilities (from "@/lib/store") and the types PricingOverride and PricingOverrideScopeKind (from "@/lib/types/governance") to use relative paths into the ui/lib directory (matching other ui/* files) so the file imports store utilities and those types via the ui/lib relative modules instead of the "@/..." aliases.transports/bifrost-http/lib/config_test.go (1)
840-858: Make pricing override mock CRUD stateful to avoid low-signal tests.These methods currently always return empty/not-found/no-op, so tests using
MockConfigStorecannot validate create/update/delete/read behavior for overrides.Proposed refactor
type MockConfigStore struct { clientConfig *configstore.ClientConfig providers map[schemas.ModelProvider]configstore.ProviderConfig + pricingOverrides map[string]tables.TablePricingOverride mcpConfig *schemas.MCPConfig governanceConfig *configstore.GovernanceConfig @@ func NewMockConfigStore() *MockConfigStore { return &MockConfigStore{ - providers: make(map[schemas.ModelProvider]configstore.ProviderConfig), + providers: make(map[schemas.ModelProvider]configstore.ProviderConfig), + pricingOverrides: make(map[string]tables.TablePricingOverride), } } @@ func (m *MockConfigStore) GetPricingOverrides(ctx context.Context, filter configstore.PricingOverrideFilter) ([]tables.TablePricingOverride, error) { - return []tables.TablePricingOverride{}, nil + overrides := make([]tables.TablePricingOverride, 0, len(m.pricingOverrides)) + for _, o := range m.pricingOverrides { + overrides = append(overrides, o) + } + return overrides, nil } func (m *MockConfigStore) GetPricingOverrideByID(ctx context.Context, id string) (*tables.TablePricingOverride, error) { - return nil, configstore.ErrNotFound + o, ok := m.pricingOverrides[id] + if !ok { + return nil, configstore.ErrNotFound + } + return &o, nil } func (m *MockConfigStore) CreatePricingOverride(ctx context.Context, override *tables.TablePricingOverride, tx ...*gorm.DB) error { + m.pricingOverrides[override.ID] = *override return nil } func (m *MockConfigStore) UpdatePricingOverride(ctx context.Context, override *tables.TablePricingOverride, tx ...*gorm.DB) error { + if _, ok := m.pricingOverrides[override.ID]; !ok { + return configstore.ErrNotFound + } + m.pricingOverrides[override.ID] = *override return nil } func (m *MockConfigStore) DeletePricingOverride(ctx context.Context, id string, tx ...*gorm.DB) error { + delete(m.pricingOverrides, id) return nil }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/lib/config_test.go` around lines 840 - 858, The MockConfigStore's pricing override methods (GetPricingOverrides, GetPricingOverrideByID, CreatePricingOverride, UpdatePricingOverride, DeletePricingOverride) are currently no-ops; modify MockConfigStore to hold an in-memory, concurrency-safe map (e.g., map[string]tables.TablePricingOverride) and a sync.Mutex or RWMutex, then implement CreatePricingOverride to insert a copy into the map (generate or use override.ID), GetPricingOverrides to return the slice of values, GetPricingOverrideByID to return the entry or configstore.ErrNotFound, UpdatePricingOverride to replace an existing entry (return ErrNotFound if missing), and DeletePricingOverride to remove the key (return ErrNotFound if missing); ensure methods return copies (not pointers into the map) and ignore the variadic tx parameter.ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx (1)
21-31: Alignui/libimports with the repository import-path rule.This file imports constants/utilities/types via alias paths; the active guideline for UI files asks for relative imports from
ui/lib.As per coding guidelines
ui/**/*.{ts,tsx}: “Use relative imports from the ui/lib directory for constants, utilities, and types.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx` around lines 21 - 31, The imports using the "@/lib/..." alias (RequestTypeLabels, ModelProvider, CreatePricingOverrideRequest, PatchPricingOverrideRequest, PricingOverride, PricingOverrideMatchType, PricingOverridePatch, PricingOverrideScopeKind, and cn) must be changed to use relative imports from the ui/lib directory per the UI import rule; locate the import block in pricingOverrideDrawer.tsx and replace each "@/lib/..." import with the corresponding relative path into ui/lib (e.g., import RequestTypeLabels from the appropriate relative ui/lib/constants/logs, ModelProvider from ui/lib/types/config, the governance types from ui/lib/types/governance, and cn from ui/lib/utils) so the file no longer uses the "@/lib" alias.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@core/schemas/pricing_overrides.go`:
- Around line 217-263: ValidatePricingOverrideScopeKind currently only trims IDs
for validation via normalizeOptionalID but doesn't return or apply the
normalized values, so callers may persist raw, untrimmed IDs; fix by either
changing ValidatePricingOverrideScopeKind (or adding a new helper like
NormalizePricingOverrideScopeIDs) to return the normalized virtualKeyID,
providerID, and providerKeyID alongside the error (e.g., return (*string,
*string, *string, error)) or update all persistence/compile call sites to call
normalizeOptionalID on virtualKeyID, providerID, providerKeyID before
constructing keys/records; reference the functions
ValidatePricingOverrideScopeKind and normalizeOptionalID and ensure the
normalized values replace the raw values before any downstream key construction
or storage.
In `@framework/modelcatalog/main.go`:
- Around line 265-267: The force-reload path currently swallows errors from
mc.loadPricingOverridesFromStore (it only logs mc.logger.Warn and continues),
allowing ForceReloadPricing to report success while scoped overrides may be
stale; change ForceReloadPricing to propagate the error from
mc.loadPricingOverridesFromStore instead of just logging it: detect the non-nil
error returned by mc.loadPricingOverridesFromStore(ctx) and return that error
(or wrap it with context) so callers receive failure; update any callers/tests
expecting success accordingly to handle the propagated error.
In `@framework/modelcatalog/pricing.go`:
- Around line 62-67: computeCacheEmbeddingCost can miss provider-scoped
overrides because it calls getPricingWithScopes without backfilling ProviderID
the way resolvePricing does; update computeCacheEmbeddingCost to populate the
scopes' ProviderID from cacheDebug.ProviderUsed (or delegate to resolvePricing)
before calling getPricingWithScopes so provider-scoped pricing is applied
correctly; reference the computeCacheEmbeddingCost function, the
getPricingWithScopes call, and the BifrostCacheDebug fields ProviderUsed and
ModelUsed when making the change.
In `@framework/modelcatalog/utils.go`:
- Around line 47-48: normalizeStreamRequestType currently doesn't map
schemas.ImageEditStreamRequest to the image generation category, causing stream
normalization to miss scoped lookups; update the normalizeStreamRequestType
function to treat schemas.ImageEditStreamRequest the same as
schemas.ImageEditRequest/schemas.ImageVariationRequest by mapping it to the
"image_generation" base type (same behavior as normalizeRequestType) so
image-edit stream requests resolve pricing/overrides correctly.
In `@transports/bifrost-http/handlers/providers.go`:
- Line 438: Update the stale comment that currently says "Provider-level pricing
overrides are deprecated and ignored" to reflect the actual behavior:
provider-level pricing overrides are rejected with a 400 Bad Request. Locate the
comment near the providers.go handlers (the comment at ~line 438) and change its
wording to state that provider-level overrides are disallowed and will result in
a 400 Bad Request, matching the validation logic that returns 400 (the rejection
code around lines 323-326).
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 264-285: Add data-testid attributes to all interactive controls in
pricingOverrideDrawer.tsx: for each mapped input inside the fields loop (the
Input component bound to form.pricingValues[field.key]) add a predictable
data-testid that includes the field.key (e.g., pricing-input-<field.key>);
likewise add data-testid attributes to any selects, checkboxes, and action
buttons in this component (referencing their component names or handler
functions such as setForm, save/close handlers) so e2e can target them. Also
scan the rest of the file (areas around the other interactive controls
referenced in the review, ~lines 545-851) and add similar testids
(workspace-selector, pricing-select-<id>, pricing-checkbox-<id>, save-button,
cancel-button) following the same naming pattern.
- Around line 507-523: When editing an override the PATCH currently omits
request_types when the user selects "All request types" because
form.requestTypes is turned into undefined, preventing the backend from clearing
the existing RequestTypes; update the payload construction in the
editingOverride branch (where PatchPricingOverrideRequest is built) to set
request_types explicitly to form.requestTypes (or to an empty array when
form.requestTypes is empty) instead of undefined so the JSON contains [] when
the user clears request types; ensure the same logic around
requestPayload.request_types / form.requestTypes is used when assigning to
PatchPricingOverrideRequest so the backend conditional (if req.RequestTypes !=
nil) receives an explicit empty array.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Line 199: The Create Override Button (onClick={openCreateDrawer}) and all
other interactive controls in scopedPricingOverridesView (create, edit, delete
buttons and any dialog/drawer action buttons such as save/cancel in the
create/edit drawer and confirm/delete dialog) must include data-testid
attributes following the workspace convention; add descriptive, convention-based
ids (e.g. scoped-pricing-overrides-create-btn,
scoped-pricing-overrides-edit-btn-<id>,
scoped-pricing-overrides-delete-btn-<id>,
scoped-pricing-overrides-drawer-save-btn,
scoped-pricing-overrides-drawer-cancel-btn,
scoped-pricing-overrides-confirm-delete-btn) to the Button component that calls
openCreateDrawer and to the corresponding edit/delete action components and
dialog action buttons so e2e tests can reliably target them.
---
Outside diff comments:
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 52-63: NewGovernanceHandler currently allows a nil modelCatalog
which can cause runtime panics; update the constructor (NewGovernanceHandler) to
validate modelCatalog and return an error if it's nil (similar to existing
checks for manager and configStore) so GovernanceHandler is never created with a
nil modelCatalog. Ensure the error message is descriptive (e.g., "model catalog
is required") and keep the returned type and structure unchanged.
---
Nitpick comments:
In `@transports/bifrost-http/handlers/providers.go`:
- Line 187: The rejection message "pricing_overrides is not a supported provider
field; use /api/governance/pricing-overrides" is duplicated; introduce a shared
constant (e.g., ErrPricingOverridesNotSupported) in
transports/bifrost-http/handlers/providers.go and replace the literal strings in
both handler functions that call SendError (the occurrences around the existing
callers at the previously noted spots) with that constant to avoid drift.
In `@transports/bifrost-http/lib/config_test.go`:
- Around line 840-858: The MockConfigStore's pricing override methods
(GetPricingOverrides, GetPricingOverrideByID, CreatePricingOverride,
UpdatePricingOverride, DeletePricingOverride) are currently no-ops; modify
MockConfigStore to hold an in-memory, concurrency-safe map (e.g.,
map[string]tables.TablePricingOverride) and a sync.Mutex or RWMutex, then
implement CreatePricingOverride to insert a copy into the map (generate or use
override.ID), GetPricingOverrides to return the slice of values,
GetPricingOverrideByID to return the entry or configstore.ErrNotFound,
UpdatePricingOverride to replace an existing entry (return ErrNotFound if
missing), and DeletePricingOverride to remove the key (return ErrNotFound if
missing); ensure methods return copies (not pointers into the map) and ignore
the variadic tx parameter.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 21-31: The imports using the "@/lib/..." alias (RequestTypeLabels,
ModelProvider, CreatePricingOverrideRequest, PatchPricingOverrideRequest,
PricingOverride, PricingOverrideMatchType, PricingOverridePatch,
PricingOverrideScopeKind, and cn) must be changed to use relative imports from
the ui/lib directory per the UI import rule; locate the import block in
pricingOverrideDrawer.tsx and replace each "@/lib/..." import with the
corresponding relative path into ui/lib (e.g., import RequestTypeLabels from the
appropriate relative ui/lib/constants/logs, ModelProvider from
ui/lib/types/config, the governance types from ui/lib/types/governance, and cn
from ui/lib/utils) so the file no longer uses the "@/lib" alias.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 22-23: The imports in scopedPricingOverridesView.tsx use alias
paths "@/lib/store" and "@/lib/types/governance" but should use relative ui/lib
imports per repo convention; update the import statements that bring in store
utilities (from "@/lib/store") and the types PricingOverride and
PricingOverrideScopeKind (from "@/lib/types/governance") to use relative paths
into the ui/lib directory (matching other ui/* files) so the file imports store
utilities and those types via the ui/lib relative modules instead of the "@/..."
aliases.
In `@ui/components/sidebar.tsx`:
- Around line 195-198: The route matcher special-casing
"/workspace/custom-pricing" is duplicated; extract a single shared helper (e.g.,
export function isWorkspaceRouteMatch(url: string, pathname: string)) and
replace local isRouteMatch implementations with calls to that helper; update
usages that reference the literal "/workspace/custom-pricing" and the local
pathname variable (including the other duplicated locations around the second
occurrence) so all route checks use the shared function to avoid drift.
In `@ui/lib/types/governance.ts`:
- Around line 391-419: Tighten the request_types fields by replacing the broad
string[] with a constrained union type (e.g., type RequestType = "chat" |
"completion" | "embeddings" | ...) and use RequestType[] for both
CreatePricingOverrideRequest.request_types and
PatchPricingOverrideRequest.request_types; add or reuse an existing RequestType
union/enum in this module and update any code that constructs or validates these
requests to use the new type so invalid values are caught at compile time (refer
to the CreatePricingOverrideRequest and PatchPricingOverrideRequest interfaces
and the request_types property).
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (39)
core/schemas/pricing_overrides.gocore/schemas/provider.goframework/configstore/clientconfig.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/store.goframework/configstore/tables/modelpricing.goframework/configstore/tables/pricingoverride.goframework/configstore/tables/provider.goframework/modelcatalog/main.goframework/modelcatalog/main_test.goframework/modelcatalog/overrides.goframework/modelcatalog/overrides_test.goframework/modelcatalog/pricing.goframework/modelcatalog/pricing_test.goframework/modelcatalog/utils.goplugins/governance/main.goplugins/logging/main.goplugins/logging/operations.goplugins/telemetry/main.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/governance_pricing_overrides.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/server/server.gotransports/config.schema.jsonui/app/workspace/custom-pricing/overrides/page.tsxui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsxui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsxui/app/workspace/providers/fragments/index.tsui/app/workspace/providers/fragments/pricingOverridesFormFragment.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/sidebar.tsxui/lib/store/apis/baseApi.tsui/lib/store/apis/governanceApi.tsui/lib/types/config.tsui/lib/types/governance.tsui/lib/types/schemas.ts
💤 Files with no reviewable changes (7)
- ui/lib/types/schemas.ts
- ui/app/workspace/providers/fragments/index.ts
- framework/configstore/clientconfig.go
- transports/config.schema.json
- ui/lib/types/config.ts
- ui/app/workspace/providers/fragments/pricingOverridesFormFragment.tsx
- transports/bifrost-http/lib/config.go
jerkeyray
commented
Feb 27, 2026
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
framework/modelcatalog/pricing.go (2)
101-127:⚠️ Potential issue | 🟠 MajorImage-edit and image-variation requests can be priced as zero.
calculateBaseCostresolves pricing for these request types (via normalization), but the final switch routes onlyImageGenerationRequest.ImageEditRequestandImageVariationRequestcurrently fall through todefaultand return0.💡 Proposed fix
switch requestType { case schemas.ChatCompletionRequest, schemas.TextCompletionRequest, schemas.ResponsesRequest: return computeTextCost(pricing, input.usage) case schemas.EmbeddingRequest: return computeEmbeddingCost(pricing, input.usage) case schemas.RerankRequest: return computeRerankCost(pricing, input.usage) case schemas.SpeechRequest: return computeSpeechCost(pricing, input.usage, input.audioSeconds) case schemas.TranscriptionRequest: return computeTranscriptionCost(pricing, input.usage, input.audioSeconds, input.audioTokenDetails) - case schemas.ImageGenerationRequest: + case schemas.ImageGenerationRequest, schemas.ImageEditRequest, schemas.ImageVariationRequest: return computeImageCost(pricing, input.imageUsage) case schemas.VideoGenerationRequest: return computeVideoCost(pricing, input.usage, input.videoSeconds) default: return 0 }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/pricing.go` around lines 101 - 127, The switch in calculateBaseCost (after normalizeStreamRequestType and resolvePricing) only handles schemas.ImageGenerationRequest so ImageEditRequest and ImageVariationRequest fall through to default and get zero cost; update the switch in calculateBaseCost to include cases for schemas.ImageEditRequest and schemas.ImageVariationRequest and route them to computeImageCost(pricing, input.imageUsage) (same as schemas.ImageGenerationRequest) so edited/variation image requests use the resolved image pricing.
431-457:⚠️ Potential issue | 🟠 MajorImage-token-specific rates are not applied in image token billing.
computeImageCostcurrently charges image tokens using generic token rates. This bypasses the newly introducedInputCostPerImageToken/OutputCostPerImageTokenfields and can misprice image workloads.💡 Proposed fix
- // Text token rates (tiered) + // Text token rates (tiered) totalTokens := imageUsage.TotalTokens inputTokenRate := tieredInputRate(pricing, totalTokens) outputTokenRate := tieredOutputRate(pricing, totalTokens) - inputCost := float64(inputTextTokens)*inputTokenRate + float64(inputImageTokens)*inputTokenRate - outputCost := float64(outputTextTokens)*outputTokenRate + float64(outputImageTokens)*outputTokenRate + inputImageRate := inputTokenRate + if pricing.InputCostPerImageToken != nil { + inputImageRate = *pricing.InputCostPerImageToken + } + outputImageRate := outputTokenRate + if pricing.OutputCostPerImageToken != nil { + outputImageRate = *pricing.OutputCostPerImageToken + } + + inputCost := float64(inputTextTokens)*inputTokenRate + float64(inputImageTokens)*inputImageRate + outputCost := float64(outputTextTokens)*outputTokenRate + float64(outputImageTokens)*outputImageRate return inputCost + outputCost }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/pricing.go` around lines 431 - 457, The computeImageCost code is charging image tokens using generic tiered rates (via tieredInputRate/tieredOutputRate) instead of the image-specific rates; change it so text tokens still use tieredInputRate/tieredOutputRate but image tokens use the pricing fields InputCostPerImageToken and OutputCostPerImageToken (e.g., set inputImageRate := pricing.InputCostPerImageToken and outputImageRate := pricing.OutputCostPerImageToken and use those to compute float64(inputImageTokens)*inputImageRate and float64(outputImageTokens)*outputImageRate), leaving the rest of the logic (totalTokens, text token handling, and function names tieredInputRate/tieredOutputRate) unchanged.
🧹 Nitpick comments (1)
ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
31-43: Extract scope-kind parsing/resolution into a shared utility.
parseScopeKind/resolveScopeKindlogic is duplicated across this file andui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx. As new scope kinds evolve in this stacked change, this can drift and silently break filtering vs. form behavior.♻️ Suggested refactor
+// ui/app/workspace/custom-pricing/overrides/scopeKind.ts +import { PricingOverride, PricingOverrideScopeKind } from "@/lib/types/governance"; + +export type ScopeFilter = "all" | PricingOverrideScopeKind; + +export function parseScopeKind(value: string | null): ScopeFilter { + if ( + value === "global" || + value === "provider" || + value === "provider_key" || + value === "virtual_key" || + value === "virtual_key_provider" || + value === "virtual_key_provider_key" + ) { + return value; + } + return "all"; +} + +export function resolveScopeKind(override: PricingOverride): PricingOverrideScopeKind { + if ( + override.scope_kind === "global" || + override.scope_kind === "provider" || + override.scope_kind === "provider_key" || + override.scope_kind === "virtual_key" || + override.scope_kind === "virtual_key_provider" || + override.scope_kind === "virtual_key_provider_key" + ) { + return override.scope_kind; + } + if (override.virtual_key_id) { + if (override.provider_key_id) return "virtual_key_provider_key"; + if (override.provider_id) return "virtual_key_provider"; + return "virtual_key"; + } + if (override.provider_key_id) return "provider_key"; + if (override.provider_id) return "provider"; + return "global"; +}Also applies to: 77-96
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 31 - 43, Duplicate scope-kind validation logic (parseScopeKind / resolveScopeKind) exists in scopedPricingOverridesView.tsx and pricingOverrideDrawer.tsx; extract this into a single shared utility function (e.g., parseScopeKind or resolveScopeKind) in a common module and replace both local implementations with an import and call to that utility. Specifically, move the allowed scope list and the normalization logic (returning the validated ScopeFilter or "all") into the new utility, update scopedPricingOverridesView.tsx to call parseScopeKind(value) instead of its inline function, and update pricingOverrideDrawer.tsx to use the same exported function so both files share one source of truth for scope-kind resolution.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 699-704: The button label can display "undefined (n)" when
form.requestTypes[0] isn't in REQUEST_TYPE_GROUPS; update the rendering logic
that uses getRequestTypeGroup(form.requestTypes[0]) (used in the
pricing-override-request-types-btn) to guard against unknown keys and provide a
fallback string (e.g. "Unknown request type" or "Other") when
getRequestTypeGroup returns undefined or the value isn't found in
REQUEST_TYPE_GROUPS; implement the guard inline where the label is computed so
the fallback is shown instead of undefined.
- Around line 338-363: The effect in useEffect that builds scopedForm from
scopeLock over-applies the lock by overwriting all ID fields and forcing a
scopeRoot, which disables Save when scopeLock is partial; change the logic in
the useEffect (the block that creates scopedForm) to only override the specific
fields present on scopeLock (virtualKeyID, providerID, providerKeyID) and leave
other fields from defaultFormState intact, and compute scopeRoot only when
scopeLock.scopeKind clearly implies a single root (otherwise keep
defaultFormState.scopeRoot); update references to FormState, setForm,
defaultFormState, scopeLock, virtualKeyID, providerID, providerKeyID, and
scopeRoot so incomplete scopeLock values do not hide/require selectors or break
validation.
---
Outside diff comments:
In `@framework/modelcatalog/pricing.go`:
- Around line 101-127: The switch in calculateBaseCost (after
normalizeStreamRequestType and resolvePricing) only handles
schemas.ImageGenerationRequest so ImageEditRequest and ImageVariationRequest
fall through to default and get zero cost; update the switch in
calculateBaseCost to include cases for schemas.ImageEditRequest and
schemas.ImageVariationRequest and route them to computeImageCost(pricing,
input.imageUsage) (same as schemas.ImageGenerationRequest) so edited/variation
image requests use the resolved image pricing.
- Around line 431-457: The computeImageCost code is charging image tokens using
generic tiered rates (via tieredInputRate/tieredOutputRate) instead of the
image-specific rates; change it so text tokens still use
tieredInputRate/tieredOutputRate but image tokens use the pricing fields
InputCostPerImageToken and OutputCostPerImageToken (e.g., set inputImageRate :=
pricing.InputCostPerImageToken and outputImageRate :=
pricing.OutputCostPerImageToken and use those to compute
float64(inputImageTokens)*inputImageRate and
float64(outputImageTokens)*outputImageRate), leaving the rest of the logic
(totalTokens, text token handling, and function names
tieredInputRate/tieredOutputRate) unchanged.
---
Nitpick comments:
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 31-43: Duplicate scope-kind validation logic (parseScopeKind /
resolveScopeKind) exists in scopedPricingOverridesView.tsx and
pricingOverrideDrawer.tsx; extract this into a single shared utility function
(e.g., parseScopeKind or resolveScopeKind) in a common module and replace both
local implementations with an import and call to that utility. Specifically,
move the allowed scope list and the normalization logic (returning the validated
ScopeFilter or "all") into the new utility, update
scopedPricingOverridesView.tsx to call parseScopeKind(value) instead of its
inline function, and update pricingOverrideDrawer.tsx to use the same exported
function so both files share one source of truth for scope-kind resolution.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
framework/modelcatalog/main.goframework/modelcatalog/overrides.goframework/modelcatalog/pricing.goframework/modelcatalog/utils.goui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsxui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx (1)
787-852:⚠️ Potential issue | 🟠 MajorAdd
data-testidto remaining interactive controls (accordion toggles + JSON editor surface).Line 789/802/815/828/841
AccordionTriggerelements are interactive but have no test IDs, and the JSON editor interaction surface at Line 860 lacks a stable selector. This still breaks the workspace e2e selector contract.As per coding guidelines `ui/app/workspace/**/*.tsx`: “must include data-testid attributes on all interactive elements.”Proposed update
<AccordionItem value="token"> - <AccordionTrigger> + <AccordionTrigger data-testid="pricing-override-token-accordion-trigger"> @@ <AccordionItem value="cache"> - <AccordionTrigger> + <AccordionTrigger data-testid="pricing-override-cache-accordion-trigger"> @@ <AccordionItem value="image"> - <AccordionTrigger> + <AccordionTrigger data-testid="pricing-override-image-accordion-trigger"> @@ <AccordionItem value="audio-video"> - <AccordionTrigger> + <AccordionTrigger data-testid="pricing-override-audio-video-accordion-trigger"> @@ <AccordionItem value="other"> - <AccordionTrigger> + <AccordionTrigger data-testid="pricing-override-other-accordion-trigger"> @@ - <div className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")}> + <div + data-testid="pricing-override-json-editor-container" + className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")} + >Also applies to: 860-869
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx` around lines 787 - 852, Add stable data-testid attributes to all interactive AccordionTrigger components and to the JSON editor interaction surface so e2e tests can reliably select them: update each AccordionTrigger in this file (the triggers for values "token", "cache", "image", "audio-video", and "other") to include a data-testid like data-testid="accordion-trigger-{value}" and update the JSON editor wrapper rendered by renderFields (or the specific JSON editor element used around lines ~860–869) to include a stable data-testid such as data-testid="json-editor-surface"; ensure attribute names are unique and follow the existing testid naming convention.
🧹 Nitpick comments (1)
ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx (1)
15-31: Switchui/libimports to relative paths in this file.Imports for store/constants/types/utils are using
@/lib/*alias, but this file is underui/and should use relative imports fromui/libper the provided repo guideline.As per coding guidelines
ui/**/*.{ts,tsx}: “Use relative imports from the ui/lib directory for constants, utilities, and types.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx` around lines 15 - 31, The imports at the top of pricingOverrideDrawer.tsx that use the "@/lib/*" alias (e.g., getErrorMessage, useCreatePricingOverrideMutation, useGetProvidersQuery, RequestTypeLabels, ModelProvider, CreatePricingOverrideRequest, cn) must be changed to relative imports that point into the ui/lib directory; locate the import block that lists these symbols and replace each "@/lib/..." import with the corresponding relative path import from ui/lib (maintain the same exported symbol names and update import paths only) so the file follows the ui/**/*.{ts,tsx} guideline to use relative ui/lib imports.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 787-852: Add stable data-testid attributes to all interactive
AccordionTrigger components and to the JSON editor interaction surface so e2e
tests can reliably select them: update each AccordionTrigger in this file (the
triggers for values "token", "cache", "image", "audio-video", and "other") to
include a data-testid like data-testid="accordion-trigger-{value}" and update
the JSON editor wrapper rendered by renderFields (or the specific JSON editor
element used around lines ~860–869) to include a stable data-testid such as
data-testid="json-editor-surface"; ensure attribute names are unique and follow
the existing testid naming convention.
---
Nitpick comments:
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 15-31: The imports at the top of pricingOverrideDrawer.tsx that
use the "@/lib/*" alias (e.g., getErrorMessage,
useCreatePricingOverrideMutation, useGetProvidersQuery, RequestTypeLabels,
ModelProvider, CreatePricingOverrideRequest, cn) must be changed to relative
imports that point into the ui/lib directory; locate the import block that lists
these symbols and replace each "@/lib/..." import with the corresponding
relative path import from ui/lib (maintain the same exported symbol names and
update import paths only) so the file follows the ui/**/*.{ts,tsx} guideline to
use relative ui/lib imports.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx (1)
871-880: Consider addingdata-testidto the JSON CodeEditor wrapper for e2e testability.The JSON patch editor lacks a
data-testid, which may complicate e2e tests that need to interact with or verify its content. Consider adding adata-testidto the wrapper div or the CodeEditor component if it supports it.♻️ Suggested improvement
- <div className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")}> + <div data-testid="pricing-override-json-editor" className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")}> <CodeEditorAs per coding guidelines
ui/app/workspace/**/*.tsx: "must include data-testid attributes on all interactive elements."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx` around lines 871 - 880, The CodeEditor instance rendering the JSON patch (CodeEditor with props lang="json", code={jsonPatch}, onChange={handleJSONChange}) is missing a data-testid which breaks e2e selectors; add a data-testid attribute (e.g., data-testid="json-patch-editor" or similar) to the outer wrapper element or directly to the CodeEditor component if it supports passthrough props so tests can reliably query the editor and its contents.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 546-559: The payload is converting optional IDs to empty strings
which relies on backend normalization; instead construct
PatchPricingOverrideRequest so virtual_key_id, provider_id and provider_key_id
are omitted when undefined (or explicitly set to null if you prefer null
semantics) rather than using "" — update the block that builds payload (used
before calling patchOverride({ id: editingOverride.id, data: payload
}).unwrap()) to only assign those keys when requestPayload.virtual_key_id /
provider_id / provider_key_id are defined (or assign null if you intentionally
want null), leaving the properties off otherwise so the JSON matches the
TypeScript optional semantics and backend omitempty handling.
---
Nitpick comments:
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideDrawer.tsx`:
- Around line 871-880: The CodeEditor instance rendering the JSON patch
(CodeEditor with props lang="json", code={jsonPatch},
onChange={handleJSONChange}) is missing a data-testid which breaks e2e
selectors; add a data-testid attribute (e.g., data-testid="json-patch-editor" or
similar) to the outer wrapper element or directly to the CodeEditor component if
it supports passthrough props so tests can reliably query the editor and its
contents.
Pratham-Mishra04
changed the base branch from
02-26-refactor_pricing_module_refactor
to
graphite-base/1825
18 tasks
|
|
18 tasks
Merged
11 tasks
Contributor
Author
|
@CodeRabbit review |
Contributor
|
@coderabbitai full review |
Contributor
|
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
framework/modelcatalog/pricing.go (1)
82-110:⚠️ Potential issue | 🟠 MajorProvider-reported totals still skip the override path.
Line 97 returns
usage.Cost.TotalCostbefore Line 110 callsresolvePricing. Any response carrying a provider-computed total will bypassapplyPricingOverrides, so global and scope-specific pricing overrides only take effect for providers that do not emit cost. Resolve pricing first and use the provider total strictly as the fallback; otherwise the same override will bill differently depending on the provider.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/pricing.go` around lines 82 - 110, In calculateBaseCost, resolve pricing before honoring provider-computed totals so overrides are applied consistently: call resolvePricing(provider, model, deployment, requestType, scopes) immediately after normalizing requestType (before checking input.usage.Cost.TotalCost), then apply applyPricingOverrides (or the existing override logic) to the resolved pricing; only if no resolved pricing/overrides exist, fall back to using input.usage.Cost.TotalCost as the final cost. Ensure extractCostInput and resolvePricing are used in that order and keep provider total strictly a fallback.ui/components/sidebar.tsx (1)
804-816:⚠️ Potential issue | 🟡 MinorInclude
itemsin the auto-expand effect dependency array.The effect reads
items, but only reruns whenpathnamechanges. If a user navigates directly to/workspace/custom-pricing/overridesbefore RBAC flags andcoreConfigresolve, the effect runs with an incompleteitemslist. Whenitemseventually updates (containing the new "Pricing Overrides" entry), the effect won't rerun becausepathnamehasn't changed, leaving the parent "Models" menu collapsed.Suggested fix
- }, [pathname]); + }, [pathname, items]);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/components/sidebar.tsx` around lines 804 - 816, The effect that auto-expands sidebar parents reads the items array but only lists pathname in its dependency array, so add items to the dependency list so the effect reruns when items change; locate the effect that defines isRouteMatch and iterates over items (uses isRouteMatch, items, newExpandedItems, and calls setExpandedItems) and include items in the useEffect dependency array alongside pathname to ensure expansion occurs when items update.
♻️ Duplicate comments (18)
core/schemas/tracer.go (1)
69-71:⚠️ Potential issue | 🟠 MajorKeep
schemas.Traceradditive; this signature change is source-breaking.Adding
*BifrostContextto an exported interface method means any external tracer implementation no longer satisfiesTracer. If this hook needs bifrost-specific state, keep the existing method compatible and add an additive escape hatch instead.Verify the in-repo surface with:
#!/bin/bash set -euo pipefail rg -n -C2 --type=go '\btype Tracer interface\b|PopulateLLMResponseAttributes\s*\(|var _ .*Tracer = \(\*.*\)\(nil\)'This only confirms in-repo implementations and call sites; any out-of-repo tracer still needs a compatibility path or an explicit breaking-change note.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/schemas/tracer.go` around lines 69 - 71, The Tracer interface was made source-breaking by adding *BifrostContext to PopulateLLMResponseAttributes; revert PopulateLLMResponseAttributes to its original signature (resp *BifrostResponse, err *BifrostError) so existing external implementations still satisfy Tracer, and implement an additive escape hatch: define a new optional interface (e.g., ContextualTracer) that declares PopulateLLMResponseAttributesWithContext(ctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError), then update call sites that currently pass a BifrostContext to type-assert the tracer to ContextualTracer and call the new method when present, otherwise fall back to the original PopulateLLMResponseAttributes; reference PopulateLLMResponseAttributes, ContextualTracer (new), PopulateLLMResponseAttributesWithContext, BifrostContext, BifrostResponse, and BifrostError in your changes.plugins/logging/main.go (1)
781-783:⚠️ Potential issue | 🟠 MajorExplicit zero-cost overrides are still dropped in logs.
Line 782 still uses
cost > 0as the presence check. That loses valid matched overrides with value0, and makes them indistinguishable from “no pricing match.”Suggested direction
- if cost := p.pricingManager.CalculateCost(result, pricingScopes); cost > 0 { - entry.Cost = &cost - } + // Use an API that separates "matched" from numeric cost value. + cost, matched := p.pricingManager.CalculateCostWithMatch(result, pricingScopes) + if matched { + entry.Cost = bifrost.Ptr(cost) + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@plugins/logging/main.go` around lines 781 - 783, The code currently treats any non-positive cost as "no match" by using cost > 0; change the flow so a valid zero cost is preserved: update p.pricingManager.CalculateCost to return both the numeric cost and a boolean (e.g. (float64, bool) or (int, bool)) indicating whether a pricing match was found, then in the caller (where pricingScopes and entry.Cost are used) check the boolean (not cost > 0) and assign entry.Cost = &cost when the boolean is true so that explicit zero-cost overrides are not dropped; refer to p.pricingManager.CalculateCost, pricingScopes (from modelcatalog.PricingLookupScopesFromContext), and entry.Cost to locate the changes.docs/providers/custom-pricing.mdx (1)
147-155:⚠️ Potential issue | 🟡 MinorDrop
chat_completion_streamfrom the example payloads.This page already says
chat_completioncovers both streaming and non-streaming requests, so repeatingchat_completion_streamhere makes the contract look broader than it is.Also applies to: 171-173
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/providers/custom-pricing.mdx` around lines 147 - 155, The example payloads include "chat_completion_stream" in the request_types array which is redundant because "chat_completion" already covers streaming; remove "chat_completion_stream" from the request_types arrays in the examples (look for the request_types field in the payload example that currently lists ["chat_completion", "chat_completion_stream"] and the duplicate occurrence around the later example) so the arrays only contain "chat_completion".docs/openapi/schemas/management/governance.yaml (1)
1160-1207:⚠️ Potential issue | 🟠 MajorEncode the scope-specific required IDs in the schema.
Right now
scope_kind: "provider_key"withoutprovider_key_idstill validates in OpenAPI. Generated clients will accept bodies the server rejects unless this is split intooneOfbranches, likeRoutingRulealready is below.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/openapi/schemas/management/governance.yaml` around lines 1160 - 1207, The CreatePricingOverrideRequest schema currently lists scope_kind and the various scope-specific ID properties but doesn't enforce that certain IDs are present for particular scope_kinds; update CreatePricingOverrideRequest to use a oneOf (like the existing RoutingRule) with separate branches for each scope_kind value (global, provider, provider_key, virtual_key, virtual_key_provider, virtual_key_provider_key) where each branch includes the common required fields (name, scope_kind, match_type, pattern, request_types, patch) and declares the additional required property for that branch (e.g., provider_id for provider and virtual_key_provider, provider_key_id for provider_key and virtual_key_provider_key, virtual_key_id for virtual_key* scopes) so OpenAPI validation will reject missing scope-specific IDs when scope_kind is a specific value.ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
52-57:⚠️ Potential issue | 🟡 MinorShow the actual virtual key in the scope column.
Every virtual-key-scoped row still renders the same
Virtual Keybadge, so different virtual keys remain indistinguishable even thoughvirtualKeyMapis already available here.Suggested fix
function scopeLabel(override: PricingOverride, virtualKeyMap: Map<string, string>): string { const scopeKind = resolveScopeKind(override); if (override.virtual_key_id && scopeKind.startsWith("virtual_key")) { - return "Virtual Key"; + return virtualKeyMap.get(override.virtual_key_id) || override.virtual_key_id; } return "Global"; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 52 - 57, scopeLabel currently always returns the static string "Virtual Key" for virtual-key-scoped overrides, which hides which virtual key is referenced; update the scopeLabel(override: PricingOverride, virtualKeyMap: Map<string,string>) function to look up override.virtual_key_id in virtualKeyMap and return a descriptive label that includes the actual virtual key (e.g. the mapped name or id fallback) when scopeKind startsWith("virtual_key"), otherwise return "Global"; reference the resolveScopeKind call and override.virtual_key_id lookup inside scopeLabel to implement this.transports/config.schema.json (2)
1932-1938:⚠️ Potential issue | 🟠 MajorRemove the deprecated provider-level override shape from the provider schemas.
The runtime now rejects
providers.*.pricing_overrides, but all provider definitions here still allow it and still referenceprovider_pricing_override. That hides the breaking change from config validation.Also applies to: 1980-1986, 2028-2034, 2076-2082, 2124-2130, 3139-3185
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/config.schema.json` around lines 1932 - 1938, Remove the deprecated provider-level override shape by deleting the "pricing_overrides" property from all provider schema objects (the properties named "pricing_overrides" that reference "$ref":"#/$defs/provider_pricing_override") and remove the "$defs/provider_pricing_override" definition itself so it cannot be referenced; update any provider schema blocks that currently include "pricing_overrides" (the occurrences around the referenced blocks) to omit that property and ensure no other $ref points to "#/$defs/provider_pricing_override".
3067-3123:⚠️ Potential issue | 🟠 MajorMake
pricing_overridereject invalid scoped configs.This schema still accepts
scope_kind: "provider_key"withoutprovider_key_id, an emptypattern, and arbitraryrequest_typesstrings. Those are all obvious config errors that should be caught here instead of leaking to runtime validation.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/config.schema.json` around lines 3067 - 3123, The pricing_override schema currently allows invalid scoped configs; update the "pricing_override" definition to validate scope-specific required fields and non-empty pattern/request_types: replace the plain "scope_kind" property with a oneOf (or conditional if/then) that enumerates each scope_kind value and lists the required IDs for that scope (e.g., require "provider_key_id" when scope_kind is "provider_key" or "virtual_key_provider_key", require "provider_id" for provider* scopes, require "virtual_key_id" for virtual_key* scopes), add "minLength": 1 to "pattern" to forbid empty patterns, and tighten "request_types" items to enforce non-empty strings (items: { "type":"string", "minLength":1 } ) or an explicit enum if known so arbitrary strings are rejected; keep additionalProperties:false as-is.ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx (1)
80-83:⚠️ Potential issue | 🟡 MinorKeep the request-type filter active while searching.
Typing in the search box currently bypasses
activeCategories, so fields from unrelated request types reappear in the flat search results.Suggested fix
const filteredFields = useMemo(() => { if (!isSearching) return null; - return PRICING_FIELDS.filter((f) => f.label.toLowerCase().includes(trimmedSearch) || f.key.toLowerCase().includes(trimmedSearch)); -}, [isSearching, trimmedSearch]); + return PRICING_FIELDS.filter((f) => { + const matchesSearch = + f.label.toLowerCase().includes(trimmedSearch) || f.key.toLowerCase().includes(trimmedSearch); + if (!matchesSearch) return false; + if (activeCategories === null) return true; + return (f.requestTypeGroups as readonly string[]).some((rg) => activeCategories.has(rg as GroupKey)); + }); +}, [activeCategories, isSearching, trimmedSearch]);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx` around lines 80 - 83, filteredFields currently ignores the activeCategories filter when isSearching is true; update the useMemo for filteredFields so that it first filters PRICING_FIELDS by the activeCategories (respecting the currently selected request-type/category) and then applies the search match on label or key using trimmedSearch; keep the existing behavior of returning null when not isSearching and ensure you reference the same variables (filteredFields, useMemo, isSearching, trimmedSearch, PRICING_FIELDS, activeCategories) when implementing the combined filter.transports/bifrost-http/lib/config.go (1)
453-456:⚠️ Potential issue | 🟠 MajorReturn the replay failure on the no-store boot path.
If
SetPricingOverridesfails here, the process still starts with base catalog pricing and silently ignores every scoped override fromconfig.json. This is the same stale-pricing startup failure mode that was already fixed inmodelcatalog.Init; this path should fail fast too.🛠️ Proposed fix
if config.ConfigStore == nil && config.ModelCatalog != nil && config.GovernanceConfig != nil && len(config.GovernanceConfig.PricingOverrides) > 0 { - if err := config.ModelCatalog.SetPricingOverrides(config.GovernanceConfig.PricingOverrides); err != nil { - logger.Warn("failed to set pricing overrides from config file: %v", err) - } + if err := config.ModelCatalog.SetPricingOverrides(config.GovernanceConfig.PricingOverrides); err != nil { + return nil, fmt.Errorf("failed to set pricing overrides from config file: %w", err) + } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/lib/config.go` around lines 453 - 456, The current no-store boot path swallows SetPricingOverrides errors (when config.ConfigStore == nil && config.ModelCatalog != nil && config.GovernanceConfig != nil) by calling logger.Warn; change this to fail fast and propagate the error instead. Replace the silent warn in the block that calls config.ModelCatalog.SetPricingOverrides(...) so that the error is returned from the enclosing initialization function (or otherwise causes boot to abort) rather than continuing with stale base catalog pricing; reference the config.ConfigStore/config.ModelCatalog/config.GovernanceConfig check and the SetPricingOverrides call and remove/replace logger.Warn with error propagation.framework/modelcatalog/overrides.go (3)
117-159:⚠️ Potential issue | 🟠 MajorEmpty-string scope identifiers pass validation but never match at runtime.
validateScopeKindchecks fornilpointers but not trimmed-empty values. A request with"provider_id": ""or"provider_id": " "passes validation here but the override will never match any runtime scope, effectively becoming a no-op.Suggested fix
Add a helper and use it in each case:
func isEmptyOrWhitespace(s *string) bool { return s == nil || strings.TrimSpace(*s) == "" }Then replace nil checks like:
case ScopeKindProvider: - if override.ProviderID == nil { + if isEmptyOrWhitespace(override.ProviderID) { return fmt.Errorf("provider_id is required for provider scope_kind") }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides.go` around lines 117 - 159, validateScopeKind currently only checks for nil pointers so empty or whitespace strings like "provider_id": "" pass validation but never match at runtime; add a helper (e.g., isEmptyOrWhitespace(*string) bool using strings.TrimSpace) and use it in validateScopeKind to treat nil or trimmed-empty values as missing. Update all checks in validateScopeKind that test VirtualKeyID, ProviderID, and ProviderKeyID (including presence-required and presence-forbidden branches such as ScopeKindProvider, ScopeKindVirtualKeyProviderKey, etc.) to call isEmptyOrWhitespace instead of comparing to nil so empty/whitespace IDs are rejected or treated as absent consistently. Ensure you import strings where needed.
381-386:⚠️ Potential issue | 🟠 MajorZero-cost overrides impossible for per-token pricing.
patchPricingtreats0as "field not present" forInputCostPerTokenandOutputCostPerToken, making it impossible to override to free-token pricing. A saved override withInputCostPerToken: 0silently leaves the catalog price unchanged.This is inconsistent with the pointer fields (lines 388-437) which use
nilto indicate "not present" and can apply explicit zero values.Suggested fix
Change
InputCostPerTokenandOutputCostPerTokeninPricingOptionsto pointer types:type PricingOptions struct { InputCostPerToken *float64 `json:"input_cost_per_token,omitempty"` OutputCostPerToken *float64 `json:"output_cost_per_token,omitempty"` // ... other fields }Then update
patchPricing:-if override.InputCostPerToken != 0 { - patched.InputCostPerToken = override.InputCostPerToken -} -if override.OutputCostPerToken != 0 { - patched.OutputCostPerToken = override.OutputCostPerToken -} +if override.InputCostPerToken != nil { + patched.InputCostPerToken = *override.InputCostPerToken +} +if override.OutputCostPerToken != nil { + patched.OutputCostPerToken = *override.OutputCostPerToken +}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides.go` around lines 381 - 386, patchPricing treats InputCostPerToken and OutputCostPerToken as zero-value signals so you cannot set them to free (0); change PricingOptions.InputCostPerToken and OutputCostPerToken from float64 to *float64 (pointer) so nil means "not present" and 0 can be explicit, then update patchPricing to check for nil (not != 0) before assigning (e.g., if override.InputCostPerToken != nil { patched.InputCostPerToken = *override.InputCostPerToken }) and ensure JSON tags/omitempty remain correct so serialization behaves like the other pointer fields.
322-330:⚠️ Potential issue | 🟠 MajorPattern stored without trimming causes lookup mismatches.
validatePatterntrims the pattern for validation (line 168) butbuildCustomPricingDatastoreso.Patternverbatim (lines 324-325). A pattern like"gpt-4 "passes validation but is indexed with the trailing space, so exact lookups for"gpt-4"won't find it.Suggested fix
for _, o := range overrides { + pattern := strings.TrimSpace(o.Pattern) entry := customPricingEntry{ id: o.ID, scopeKind: o.ScopeKind, options: o.Options, } // ... existing code ... switch o.MatchType { case MatchTypeExact: - entry.pattern = o.Pattern - data.exact[o.Pattern] = append(data.exact[o.Pattern], entry) + entry.pattern = pattern + data.exact[pattern] = append(data.exact[pattern], entry) case MatchTypeWildcard: - entry.pattern = strings.TrimSuffix(o.Pattern, "*") + entry.pattern = strings.TrimSuffix(pattern, "*") entry.wildcard = true data.wildcard = append(data.wildcard, entry) } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides.go` around lines 322 - 330, The issue: buildCustomPricingData stores o.Pattern verbatim causing lookup mismatches; fix by normalizing the pattern the same way validatePattern does before storing—use a trimmed pattern variable (e.g., p := strings.TrimSpace(o.Pattern)), for MatchTypeWildcard also remove the trailing '*' (e.g., p = strings.TrimSuffix(p, "*")), then assign entry.pattern = p and index using p into data.exact or append to data.wildcard instead of using o.Pattern; update both MatchTypeExact and MatchTypeWildcard branches in buildCustomPricingData accordingly.ui/lib/store/apis/governanceApi.ts (2)
592-608:⚠️ Potential issue | 🟠 MajorCache mutations don't update
count/total_countand ignore query filters.The
createPricingOverridehandler unshifts the new override into every fulfilledgetPricingOverridescache entry without:
- Checking if the override matches the cached query's filters (
scopeKind,virtualKeyID, etc.)- Incrementing
countandtotal_countThis causes the table to show stale counts and may display the new override in unrelated filtered views until the next poll/refetch.
Compare with
createTeam(lines 116-136) which:
- Checks if the new item matches the
searchfilter before inserting- Increments both
countandtotal_countSuggested improvement
async onQueryStarted(_arg, { dispatch, getState, queryFulfilled }) { try { const { data } = await queryFulfilled; const queries = (getState() as any).api.queries; for (const entry of Object.values(queries) as any[]) { if (entry?.endpointName !== "getPricingOverrides" || entry?.status !== "fulfilled") continue; + // Only update cache entries where the new override matches the query filters + const args = entry.originalArgs as PricingOverrideQueryArgs | undefined; + if (args?.scopeKind && args.scopeKind !== data.pricing_override.scope_kind) continue; + if (args?.virtualKeyID && args.virtualKeyID !== data.pricing_override.virtual_key_id) continue; + if (args?.providerID && args.providerID !== data.pricing_override.provider_id) continue; + if (args?.providerKeyID && args.providerKeyID !== data.pricing_override.provider_key_id) continue; + if (args?.search && !data.pricing_override.name?.toLowerCase().includes(args.search.toLowerCase())) continue; dispatch( governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, (draft) => { if (!draft.pricing_overrides) draft.pricing_overrides = []; draft.pricing_overrides.unshift(data.pricing_override); + draft.count = (draft.count || 0) + 1; + draft.total_count = (draft.total_count || 0) + 1; }), ); } } catch { // Mutation failed } },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 592 - 608, The createPricingOverride onQueryStarted cache update is inserting the new pricing override into every fulfilled getPricingOverrides cache entry without checking that the new item matches that query’s filters and without updating counts; update the loop in the onQueryStarted handler (the block using governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, ...)) to read the cached query filters from entry.originalArgs (e.g., scopeKind, virtualKeyID, search/pagination params), only insert (unshift) the new data.pricing_override when it matches those filters (same matching logic as used in createTeam), and increment draft.count and draft.total_count (if present) when you insert so counts stay correct; ensure you skip updates for queries that don’t match.
647-663:⚠️ Potential issue | 🟡 Minor
deletePricingOverridedoesn't decrement counts.The delete handler removes the override from the list but doesn't update
countortotal_count, leaving the pagination metadata stale.Suggested fix
governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, (draft) => { if (!draft.pricing_overrides) return; + const before = draft.pricing_overrides.length; draft.pricing_overrides = draft.pricing_overrides.filter((o) => o.id !== id); + if (draft.pricing_overrides.length < before) { + draft.count = Math.max(0, (draft.count || 0) - 1); + draft.total_count = Math.max(0, (draft.total_count || 0) - 1); + } }),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 647 - 663, The onQueryStarted handler for deletePricingOverride in governanceApi currently removes the item from draft.pricing_overrides but doesn't update pagination metadata; update the updateQueryData callback used for the "getPricingOverrides" query so that after filtering out the removed override (in the async onQueryStarted block) you also decrement draft.count and draft.total_count (if present) by 1 and ensure they never go below 0; reference the onQueryStarted function and the governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, ...) call and apply the count/total_count adjustments alongside removing the item.transports/bifrost-http/handlers/governance.go (1)
3418-3435:⚠️ Potential issue | 🟠 MajorDB/memory sync failure leaves inconsistent state.
The
configStore.CreatePricingOverridecall is not wrapped in a transaction with themodelCatalog.UpsertPricingOverridescall. If the DB write succeeds but the in-memory upsert fails, the client receives HTTP 500 while the override is already persisted. On next restart or catalog reload, the override will appear—confusing for operators debugging the "failed" create.Consider either:
- Rolling back the DB insert when the in-memory sync fails (requires transaction), or
- Returning success with a warning that in-memory state is stale (similar to other handlers that log and continue), or
- Documenting that a restart is required if sync fails (current error message doesn't clarify this).
Based on learnings: "In transports/bifrost-http/handlers/governance.go, if the database update succeeds but the in-memory GovernanceManager reload fails, respond with HTTP 500 to the client rather than signaling success; DB and memory must stay in sync."
The same issue applies to
updatePricingOverrideat lines 3498-3510.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3418 - 3435, The handler currently writes the pricing override to persistent storage via h.configStore.CreatePricingOverride and then upserts in-memory via h.modelCatalog.UpsertPricingOverrides, but if the in-memory upsert fails the DB write remains and the client gets a 500 while the override exists; fix both CreatePricingOverride and updatePricingOverride to keep DB and memory consistent by attempting a compensating rollback when the in-memory upsert fails: after a failed h.modelCatalog.UpsertPricingOverrides call, call the corresponding rollback on the config store (e.g., h.configStore.DeletePricingOverride(ctx, override.ID) or use an available transaction API) and if the rollback succeeds respond with a 500 including that the create/upsert failed and the DB was rolled back; if the rollback also fails, log both errors and respond 500 stating there is a persistent inconsistency and include actionable logs so operators can recover.framework/configstore/rdb.go (1)
1320-1320:⚠️ Potential issue | 🟡 MinorMake override precedence deterministic.
These queries are the load order for scoped pricing overrides, so
created_at ASCalone can flip which override wins when two rows share the same timestamp. Add a stable tie-breaker such asid ASCto both reads.💡 Stable ordering
- if err := q.Order("created_at ASC").Find(&overrides).Error; err != nil { + if err := q.Order("created_at ASC, id ASC").Find(&overrides).Error; err != nil { @@ - Order("created_at ASC"). + Order("created_at ASC, id ASC").Also applies to: 1365-1366
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/rdb.go` at line 1320, The query ordering for scoped pricing overrides uses only created_at which can produce nondeterministic precedence when timestamps tie; update the GORM Order clauses (e.g. the q.Order("created_at ASC").Find(&overrides).Error call and the similar read around the override load at the other occurrence) to add a stable tie-breaker like id ASC (e.g. Order("created_at ASC, id ASC")) so override precedence is deterministic.ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx (2)
821-833:⚠️ Potential issue | 🟡 MinorExpose a stable selector for the JSON editor.
This is the only editable control in the sheet without a
data-testid, so e2e coverage of patch editing has to fall back to fragile editor internals. Add the selector on the wrapper (or forward one to the focusable editor element).🧪 Add a stable test hook
- <div className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")}> + <div + data-testid="pricing-override-json-editor" + className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")} + > <CodeEditorAs per coding guidelines
ui/**/*.{ts,tsx}: UI interactive elements must have data-testid attributes following the patterndata-testid="<entity>-<element>-<qualifier>".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 821 - 833, The JSON editor lacks a stable test selector—add a data-testid following the project's pattern (e.g., data-testid="pricingoverride-json-editor" or similar) to the wrapper div around the CodeEditor (the div using cn("bg-muted/50 ...", jsonError && "border-destructive")) or forward the test id into the focusable element of the CodeEditor component; update the wrapper or the CodeEditor props so tests can reliably select it while keeping existing props (lang, code=jsonPatch, onChange=handleJSONChange) unchanged.
139-149:⚠️ Potential issue | 🟠 MajorStop rehydrating the sheet on provider refetches.
Because this effect depends on
providerKeyOptions, any provider query resolve/refetch while the sheet is open reruns the full hydration path and wipes unsaved edits. Reusing the shareddefaultFormStateobject also leaves stale JSON behind when reopening after a JSON-only invalid edit, becausesetForm(defaultFormState)can be a no-op. Rehydrate only on the false→true open transition (or target override change), reset with a fresh form object, and keep the provider-key backfill in a separate guarded effect.💡 One-shot hydration + fresh reset state
export const defaultFormState: FormState = { name: "", scopeRoot: "global", virtualKeyID: "", providerID: "", providerKeyID: "", matchType: "exact", pattern: "", requestTypes: [], pricingValues: {}, }; + +const createDefaultFormState = (): FormState => ({ + ...defaultFormState, + requestTypes: [], + pricingValues: {}, +}); @@ - const [form, setForm] = useState<FormState>(defaultFormState); + const [form, setForm] = useState<FormState>(() => createDefaultFormState()); + const previousOpenRef = useRef(false); @@ useEffect(() => { - if (!open) return; + const justOpened = open && !previousOpenRef.current; + previousOpenRef.current = open; + if (!justOpened) return; jsonEditingRef.current = false; setJSONError(undefined); + setJSONPatch(""); @@ - setForm(defaultFormState); - }, [open, editingOverride, scopeLock, shouldLockScope, providerKeyOptions]); + setForm(createDefaultFormState()); + }, [open, editingOverride, scopeLock, shouldLockScope]);Keep the
providerKeyOptions-based provider backfill in a second guarded effect so refetches only fillproviderIDwhen it is still blank.Also applies to: 334-339, 361-394, 428-435
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 139 - 149, The sheet is being fully rehydrated on any provider refetch because the hydration effect depends on providerKeyOptions and it reuses the shared defaultFormState (causing stale/no-op resets); change the hydration logic so you only initialize/reset the form when open transitions false→true or when the selected target override changes, and when resetting use a fresh object (e.g., clone defaultFormState via spread or structuredClone) instead of setForm(defaultFormState). Move the provider-key backfill into a separate effect that watches providerKeyOptions but only writes providerID if the current form.providerID is empty (guarded update), so providerKeyOptions refetches no longer wipe unsaved edits.
🧹 Nitpick comments (4)
docs/architecture/framework/model-catalog.mdx (1)
190-203: Show one non-nilscope example in the docs.Both updated examples still teach the unscoped call path, so readers never see how the new parameter should be populated when provider-key or virtual-key overrides are active.
📘 Suggested doc tweak
-// Calculate cost for a completed request -cost := modelCatalog.CalculateCost( - result, // *schemas.BifrostResponse - nil, // *PricingLookupScopes (nil = no scoped overrides) -) +// Calculate cost for a completed request with scoped overrides +scopes := &modelcatalog.PricingLookupScopes{ + Provider: "openai", + SelectedKeyID: "pk_123", + VirtualKeyID: "vk_123", +} +cost := modelCatalog.CalculateCost(result, scopes)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/architecture/framework/model-catalog.mdx` around lines 190 - 203, Add a concise non-nil example that demonstrates populating a PricingLookupScopes value and passing it to modelCatalog.CalculateCost so readers see how to supply provider-key or virtual-key overrides; update the docs around CalculateCost to include a short snippet description referencing CalculateCost and the PricingLookupScopes type, showing creation of a PricingLookupScopes with at least one override (e.g., providerKey or virtualKey) and passing it instead of nil so both unscoped and scoped call paths are documented.framework/modelcatalog/overrides_test.go (1)
395-487: This precedence test misses the two new hybrid scopes.The resolver order in this PR starts with
virtual_key_provider_keyandvirtual_key_provider, but this test only pinsvirtual_key > provider_key > provider > global. Please add cases for the hybrid scopes so the new highest-precedence branches are actually protected.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides_test.go` around lines 395 - 487, Add two test cases to TestApplyScopedPricingOverrides_ScopePrecedence that exercise the new hybrid resolver orders: one where PricingLookupScopes has VirtualKeyID and SelectedKeyID set (virtual_key_provider_key) and one where VirtualKeyID and Provider are set (virtual_key_provider); call mc.applyPricingOverrides with those scopes and assert the patched.InputCostPerToken equals the expected override (use the existing override values, e.g. 5.0 for the virtual-key highest-precedence case). Reference the test function name TestApplyScopedPricingOverrides_ScopePrecedence, the applyPricingOverrides method and the PricingLookupScopes fields VirtualKeyID, SelectedKeyID and Provider so the new branches are covered.transports/bifrost-http/handlers/governance.go (1)
3482-3496: Same timestamp concern for update handler.If
TablePricingOverrideuses GORM'sautoUpdateTimetag, the manualUpdatedAt: time.Now()assignment at line 3495 is unnecessary. Verify as noted in the previous comment.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3482 - 3496, The update handler is manually setting UpdatedAt: time.Now() on the TablePricingOverride struct (variable override) which is redundant if the struct uses GORM's autoUpdateTime tag; remove the manual UpdatedAt assignment from the override construction (or alternatively remove/adjust the autoUpdateTime tag on configstoreTables.TablePricingOverride) so only one mechanism updates the timestamp and avoid conflicting timestamp handling.ui/lib/store/apis/governanceApi.ts (1)
783-786: Consider exporting lazy query hook.Other governance endpoints export lazy query hooks (e.g.,
useLazyGetModelConfigsQuery). If lazy fetching of pricing overrides is needed elsewhere, consider addinguseLazyGetPricingOverridesQueryto the exports.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 783 - 786, Add and export the lazy query hook for pricing overrides by including useLazyGetPricingOverridesQuery in the export list alongside useGetPricingOverridesQuery, useCreatePricingOverrideMutation, useUpdatePricingOverrideMutation, and useDeletePricingOverrideMutation in governanceApi.ts; if the generated hook name differs, export the correct lazy hook name (e.g., useLazyGetPricingOverridesQuery) so consumers can perform lazy fetching of pricing overrides.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/providers/custom-pricing.mdx`:
- Around line 180-192: The PUT example payload in the docs omits the required
field request_types; update the example JSON payload used in the HTTP PUT (the
block showing the call to /api/governance/pricing-overrides/{id}) to include a
request_types array (e.g., ["completions"] or appropriate types) and update the
surrounding prose that describes the update shape to mention request_types as
required; ensure the same fix is applied to the other occurrence referenced
(line ~212) so both the example and the description consistently include
request_types.
- Around line 381-390: The example JSON for the "dall-e-3-rate" custom rate uses
an unsupported field output_cost_per_image_standard; update the pricing_patch to
only use fields that exist in the pricing-override schema (e.g., replace
output_cost_per_image_standard with the supported field name used by the
contract such as output_cost_per_image or the documented standard-quality
field), ensuring the keys in pricing_patch (for example
output_cost_per_image_high_quality and the corrected standard-quality key) match
the pricing-field reference exactly.
In `@framework/configstore/clientconfig.go`:
- Around line 970-985: The GeneratePricingOverrideHash currently hashes the
storage column RequestTypesJSON which can miss in-memory changes and is
sensitive to ordering; update GeneratePricingOverrideHash (and any helpers it
calls) to use the parsed in-memory RequestTypes slice from
tables.TablePricingOverride instead of RequestTypesJSON, canonicalize the slice
(e.g., sort the entries deterministically) and then serialize that canonical
form for hashing so reordering doesn't change the hash and in-memory changes are
reflected prior to persistence.
In `@framework/logstore/tables.go`:
- Around line 32-47: The diff contains whitespace-only alignment changes to
struct fields in SearchFilters (and likewise in Log) that deviate from gofmt
style and are unrelated to the pricing override work; revert those spacing-only
edits or simply run gofmt -w on the file to restore canonical formatting so the
structs (SearchFilters, Log) match gofmt/goimports output and remove noise from
this PR.
In `@framework/modelcatalog/main.go`:
- Around line 801-815: SetPricingOverrides and UpsertPricingOverrides must
ensure mc.rawOverrides contains at most one PricingOverride per ID; currently
duplicates from the input batch are preserved. Fix both functions by
deduplicating the incoming overrides by ID (use a map[string]PricingOverride
keyed by the override ID, iterating the input rows and letting later entries
overwrite earlier ones to ensure a single canonical row), then build the final
slice from that map in a deterministic way (e.g., iterate the original rows and
append the map entry the first time you see its ID, or sort keys) before
assigning to mc.rawOverrides and calling buildCustomPricingData; preserve the
existing mc.overridesMu locking around the assignment and rebuild.
In `@framework/modelcatalog/utils.go`:
- Around line 239-255: The unmarshaling in
convertTablePricingOverrideToPricingOverride currently decodes
override.PricingPatchJSON into PricingOptions which has two plain float64 fields
(InputCostPerToken, OutputCostPerToken) that will default to 0.0 when omitted
and break patch semantics; update PricingOptions so those two fields are
*float64 (matching the other override-able fields) or implement a custom
UnmarshalJSON for PricingOptions that detects whether those keys are present (so
omitted keys remain nil) and then decode override.PricingPatchJSON accordingly;
adjust any usages of PricingOptions.InputCostPerToken / OutputCostPerToken to
handle pointer semantics if you choose the *float64 approach.
In `@transports/bifrost-http/handlers/providers.go`:
- Around line 315-326: The handlers addProvider and updateProvider currently
unmarshal request bodies without rejecting deprecated provider-level
pricing_overrides; before calling sonic.Unmarshal/json.Unmarshal, inspect the
raw body (ctx.PostBody()) and reject requests that include a top-level
"pricing_overrides" key by returning SendError(ctx, fasthttp.StatusBadRequest,
"...") — implement this by unmarshaling the raw body into a generic
map[string]json.RawMessage (or use a json.Decoder with DisallowUnknownFields) to
check for the presence of "pricing_overrides" and early-return an error if
found, then proceed with the existing payload unmarshalling and processing.
In `@transports/bifrost-http/lib/config.go`:
- Around line 1498-1519: The pricing overrides are being persisted (loop over
config.GovernanceConfig.PricingOverrides using GeneratePricingOverrideHash and
ConfigStore.CreatePricingOverride) before virtual key scope targets and
virtual-key/provider bindings exist, causing valid configs to fail on first
startup; move the entire pricing override processing and creation block (the
loop that marshals RequestTypes, sets RequestTypesJSON, computes
override.ConfigHash via configstore.GeneratePricingOverrideHash and calls
config.ConfigStore.CreatePricingOverride) to run after the code that creates
virtual key scope targets and their provider bindings (the virtual_key* scope
creation logic) so that referenced virtual keys exist before
CreatePricingOverride is called.
In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx`:
- Around line 53-64: The effect currently only adds keys to activeFields via
setActiveFields and never removes stale keys, so when values changes to a
different override removed fields remain active; update the effect (the
useEffect that references setActiveFields, PRICING_FIELDS, values, and
activeFields) to compute a fresh Set of active keys based solely on the current
values (e.g., build a new Set, iterate PRICING_FIELDS and add f.key only when
values[f.key] is non-null/non-empty) and replace activeFields with that Set so
obsolete keys are removed rather than accumulated.
---
Outside diff comments:
In `@framework/modelcatalog/pricing.go`:
- Around line 82-110: In calculateBaseCost, resolve pricing before honoring
provider-computed totals so overrides are applied consistently: call
resolvePricing(provider, model, deployment, requestType, scopes) immediately
after normalizing requestType (before checking input.usage.Cost.TotalCost), then
apply applyPricingOverrides (or the existing override logic) to the resolved
pricing; only if no resolved pricing/overrides exist, fall back to using
input.usage.Cost.TotalCost as the final cost. Ensure extractCostInput and
resolvePricing are used in that order and keep provider total strictly a
fallback.
In `@ui/components/sidebar.tsx`:
- Around line 804-816: The effect that auto-expands sidebar parents reads the
items array but only lists pathname in its dependency array, so add items to the
dependency list so the effect reruns when items change; locate the effect that
defines isRouteMatch and iterates over items (uses isRouteMatch, items,
newExpandedItems, and calls setExpandedItems) and include items in the useEffect
dependency array alongside pathname to ensure expansion occurs when items
update.
---
Duplicate comments:
In `@core/schemas/tracer.go`:
- Around line 69-71: The Tracer interface was made source-breaking by adding
*BifrostContext to PopulateLLMResponseAttributes; revert
PopulateLLMResponseAttributes to its original signature (resp *BifrostResponse,
err *BifrostError) so existing external implementations still satisfy Tracer,
and implement an additive escape hatch: define a new optional interface (e.g.,
ContextualTracer) that declares PopulateLLMResponseAttributesWithContext(ctx
*BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError),
then update call sites that currently pass a BifrostContext to type-assert the
tracer to ContextualTracer and call the new method when present, otherwise fall
back to the original PopulateLLMResponseAttributes; reference
PopulateLLMResponseAttributes, ContextualTracer (new),
PopulateLLMResponseAttributesWithContext, BifrostContext, BifrostResponse, and
BifrostError in your changes.
In `@docs/openapi/schemas/management/governance.yaml`:
- Around line 1160-1207: The CreatePricingOverrideRequest schema currently lists
scope_kind and the various scope-specific ID properties but doesn't enforce that
certain IDs are present for particular scope_kinds; update
CreatePricingOverrideRequest to use a oneOf (like the existing RoutingRule) with
separate branches for each scope_kind value (global, provider, provider_key,
virtual_key, virtual_key_provider, virtual_key_provider_key) where each branch
includes the common required fields (name, scope_kind, match_type, pattern,
request_types, patch) and declares the additional required property for that
branch (e.g., provider_id for provider and virtual_key_provider, provider_key_id
for provider_key and virtual_key_provider_key, virtual_key_id for virtual_key*
scopes) so OpenAPI validation will reject missing scope-specific IDs when
scope_kind is a specific value.
In `@docs/providers/custom-pricing.mdx`:
- Around line 147-155: The example payloads include "chat_completion_stream" in
the request_types array which is redundant because "chat_completion" already
covers streaming; remove "chat_completion_stream" from the request_types arrays
in the examples (look for the request_types field in the payload example that
currently lists ["chat_completion", "chat_completion_stream"] and the duplicate
occurrence around the later example) so the arrays only contain
"chat_completion".
In `@framework/configstore/rdb.go`:
- Line 1320: The query ordering for scoped pricing overrides uses only
created_at which can produce nondeterministic precedence when timestamps tie;
update the GORM Order clauses (e.g. the q.Order("created_at
ASC").Find(&overrides).Error call and the similar read around the override load
at the other occurrence) to add a stable tie-breaker like id ASC (e.g.
Order("created_at ASC, id ASC")) so override precedence is deterministic.
In `@framework/modelcatalog/overrides.go`:
- Around line 117-159: validateScopeKind currently only checks for nil pointers
so empty or whitespace strings like "provider_id": "" pass validation but never
match at runtime; add a helper (e.g., isEmptyOrWhitespace(*string) bool using
strings.TrimSpace) and use it in validateScopeKind to treat nil or trimmed-empty
values as missing. Update all checks in validateScopeKind that test
VirtualKeyID, ProviderID, and ProviderKeyID (including presence-required and
presence-forbidden branches such as ScopeKindProvider,
ScopeKindVirtualKeyProviderKey, etc.) to call isEmptyOrWhitespace instead of
comparing to nil so empty/whitespace IDs are rejected or treated as absent
consistently. Ensure you import strings where needed.
- Around line 381-386: patchPricing treats InputCostPerToken and
OutputCostPerToken as zero-value signals so you cannot set them to free (0);
change PricingOptions.InputCostPerToken and OutputCostPerToken from float64 to
*float64 (pointer) so nil means "not present" and 0 can be explicit, then update
patchPricing to check for nil (not != 0) before assigning (e.g., if
override.InputCostPerToken != nil { patched.InputCostPerToken =
*override.InputCostPerToken }) and ensure JSON tags/omitempty remain correct so
serialization behaves like the other pointer fields.
- Around line 322-330: The issue: buildCustomPricingData stores o.Pattern
verbatim causing lookup mismatches; fix by normalizing the pattern the same way
validatePattern does before storing—use a trimmed pattern variable (e.g., p :=
strings.TrimSpace(o.Pattern)), for MatchTypeWildcard also remove the trailing
'*' (e.g., p = strings.TrimSuffix(p, "*")), then assign entry.pattern = p and
index using p into data.exact or append to data.wildcard instead of using
o.Pattern; update both MatchTypeExact and MatchTypeWildcard branches in
buildCustomPricingData accordingly.
In `@plugins/logging/main.go`:
- Around line 781-783: The code currently treats any non-positive cost as "no
match" by using cost > 0; change the flow so a valid zero cost is preserved:
update p.pricingManager.CalculateCost to return both the numeric cost and a
boolean (e.g. (float64, bool) or (int, bool)) indicating whether a pricing match
was found, then in the caller (where pricingScopes and entry.Cost are used)
check the boolean (not cost > 0) and assign entry.Cost = &cost when the boolean
is true so that explicit zero-cost overrides are not dropped; refer to
p.pricingManager.CalculateCost, pricingScopes (from
modelcatalog.PricingLookupScopesFromContext), and entry.Cost to locate the
changes.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3418-3435: The handler currently writes the pricing override to
persistent storage via h.configStore.CreatePricingOverride and then upserts
in-memory via h.modelCatalog.UpsertPricingOverrides, but if the in-memory upsert
fails the DB write remains and the client gets a 500 while the override exists;
fix both CreatePricingOverride and updatePricingOverride to keep DB and memory
consistent by attempting a compensating rollback when the in-memory upsert
fails: after a failed h.modelCatalog.UpsertPricingOverrides call, call the
corresponding rollback on the config store (e.g.,
h.configStore.DeletePricingOverride(ctx, override.ID) or use an available
transaction API) and if the rollback succeeds respond with a 500 including that
the create/upsert failed and the DB was rolled back; if the rollback also fails,
log both errors and respond 500 stating there is a persistent inconsistency and
include actionable logs so operators can recover.
In `@transports/bifrost-http/lib/config.go`:
- Around line 453-456: The current no-store boot path swallows
SetPricingOverrides errors (when config.ConfigStore == nil &&
config.ModelCatalog != nil && config.GovernanceConfig != nil) by calling
logger.Warn; change this to fail fast and propagate the error instead. Replace
the silent warn in the block that calls
config.ModelCatalog.SetPricingOverrides(...) so that the error is returned from
the enclosing initialization function (or otherwise causes boot to abort) rather
than continuing with stale base catalog pricing; reference the
config.ConfigStore/config.ModelCatalog/config.GovernanceConfig check and the
SetPricingOverrides call and remove/replace logger.Warn with error propagation.
In `@transports/config.schema.json`:
- Around line 1932-1938: Remove the deprecated provider-level override shape by
deleting the "pricing_overrides" property from all provider schema objects (the
properties named "pricing_overrides" that reference
"$ref":"#/$defs/provider_pricing_override") and remove the
"$defs/provider_pricing_override" definition itself so it cannot be referenced;
update any provider schema blocks that currently include "pricing_overrides"
(the occurrences around the referenced blocks) to omit that property and ensure
no other $ref points to "#/$defs/provider_pricing_override".
- Around line 3067-3123: The pricing_override schema currently allows invalid
scoped configs; update the "pricing_override" definition to validate
scope-specific required fields and non-empty pattern/request_types: replace the
plain "scope_kind" property with a oneOf (or conditional if/then) that
enumerates each scope_kind value and lists the required IDs for that scope
(e.g., require "provider_key_id" when scope_kind is "provider_key" or
"virtual_key_provider_key", require "provider_id" for provider* scopes, require
"virtual_key_id" for virtual_key* scopes), add "minLength": 1 to "pattern" to
forbid empty patterns, and tighten "request_types" items to enforce non-empty
strings (items: { "type":"string", "minLength":1 } ) or an explicit enum if
known so arbitrary strings are rejected; keep additionalProperties:false as-is.
In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx`:
- Around line 80-83: filteredFields currently ignores the activeCategories
filter when isSearching is true; update the useMemo for filteredFields so that
it first filters PRICING_FIELDS by the activeCategories (respecting the
currently selected request-type/category) and then applies the search match on
label or key using trimmedSearch; keep the existing behavior of returning null
when not isSearching and ensure you reference the same variables
(filteredFields, useMemo, isSearching, trimmedSearch, PRICING_FIELDS,
activeCategories) when implementing the combined filter.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 821-833: The JSON editor lacks a stable test selector—add a
data-testid following the project's pattern (e.g.,
data-testid="pricingoverride-json-editor" or similar) to the wrapper div around
the CodeEditor (the div using cn("bg-muted/50 ...", jsonError &&
"border-destructive")) or forward the test id into the focusable element of the
CodeEditor component; update the wrapper or the CodeEditor props so tests can
reliably select it while keeping existing props (lang, code=jsonPatch,
onChange=handleJSONChange) unchanged.
- Around line 139-149: The sheet is being fully rehydrated on any provider
refetch because the hydration effect depends on providerKeyOptions and it reuses
the shared defaultFormState (causing stale/no-op resets); change the hydration
logic so you only initialize/reset the form when open transitions false→true or
when the selected target override changes, and when resetting use a fresh object
(e.g., clone defaultFormState via spread or structuredClone) instead of
setForm(defaultFormState). Move the provider-key backfill into a separate effect
that watches providerKeyOptions but only writes providerID if the current
form.providerID is empty (guarded update), so providerKeyOptions refetches no
longer wipe unsaved edits.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 52-57: scopeLabel currently always returns the static string
"Virtual Key" for virtual-key-scoped overrides, which hides which virtual key is
referenced; update the scopeLabel(override: PricingOverride, virtualKeyMap:
Map<string,string>) function to look up override.virtual_key_id in virtualKeyMap
and return a descriptive label that includes the actual virtual key (e.g. the
mapped name or id fallback) when scopeKind startsWith("virtual_key"), otherwise
return "Global"; reference the resolveScopeKind call and override.virtual_key_id
lookup inside scopeLabel to implement this.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 592-608: The createPricingOverride onQueryStarted cache update is
inserting the new pricing override into every fulfilled getPricingOverrides
cache entry without checking that the new item matches that query’s filters and
without updating counts; update the loop in the onQueryStarted handler (the
block using governanceApi.util.updateQueryData("getPricingOverrides",
entry.originalArgs, ...)) to read the cached query filters from
entry.originalArgs (e.g., scopeKind, virtualKeyID, search/pagination params),
only insert (unshift) the new data.pricing_override when it matches those
filters (same matching logic as used in createTeam), and increment draft.count
and draft.total_count (if present) when you insert so counts stay correct;
ensure you skip updates for queries that don’t match.
- Around line 647-663: The onQueryStarted handler for deletePricingOverride in
governanceApi currently removes the item from draft.pricing_overrides but
doesn't update pagination metadata; update the updateQueryData callback used for
the "getPricingOverrides" query so that after filtering out the removed override
(in the async onQueryStarted block) you also decrement draft.count and
draft.total_count (if present) by 1 and ensure they never go below 0; reference
the onQueryStarted function and the
governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs,
...) call and apply the count/total_count adjustments alongside removing the
item.
---
Nitpick comments:
In `@docs/architecture/framework/model-catalog.mdx`:
- Around line 190-203: Add a concise non-nil example that demonstrates
populating a PricingLookupScopes value and passing it to
modelCatalog.CalculateCost so readers see how to supply provider-key or
virtual-key overrides; update the docs around CalculateCost to include a short
snippet description referencing CalculateCost and the PricingLookupScopes type,
showing creation of a PricingLookupScopes with at least one override (e.g.,
providerKey or virtualKey) and passing it instead of nil so both unscoped and
scoped call paths are documented.
In `@framework/modelcatalog/overrides_test.go`:
- Around line 395-487: Add two test cases to
TestApplyScopedPricingOverrides_ScopePrecedence that exercise the new hybrid
resolver orders: one where PricingLookupScopes has VirtualKeyID and
SelectedKeyID set (virtual_key_provider_key) and one where VirtualKeyID and
Provider are set (virtual_key_provider); call mc.applyPricingOverrides with
those scopes and assert the patched.InputCostPerToken equals the expected
override (use the existing override values, e.g. 5.0 for the virtual-key
highest-precedence case). Reference the test function name
TestApplyScopedPricingOverrides_ScopePrecedence, the applyPricingOverrides
method and the PricingLookupScopes fields VirtualKeyID, SelectedKeyID and
Provider so the new branches are covered.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3482-3496: The update handler is manually setting UpdatedAt:
time.Now() on the TablePricingOverride struct (variable override) which is
redundant if the struct uses GORM's autoUpdateTime tag; remove the manual
UpdatedAt assignment from the override construction (or alternatively
remove/adjust the autoUpdateTime tag on configstoreTables.TablePricingOverride)
so only one mechanism updates the timestamp and avoid conflicting timestamp
handling.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 783-786: Add and export the lazy query hook for pricing overrides
by including useLazyGetPricingOverridesQuery in the export list alongside
useGetPricingOverridesQuery, useCreatePricingOverrideMutation,
useUpdatePricingOverrideMutation, and useDeletePricingOverrideMutation in
governanceApi.ts; if the generated hook name differs, export the correct lazy
hook name (e.g., useLazyGetPricingOverridesQuery) so consumers can perform lazy
fetching of pricing overrides.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
♻️ Duplicate comments (19)
framework/logstore/tables.go (1)
32-47:⚠️ Potential issue | 🟡 MinorFormatting issue persists: revert non-gofmt alignment changes.
This issue was flagged in a previous review. The manual vertical alignment of struct field types and tags violates the coding guideline that Go code must follow gofmt/goimports style. Running
gofmtwould remove these spacing adjustments.These formatting changes remain unrelated to the PR's pricing override functionality and add noise to the diff.
🔧 Recommended fix
Run
gofmtto restore canonical formatting:gofmt -w framework/logstore/tables.goOr revert the whitespace-only changes to keep the diff focused on the PR's actual functionality.
As per coding guidelines: Go code must follow gofmt/goimports style with no custom linter config.
Also applies to: 81-133
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/logstore/tables.go` around lines 32 - 47, The struct block containing the fields Providers, Models, Status, Objects, SelectedKeyIDs, VirtualKeyIDs, RoutingRuleIDs, RoutingEngineUsed, StartTime, EndTime, MinLatency, MaxLatency, MinTokens, MaxTokens, MinCost and MaxCost was manually aligned and doesn't follow gofmt style; revert the whitespace-only alignment changes (or run gofmt -w on the file) so the struct fields and tags return to canonical gofmt/goimports formatting and remove the unrelated formatting noise from the PR.framework/configstore/tables/pricingoverride.go (1)
24-25:⚠️ Potential issue | 🟡 MinorAdd explicit GORM auto-timestamp tags for consistency.
Line 24 and Line 25 should include explicit
autoCreateTime/autoUpdateTimetags to match repository conventions and avoid timestamp behavior drift across models.🔧 Suggested change
- CreatedAt time.Time `gorm:"index;not null" json:"created_at"` - UpdatedAt time.Time `gorm:"index;not null" json:"updated_at"` + CreatedAt time.Time `gorm:"autoCreateTime;index;not null" json:"created_at"` + UpdatedAt time.Time `gorm:"autoUpdateTime;index;not null" json:"updated_at"`Based on learnings: “models with CreatedAt and UpdatedAt fields of type time.Time tagged with gorm:"autoCreateTime" and gorm:"autoUpdateTime" are populated automatically by GORM on insert/update.”
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/tables/pricingoverride.go` around lines 24 - 25, The CreatedAt and UpdatedAt fields on the PricingOverride model are missing explicit GORM auto-timestamp tags; update the struct fields (CreatedAt and UpdatedAt) to include gorm:"autoCreateTime" for CreatedAt and gorm:"autoUpdateTime" for UpdatedAt (preserving any existing tags like index and not null) so GORM will automatically populate them on insert/update and match repo conventions.core/schemas/tracer.go (1)
71-71:⚠️ Potential issue | 🟠 MajorAvoid a source-breaking
Tracerinterface change without an additive path.Line 71 changes a public interface method to require
*BifrostContext, which breaks existing externalTracerimplementations and hard-couples the API to a concrete context type. Keepcontext.Contextin the existing method and add a new optional/additive method (or context value contract) for Bifrost-specific scope data.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/schemas/tracer.go` at line 71, The change to the Tracer interface replaced context.Context with *BifrostContext on PopulateLLMResponseAttributes which will break external implementations; revert PopulateLLMResponseAttributes to accept context.Context (keep its original signature) and add an additive method such as PopulateLLMResponseAttributesWithBifrostContext(ctx context.Context, bctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError) or a V2 variant so callers that need Bifrost-specific scope can opt in without breaking existing Tracer implementations; update internal callers to use the new method where the concrete *BifrostContext is available while leaving the original method intact for backward compatibility.plugins/logging/main.go (1)
781-783:⚠️ Potential issue | 🟠 MajorPreserve explicit zero-cost overrides in logging.
Line 782 drops valid
0prices (cost > 0), so free-by-policy overrides are logged as “no cost”. Separate lookup success from numeric value and persist0when pricing is found.💡 Suggested direction
- if cost := p.pricingManager.CalculateCost(result, pricingScopes); cost > 0 { - entry.Cost = &cost - } + if cost, ok := p.pricingManager.CalculateCost(result, pricingScopes); ok { + entry.Cost = bifrost.Ptr(cost) + }(If changing
CalculateCostsignature is too broad for this PR, add an intermediate helper returning(float64, bool)and migrate call sites incrementally.)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@plugins/logging/main.go` around lines 781 - 783, The code currently treats a zero cost as "not found" because CalculateCost is checked with cost > 0; update the logic so lookup success is separated from numeric value: change p.pricingManager.CalculateCost (or add a thin helper) to return (float64, bool) — e.g., (cost, found) — then call it here using pricingScopes := modelcatalog.PricingLookupScopesFromContext(ctx, string(entry.Provider)) and if found { entry.Cost = &cost } (so a legitimate 0.0 is preserved) otherwise leave entry.Cost nil; keep any downstream callers updated or add a compatibility wrapper if you cannot change all call sites at once.docs/openapi/schemas/management/governance.yaml (1)
1160-1207:⚠️ Potential issue | 🟡 MinorMake
CreatePricingOverrideRequestself-validating.Right now the spec accepts payloads the server rejects: scope-specific IDs are only described in prose, and
match_type: wildcarddoes not require a trailing*. Encode those rules withoneOfbranches so generated clients enforce the real contract.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/openapi/schemas/management/governance.yaml` around lines 1160 - 1207, CreatePricingOverrideRequest must be made self-validating: replace the loose properties with a oneOf that defines per-scope variants and enforces pattern rules—add oneOf branches for scope_kind values (global, provider, provider_key, virtual_key, virtual_key_provider, virtual_key_provider_key) that require the corresponding ID fields (provider_id, provider_key_id, virtual_key_id, or combinations) and keep common required fields (name, scope_kind, match_type, pattern, request_types); additionally constrain match_type/wildcard by adding a pattern/regex for pattern when match_type is "wildcard" to require a trailing "*" (and a complementary branch for "exact" that disallows "*"), and preserve existing request_types array semantics (minItems:1) via the shared schema or $ref so generated clients will enforce the real contract for CreatePricingOverrideRequest.transports/bifrost-http/handlers/providers.go (1)
177-188:⚠️ Potential issue | 🟠 MajorReject legacy
pricing_overridesexplicitly.Removing the field from these payload structs is not enough: both unmarshallers still ignore unknown top-level fields, so legacy clients can send
pricing_overridesand get a successful response with the override silently discarded. Return400 Bad Requestwhen that key is present instead of treating it as a no-op.Also applies to: 315-326
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/providers.go` around lines 177 - 188, The handler currently unmarshals ctx.PostBody() directly into the payload struct which ignores unknown top-level fields so legacy clients can send pricing_overrides silently; before calling json.Unmarshal into payload, first unmarshal ctx.PostBody() into a map[string]json.RawMessage (or otherwise parse the top-level keys), check for the presence of the "pricing_overrides" key and if found call SendError(ctx, fasthttp.StatusBadRequest, ...) to reject it, otherwise proceed to json.Unmarshal into the existing payload struct as currently done; apply the same check for the other occurrence around the payload handling at the 315-326 region.framework/configstore/clientconfig.go (1)
972-983:⚠️ Potential issue | 🟠 MajorHash canonical
request_types, notRequestTypesJSON.
RequestTypesJSONis a persistence detail. If this hash is computed beforeBeforeSave, an in-memory change toRequestTypeswill be missed, and reordered equivalent sets can hash differently. Canonicalize the parsed slice and hash that representation instead.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/clientconfig.go` around lines 972 - 983, GeneratePricingOverrideHash currently hashes the persistence string RequestTypesJSON which can miss in-memory changes and vary with ordering; instead parse and canonicalize the request types slice and hash that canonical representation (e.g., sort and join the parsed []string) before writing into the sha256 stream. Update GeneratePricingOverrideHash to use the parsed/canonicalized request types slice (or p.RequestTypes if present) rather than p.RequestTypesJSON, keeping the rest of the fields (ID, Name, ScopeKind, VirtualKeyID, ProviderID, ProviderKeyID, MatchType, Pattern, PricingPatchJSON) hashed as before.docs/providers/custom-pricing.mdx (3)
381-390:⚠️ Potential issue | 🟡 MinorUse a supported pricing field in the DALL-E 3 example.
output_cost_per_image_standardis not listed in the pricing fields reference (lines 308-311). The supported fields areoutput_cost_per_image_low_quality,output_cost_per_image_medium_quality,output_cost_per_image_high_quality, andoutput_cost_per_image_auto_quality.📝 Suggested fix
- "pricing_patch": "{\"output_cost_per_image_high_quality\":0.04,\"output_cost_per_image_standard\":0.02}" + "pricing_patch": "{\"output_cost_per_image_high_quality\":0.04,\"output_cost_per_image_medium_quality\":0.02}"Or use
output_cost_per_imageif a generic per-image rate is intended.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/providers/custom-pricing.mdx` around lines 381 - 390, The DALL-E 3 example JSON uses an unsupported pricing field "output_cost_per_image_standard"; update the "pricing_patch" to use a supported field (e.g., "output_cost_per_image_medium_quality" or the generic "output_cost_per_image") instead of "output_cost_per_image_standard" so the "pricing_patch" object matches the documented fields referenced for DALL-E 3; adjust the value accordingly in the JSON for the "dall-e-3-rate" entry.
154-154:⚠️ Potential issue | 🟡 MinorRemove redundant streaming variant from example.
Line 103 states "specifying
chat_completioncovers both streaming and non-streaming chat requests," but the example includes both"chat_completion"and"chat_completion_stream". This is redundant and may confuse users.📝 Suggested fix
- "request_types": ["chat_completion", "chat_completion_stream"], + "request_types": ["chat_completion"],🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/providers/custom-pricing.mdx` at line 154, The example's request_types array is redundant: remove the "chat_completion_stream" entry so that "request_types" only contains "chat_completion" (reflecting the note that "chat_completion" covers both streaming and non-streaming requests); update the request_types array in the example JSON (the line containing "request_types": ["chat_completion", "chat_completion_stream"]) to only include "chat_completion".
180-192:⚠️ Potential issue | 🟡 MinorAdd
request_typesto the PUT example.The documentation states
request_typesis required, but this update example omits it. Users copying this example will create invalid payloads.📝 Suggested fix
-d '{ "name": "GPT-4o reduced input cost", "scope_kind": "global", "match_type": "exact", "pattern": "gpt-4o", + "request_types": ["chat_completion"], "patch": { "input_cost_per_token": 0.000002 } }'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/providers/custom-pricing.mdx` around lines 180 - 192, The PUT example JSON payload is missing the required request_types field; update the example body (the same object containing "name", "scope_kind", "match_type", "pattern", and "patch") to include a request_types array with the appropriate request type strings as documented (e.g., the valid request type values used elsewhere in the docs) so the example produces a valid payload.framework/modelcatalog/main.go (1)
800-815:⚠️ Potential issue | 🟡 MinorKeep
rawOverridesunique by ID.Both
SetPricingOverridesandUpsertPricingOverridespreserve duplicate IDs from the input batch. If callers pass rows with the same ID multiple times,mc.rawOverrideswill contain multiple records for that override, making effective pricing dependent on iteration order rather than a single canonical row.For
SetPricingOverrides: deduplicate by ID before assigning.
ForUpsertPricingOverrides: theincomingmap deduplicates keys butoverridesslice still contains all converted rows—append only unique entries.,
🛠️ Proposed fix for SetPricingOverrides
func (mc *ModelCatalog) SetPricingOverrides(rows []configstoreTables.TablePricingOverride) error { - overrides := make([]PricingOverride, 0, len(rows)) + seen := make(map[string]int, len(rows)) // id -> index in overrides + overrides := make([]PricingOverride, 0, len(rows)) for i := range rows { o, err := convertTablePricingOverrideToPricingOverride(&rows[i]) if err != nil { return err } - overrides = append(overrides, o) + if idx, exists := seen[o.ID]; exists { + overrides[idx] = o // replace with later entry + } else { + seen[o.ID] = len(overrides) + overrides = append(overrides, o) + } }Also applies to: 817-848
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/main.go` around lines 800 - 815, SetPricingOverrides currently preserves duplicate IDs from the input batch; before assigning mc.rawOverrides you must deduplicate the converted overrides by their ID so only one canonical PricingOverride per ID remains (e.g., build a local map[id]PricingOverride while iterating in SetPricingOverrides, then populate the overrides slice from that map), then set mc.rawOverrides and mc.customPricing under mc.overridesMu. Likewise in UpsertPricingOverrides, after converting rows and before appending to mc.rawOverrides ensure you only append unique IDs (use the existing incoming map or a temporary seen set to skip duplicates so overrides slice contains each ID once) and keep locking logic around mc.rawOverrides and mc.customPricing unchanged.transports/bifrost-http/lib/config.go (1)
1498-1519:⚠️ Potential issue | 🟠 MajorPricing overrides are persisted before virtual keys and their provider bindings.
This bootstrap path creates pricing overrides (lines 1498-1519) before virtual keys (lines 1521-1562). A valid
config.jsonthat scopes an override to a virtual key or virtual-key/provider combination will fail on first startup with a foreign key or scope validation error, even though the same data becomes valid once those rows exist.Consider moving the pricing override creation loop after the virtual key creation loop.
,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/lib/config.go` around lines 1498 - 1519, The pricing override creation loop that iterates over config.GovernanceConfig.PricingOverrides and calls configstore.GeneratePricingOverrideHash and config.ConfigStore.CreatePricingOverride should be moved to run after the virtual key creation loop (the block that creates virtual keys and their provider bindings) so foreign-key/scope validations succeed; locate the loop referencing override.RequestTypesJSON and override.ConfigHash and relocate it below the code that creates virtual keys/provider bindings (i.e., after the virtual key creation logic and before committing the transaction) ensuring RequestTypes JSON marshaling and hash generation still occur before calling CreatePricingOverride.transports/bifrost-http/handlers/governance.go (1)
50-64:⚠️ Potential issue | 🟠 MajorDon't make pricing-override sync optional.
These routes are always mounted, but create/update/delete silently skip the in-memory pricing sync when
modelCatalog == nil. That lets the API return success while live pricing resolution keeps using stale overrides. Please either requiremodelCatalogin this handler or refuse/mask these routes when it isn't available.Based on learnings: if a governance update succeeds in DB but the in-memory reload fails, respond with HTTP 500 because DB and memory must stay in sync.
Also applies to: 303-307, 3424-3430, 3504-3510, 3529-3531
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 50 - 64, The handler currently allows NewGovernanceHandler to be constructed with a nil modelCatalog which causes create/update/delete routes to skip the in-memory pricing override sync; change NewGovernanceHandler to require modelCatalog (return an error if modelCatalog == nil) and update any route handlers in GovernanceHandler that perform DB updates so they attempt the in-memory reload via modelCatalog and, if the DB write succeeds but the in-memory reload fails, return an HTTP 500 error (do not return success); refer to NewGovernanceHandler, GovernanceHandler, modelCatalog, governanceManager and configStore to locate and update constructor and the create/update/delete route code paths to enforce this behavior.ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
52-58:⚠️ Potential issue | 🟡 MinorShow the concrete virtual key in this badge.
scopeLabelignoresvirtualKeyMap, so every virtual-key-scoped row renders the same"Virtual Key"label. Different virtual keys become indistinguishable in the table.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 52 - 58, The scopeLabel function currently ignores virtualKeyMap and always returns "Virtual Key" for virtual-key-scoped overrides; update scopeLabel (used for PricingOverride rows) to look up the concrete virtual key name from virtualKeyMap using override.virtual_key_id and return that (e.g., the mapped name or a sensible fallback like "Virtual Key (unknown)") when scopeKind startsWith("virtual_key"); otherwise continue returning "Global".ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx (2)
80-83:⚠️ Potential issue | 🟡 MinorDon't bypass request-type filtering while searching.
filteredFieldsonly applies the text match, so typing into search re-exposes fields from categories hidden byselectedRequestTypes. That lets users add rates the non-search view intentionally suppresses.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx` around lines 80 - 83, filteredFields currently only applies the text match when isSearching, which bypasses the request-type filtering and re-exposes fields hidden by selectedRequestTypes; update the useMemo for filteredFields to first filter PRICING_FIELDS by the active selectedRequestTypes (respecting whatever logic/utility is used elsewhere to check a field's request-type membership) and then apply the text-match against trimmedSearch, so that searching narrows within the already-request-type-filtered set (keep the dependency array including isSearching and trimmedSearch and reference filteredFields/PRICING_FIELDS/selectedRequestTypes/isSearching/trimmedSearch accordingly).
53-64:⚠️ Potential issue | 🟡 MinorReset stale active rows when the backing values change.
This effect only ever adds to
activeFields. When the parent loads a different override or replacesvalueswith fewer keys, removed fields stay rendered as empty rows from the previous override.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx` around lines 53 - 64, The effect currently only adds keys to activeFields; change it to also remove stale keys when the backing values change by computing the set of keys present in values (and the subset with non-empty strings) and then calling setActiveFields to (a) preserve previous active keys only if that key still exists in values, and (b) add any keys that now have non-empty values; update the useEffect containing PRICING_FIELDS, setActiveFields, and values so it filters prev by keys present in values and unions that with the keys that have non-empty values.ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx (1)
361-394:⚠️ Potential issue | 🟠 MajorThis hydration effect can still wipe unsaved edits.
Because the effect depends on
providerKeyOptions, any provider query resolve/refetch while the sheet is open re-runs the initialization path and can resetformback toeditingOverride,scopeLock, ordefaultFormState. Gate this to the actual open transition, or skip rehydration once the user has started editing.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 361 - 394, The effect currently re-runs whenever providerKeyOptions changes and can overwrite unsaved user edits; modify the useEffect so it only performs the initialization on the open transition (when open goes from false to true) or when the user hasn't started editing. Concretely, add a persistent ref (e.g., prevOpenRef or hasUserEditedRef) and: 1) detect the rising edge (prevOpenRef.current === false && open === true) before running the init logic, or 2) skip rehydration if a hasUserEditedRef (set when user changes any form field or jsonEditingRef is true) indicates the user has modified the form; keep all existing branches (editingOverride, scopeLock, defaultFormState) but only call setForm when the guard allows it. Ensure to update prevOpenRef at the end of the effect and set/reset hasUserEditedRef when opening/closing or when user edits.ui/lib/store/apis/governanceApi.ts (1)
586-609:⚠️ Potential issue | 🟡 MinorMissing
countandtotal_countupdates in cache patch.The
createPricingOverridemutation doesn't incrementcount/total_countwhen patching the cache, unlike other create mutations in this file (e.g.,createTeamat lines 128-129,createCustomerat lines 237-238).♻️ Suggested fix
dispatch( governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, (draft) => { if (!draft.pricing_overrides) draft.pricing_overrides = []; draft.pricing_overrides.unshift(data.pricing_override); + draft.count = (draft.count || 0) + 1; + draft.total_count = (draft.total_count || 0) + 1; }), );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 586 - 609, The cache patch in createPricingOverride's onQueryStarted is missing updates to the list counts; in the onQueryStarted handler (inside createPricingOverride mutation) after unshifting data.pricing_override into draft.pricing_overrides, increment draft.count and draft.total_count (if those fields exist) the same way other create mutations do (see createTeam/createCustomer logic) so the cached getPricingOverrides response reflects the new item and updated totals.framework/configstore/rdb.go (1)
1320-1320:⚠️ Potential issue | 🟡 MinorMake override ordering deterministic.
Line 1320 and Line 1366 still sort only by
created_at. In this stack, both the resolver and the paginated list depend on stable ordering, so tied timestamps can flip the winning override or reshuffle rows between pages after reconcile/migration.💡 Stable ordering
- if err := q.Order("created_at ASC").Find(&overrides).Error; err != nil { + if err := q.Order("created_at ASC, id ASC").Find(&overrides).Error; err != nil { return nil, s.parseGormError(err) } @@ - Order("created_at ASC"). + Order("created_at ASC, id ASC"). Offset(offset). Limit(limit). Find(&overrides).Error; err != nil {Also applies to: 1365-1366
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/rdb.go` at line 1320, The ORDER clause that currently uses only "created_at ASC" is not deterministic when timestamps tie; update the queries that call q.Order("created_at ASC").Find(&overrides) (and the similar ordering at the resolver/paginated list locations) to include a stable tiebreaker such as the primary key (e.g., "created_at ASC, id ASC" or the table's primary key column) so ordering is stable across tied timestamps and pagination/reconcile operations.
🧹 Nitpick comments (8)
framework/modelcatalog/utils.go (1)
239-243: Add override context to patch decode failures.If one stored row has malformed
PricingPatchJSON, the raw unmarshal error here will not tell you which override blocked loading/reconcile. Wrapping it withoverride.ID(and optionallyoverride.Name) would make the failure actionable.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/utils.go` around lines 239 - 243, The unmarshal error in convertTablePricingOverrideToPricingOverride currently returns the raw sonic.Unmarshal error without context; change the error handling around sonic.Unmarshal([]byte(override.PricingPatchJSON), &options) to wrap or annotate the error with the override identity (at minimum override.ID, optionally override.Name) so the returned error clearly states which TablePricingOverride failed to decode (e.g., "failed to unmarshal PricingPatchJSON for override ID=<id> Name=<name>: <err>"). Ensure the function still returns a non-nil error on failure.ui/app/workspace/custom-pricing/overrides/pricingOverridesEmptyState.tsx (1)
27-27: Align the button test IDs with the repo's selector convention.These two selectors mix singular/plural entities and different token orders. Using one
pricing-overrides-{action}-btnshape will make the new E2E hooks easier to predict and reuse.♻️ Suggested rename
- data-testid="pricing-overrides-button-read-more" + data-testid="pricing-overrides-read-more-btn" ... - data-testid="pricing-override-create-btn" + data-testid="pricing-overrides-create-btn"Based on learnings, maintain the data-testid conventions across the UI codebase using consistent
{entity}-{action}-{element}-style selectors and verify sibling testids in the same file for consistency.Also applies to: 36-36
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverridesEmptyState.tsx` at line 27, Rename the inconsistent data-testid values to follow the repo convention {entity}-{action}-btn; specifically replace data-testid="pricing-overrides-button-read-more" with data-testid="pricing-overrides-read-more-btn" and update the other sibling test id in this file (the one at the other occurrence referenced in the review) to the matching pattern (e.g., pricing-overrides-{action}-btn) so both selectors use the same entity-action-element order and pluralization.framework/configstore/migrations.go (1)
4058-4072: Run the reconcile path afterCreateTabletoo.Line 4062 exits before the explicit
AutoMigrate/CreateIndexblock, so upgrades that creategovernance_pricing_overrideshere depend on GORMCreateTablealso materializingidx_pricing_override_scopeandidx_pricing_override_match. Keeping both branches on the same path makes fresh-table and existing-table upgrades deterministic.♻️ Proposed fix
if !mgr.HasTable(&tables.TablePricingOverride{}) { if err := mgr.CreateTable(&tables.TablePricingOverride{}); err != nil { return fmt.Errorf("failed to create governance_pricing_overrides table: %w", err) } - return nil } if err := tx.AutoMigrate(&tables.TablePricingOverride{}); err != nil { return fmt.Errorf("failed to automigrate governance_pricing_overrides table: %w", err) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/migrations.go` around lines 4058 - 4072, The early return after mgr.CreateTable(&tables.TablePricingOverride{}) skips the subsequent reconciliation (tx.AutoMigrate and index creation), so newly-created governance_pricing_overrides never get their indexes; remove the early return (or explicitly run the same reconcile steps) so that after calling mgr.CreateTable for TablePricingOverride you continue to call tx.AutoMigrate(&tables.TablePricingOverride{}) and then loop over the index names ("idx_pricing_override_scope", "idx_pricing_override_match") using mgr.HasIndex and mgr.CreateIndex to ensure indexes are created.framework/modelcatalog/pricing_test.go (1)
1139-1235: Add cases for override precedence and trailing-*matching.These tests still only prove provider/model fallback. It’d be worth adding a small table-driven matrix for the new resolver behavior introduced by this PR — precedence from
virtual_key_provider_keydown toglobal, plus exact vs trailing-*matches — so the core override logic has direct regression coverage here.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/pricing_test.go` around lines 1139 - 1235, Add table-driven tests in pricing_test.go that exercise resolvePricing's override precedence (virtual_key_provider_key → provider_key → provider_model_key → provider → global) and both exact and trailing-`*` pattern matches; use testCatalogWithPricing and makeKey to inject pricing entries and vary PricingLookupScopes and model/deployment inputs to assert which TableModelPricing is selected, and include cases where a trailing-`*` entry should match (e.g., "gpt-4*") versus an exact name to ensure exact wins over wildcard and higher-precedence virtual/provider-specific keys win over lower-precedence global entries.ui/components/sidebar.tsx (1)
197-200: Extract the custom-pricing matcher into one helper.The
/workspace/custom-pricingexact-match special case now exists in two places. Pull it up to a shared helper so active-state and auto-expand behavior do not drift later.Also applies to: 804-807
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/components/sidebar.tsx` around lines 197 - 200, Two identical special-case checks for "/workspace/custom-pricing" are duplicated; extract the logic into a shared helper (e.g., add a function like isCustomPricingRoute or normalizeCustomPricingMatch and use it inside isRouteMatch and the other occurrence at the other location) so both active-state and auto-expand use the same exact-match rule; update isRouteMatch to call this helper (using pathname and the url argument) and replace the duplicate logic at the other spot (currently performing the same exact-match) with a call to the new helper.docs/openapi/paths/management/governance.yaml (1)
1010-1031: Missing404response fordeletePricingOverride.Other delete operations in this file (e.g.,
deleteRoutingRule,deleteVirtualKey,deleteModelConfig) include a404response for the "not found" case. ThedeletePricingOverrideoperation only documents200and500, which is inconsistent and may mislead API consumers.📘 Suggested OpenAPI fix
responses: '200': description: Pricing override deleted successfully content: application/json: schema: $ref: '../../schemas/management/common.yaml#/MessageResponse' + '404': + description: Pricing override not found + content: + application/json: + schema: + $ref: '../../schemas/inference/common.yaml#/BifrostError' '500': $ref: '../../openapi.yaml#/components/responses/InternalError'🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/openapi/paths/management/governance.yaml` around lines 1010 - 1031, The deletePricingOverride operation (operationId: deletePricingOverride) is missing a 404 "not found" response; update its responses block to include a 404 response for the not-found case using the same response component used by other deletes (e.g., the project's NotFound/NotFoundResponse component or the ../../openapi.yaml#/components/responses/NotFound reference) so it matches deleteRoutingRule/deleteVirtualKey/deleteModelConfig conventions and returns the same schema/message used elsewhere for 404s.framework/modelcatalog/overrides_test.go (1)
395-487: Cover the two hybrid scopes in the precedence test as well.The PR's precedence ladder starts with
virtual_key_provider_keyandvirtual_key_provider, but this test only exercisesvirtual_key,provider_key,provider, andglobal. A regression in the two highest-precedence branches will still leave this test green.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides_test.go` around lines 395 - 487, The test TestApplyScopedPricingOverrides_ScopePrecedence is missing the two hybrid highest-precedence cases (virtual_key_provider_key and virtual_key_provider); add two table-driven test entries that pass PricingLookupScopes with (1) VirtualKeyID set to virtualKeyScopeID and SelectedKeyID set to providerKeyScopeID, and (2) VirtualKeyID set to virtualKeyScopeID and Provider set to providerScopeID, then call mc.applyPricingOverrides("gpt-5-nano", schemas.ChatCompletionRequest, base, tc.scopes) and assert patched.InputCostPerToken == 5.0 for both to ensure the virtual-key overrides win in those combined-scope branches.ui/lib/store/apis/governanceApi.ts (1)
570-584: Scope parameters are included even when undefined.Lines 574-577 always include
scope_kind,virtual_key_id,provider_id, andprovider_key_idin the query params, even whenundefined. This sends?scope_kind=undefinedto the API, unlikelimit,offset, andsearchwhich use conditional spread.♻️ Suggested fix
getPricingOverrides: builder.query<GetPricingOverridesResponse, PricingOverrideQueryArgs | void>({ query: (params) => ({ url: "/governance/pricing-overrides", params: { - scope_kind: params?.scopeKind, - virtual_key_id: params?.virtualKeyID, - provider_id: params?.providerID, - provider_key_id: params?.providerKeyID, + ...(params?.scopeKind && { scope_kind: params.scopeKind }), + ...(params?.virtualKeyID && { virtual_key_id: params.virtualKeyID }), + ...(params?.providerID && { provider_id: params.providerID }), + ...(params?.providerKeyID && { provider_key_id: params.providerKeyID }), ...(params?.limit !== undefined && { limit: params.limit }), ...(params?.offset !== undefined && { offset: params.offset }), ...(params?.search && { search: params.search }), }, }), providesTags: ["PricingOverrides"], }),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 570 - 584, getPricingOverrides query always emits scope_kind, virtual_key_id, provider_id and provider_key_id even when undefined; update the params construction in the getPricingOverrides builder.query so those four keys are added conditionally (like limit/offset/search) — e.g. use conditional spreads checking !== undefined on params?.scopeKind, params?.virtualKeyID, params?.providerID and params?.providerKeyID inside the params object so undefined values are not sent to the API.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@framework/modelcatalog/overrides.go`:
- Around line 21-35: PricingLookupScopesFromContext currently returns nil when
virtualKeyID, selectedKeyID, and provider are empty which lets callers skip
override lookup and thus bypass global overrides; change it to return an empty
&PricingLookupScopes{} (zero-value struct) instead of nil so
scopePriorityOrder() (which includes ScopeKindGlobal) and applyPricingOverrides
still run for global overrides; update callers if they rely on nil vs empty
struct semantics to treat nil and empty struct equivalently.
- Around line 251-264: The wildcard lookup in customPricingData.resolve is
non-deterministic because it returns the first insertion-order match; change it
to collect the wildcard entries for the current scopeKind into a local slice,
sort that slice by descending pattern length (use sort.SliceStable to keep a
stable tie-breaker), then iterate the sorted slice and apply the existing checks
(scopeKind, matchesScope, strings.HasPrefix(model, e.pattern), matchesMode) to
return the options; make the same change to the other resolve block mentioned
(lines ~299-329) so wildcard precedence is always longest-prefix-first and
stable.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3440-3496: The handler is treating CreatePricingOverrideRequest as
a full-replace because omitted fields become zero-values; change the flow to
accept an update-specific payload with pointer/nullable fields (e.g., new
UpdatePricingOverrideRequest with *string / *[]string / *MatchType / *Patch
types) or update CreatePricingOverrideRequest to use pointer fields, then
overlay non-nil fields from req onto existing (from
h.configStore.GetPricingOverrideByID) before running
normalizeAndValidatePricingOverrideName, building the
modelcatalog.PricingOverride shape and calling shape.IsValid; ensure clearing is
possible via explicit nulls (e.g., distinguish nil vs empty string), marshal the
merged Patch to JSON, and then populate configstoreTables.TablePricingOverride
using merged values (preserving existing.ConfigHash/CreatedAt and updating
UpdatedAt) so the update is a true patch/merge instead of a replace.
In `@transports/bifrost-http/handlers/pricing_override_test.go`:
- Around line 168-178: After JSON unmarshalling in both addProvider and
updateProvider handlers of ProviderHandler, explicitly inspect the raw request
JSON (e.g., by unmarshalling into a map[string]json.RawMessage or using a
json.Decoder with DisallowUnknownFields) to detect the presence of the
pricing_overrides key; if found, immediately set
ctx.Response.SetStatusCode(fasthttp.StatusBadRequest) and write
"pricing_overrides is not a supported provider field" to the response body and
return before any use of h.inMemoryStore or other struct fields so the handlers
return a graceful BadRequest instead of panicking.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 157-169: patternError currently lets exact-match patterns contain
'*' and disallows a bare '*' while backend does the opposite; update
patternError (and keep behavior consistent with
(*PricingOverride).validatePattern) so that when matchType !== "wildcard" any
'*' in trimmed returns an error like "Exact pattern cannot contain *", and when
matchType === "wildcard" allow trimmed === "*" (remove the "Pattern cannot be
just *" rejection) while still enforcing a single '*' and that if present it
must be a trailing prefix wildcard (endsWith("*")) and no other '*' in the
middle; reference function patternError and the PricingOverride validatePattern
behavior when making the changes.
- Around line 329-343: The current code collapses query failures into empty
option lists by only reading data from useGetProvidersQuery and
useGetVirtualKeysQuery; change the destructuring to also pull out isLoading and
error (e.g., const { data: providersData, isLoading: isProvidersLoading, error:
providersError } = useGetProvidersQuery(); same for virtual keys) and stop
masking errors by relying solely on providersData/virtualKeysData; instead,
compute providers and virtualKeys only when there is no error (or expose
separate flags like hasProvidersError/hasVirtualKeysError), pass those
loading/error flags into the selector components to disable them while loading
and render an inline error message when providersError or virtualKeysError is
present, and update any useMemo that builds providers/virtualKeys to depend on
the new error/loading flags (references: useGetProvidersQuery,
useGetVirtualKeysQuery, providersData, virtualKeysData, providers, virtualKeys).
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 361-372: The dialog is being closed immediately because
AlertDialogAction auto-closes the dialog before handleDeleteConfirm finishes;
update the delete confirmation so the dialog stays open until the mutation
succeeds by preventing the automatic close: remove the auto-closing
AlertDialogAction usage and instead render a regular button (or use
AlertDialogAction with asChild and pass a button) that calls handleDeleteConfirm
directly, rely on handleDeleteConfirm to call setDeleteTarget(null) only on
success, and keep using isDeleting to disable the button; refer to AlertDialog,
AlertDialogAction, onOpenChange, deleteTarget, setDeleteTarget, and
handleDeleteConfirm to locate and change the behavior.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 642-664: The deletePricingOverride mutation's cache patch (in
deletePricingOverride's onQueryStarted) removes the item from
draft.pricing_overrides but doesn't decrement pagination counters; update the
governanceApi.util.updateQueryData callback to also decrement draft.count and
draft.total_count (if they exist) when an override is removed, guarding so they
don't go below zero (mirror the approach used in deleteTeam/deleteCustomer
updateQueryData logic). Ensure you reference deletePricingOverride and the
updateQueryData("getPricingOverrides", ...) patch and update both draft.count
and draft.total_count whenever an item is filtered out.
---
Duplicate comments:
In `@core/schemas/tracer.go`:
- Line 71: The change to the Tracer interface replaced context.Context with
*BifrostContext on PopulateLLMResponseAttributes which will break external
implementations; revert PopulateLLMResponseAttributes to accept context.Context
(keep its original signature) and add an additive method such as
PopulateLLMResponseAttributesWithBifrostContext(ctx context.Context, bctx
*BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError) or
a V2 variant so callers that need Bifrost-specific scope can opt in without
breaking existing Tracer implementations; update internal callers to use the new
method where the concrete *BifrostContext is available while leaving the
original method intact for backward compatibility.
In `@docs/openapi/schemas/management/governance.yaml`:
- Around line 1160-1207: CreatePricingOverrideRequest must be made
self-validating: replace the loose properties with a oneOf that defines
per-scope variants and enforces pattern rules—add oneOf branches for scope_kind
values (global, provider, provider_key, virtual_key, virtual_key_provider,
virtual_key_provider_key) that require the corresponding ID fields (provider_id,
provider_key_id, virtual_key_id, or combinations) and keep common required
fields (name, scope_kind, match_type, pattern, request_types); additionally
constrain match_type/wildcard by adding a pattern/regex for pattern when
match_type is "wildcard" to require a trailing "*" (and a complementary branch
for "exact" that disallows "*"), and preserve existing request_types array
semantics (minItems:1) via the shared schema or $ref so generated clients will
enforce the real contract for CreatePricingOverrideRequest.
In `@docs/providers/custom-pricing.mdx`:
- Around line 381-390: The DALL-E 3 example JSON uses an unsupported pricing
field "output_cost_per_image_standard"; update the "pricing_patch" to use a
supported field (e.g., "output_cost_per_image_medium_quality" or the generic
"output_cost_per_image") instead of "output_cost_per_image_standard" so the
"pricing_patch" object matches the documented fields referenced for DALL-E 3;
adjust the value accordingly in the JSON for the "dall-e-3-rate" entry.
- Line 154: The example's request_types array is redundant: remove the
"chat_completion_stream" entry so that "request_types" only contains
"chat_completion" (reflecting the note that "chat_completion" covers both
streaming and non-streaming requests); update the request_types array in the
example JSON (the line containing "request_types": ["chat_completion",
"chat_completion_stream"]) to only include "chat_completion".
- Around line 180-192: The PUT example JSON payload is missing the required
request_types field; update the example body (the same object containing "name",
"scope_kind", "match_type", "pattern", and "patch") to include a request_types
array with the appropriate request type strings as documented (e.g., the valid
request type values used elsewhere in the docs) so the example produces a valid
payload.
In `@framework/configstore/clientconfig.go`:
- Around line 972-983: GeneratePricingOverrideHash currently hashes the
persistence string RequestTypesJSON which can miss in-memory changes and vary
with ordering; instead parse and canonicalize the request types slice and hash
that canonical representation (e.g., sort and join the parsed []string) before
writing into the sha256 stream. Update GeneratePricingOverrideHash to use the
parsed/canonicalized request types slice (or p.RequestTypes if present) rather
than p.RequestTypesJSON, keeping the rest of the fields (ID, Name, ScopeKind,
VirtualKeyID, ProviderID, ProviderKeyID, MatchType, Pattern, PricingPatchJSON)
hashed as before.
In `@framework/configstore/rdb.go`:
- Line 1320: The ORDER clause that currently uses only "created_at ASC" is not
deterministic when timestamps tie; update the queries that call
q.Order("created_at ASC").Find(&overrides) (and the similar ordering at the
resolver/paginated list locations) to include a stable tiebreaker such as the
primary key (e.g., "created_at ASC, id ASC" or the table's primary key column)
so ordering is stable across tied timestamps and pagination/reconcile
operations.
In `@framework/configstore/tables/pricingoverride.go`:
- Around line 24-25: The CreatedAt and UpdatedAt fields on the PricingOverride
model are missing explicit GORM auto-timestamp tags; update the struct fields
(CreatedAt and UpdatedAt) to include gorm:"autoCreateTime" for CreatedAt and
gorm:"autoUpdateTime" for UpdatedAt (preserving any existing tags like index and
not null) so GORM will automatically populate them on insert/update and match
repo conventions.
In `@framework/logstore/tables.go`:
- Around line 32-47: The struct block containing the fields Providers, Models,
Status, Objects, SelectedKeyIDs, VirtualKeyIDs, RoutingRuleIDs,
RoutingEngineUsed, StartTime, EndTime, MinLatency, MaxLatency, MinTokens,
MaxTokens, MinCost and MaxCost was manually aligned and doesn't follow gofmt
style; revert the whitespace-only alignment changes (or run gofmt -w on the
file) so the struct fields and tags return to canonical gofmt/goimports
formatting and remove the unrelated formatting noise from the PR.
In `@framework/modelcatalog/main.go`:
- Around line 800-815: SetPricingOverrides currently preserves duplicate IDs
from the input batch; before assigning mc.rawOverrides you must deduplicate the
converted overrides by their ID so only one canonical PricingOverride per ID
remains (e.g., build a local map[id]PricingOverride while iterating in
SetPricingOverrides, then populate the overrides slice from that map), then set
mc.rawOverrides and mc.customPricing under mc.overridesMu. Likewise in
UpsertPricingOverrides, after converting rows and before appending to
mc.rawOverrides ensure you only append unique IDs (use the existing incoming map
or a temporary seen set to skip duplicates so overrides slice contains each ID
once) and keep locking logic around mc.rawOverrides and mc.customPricing
unchanged.
In `@plugins/logging/main.go`:
- Around line 781-783: The code currently treats a zero cost as "not found"
because CalculateCost is checked with cost > 0; update the logic so lookup
success is separated from numeric value: change p.pricingManager.CalculateCost
(or add a thin helper) to return (float64, bool) — e.g., (cost, found) — then
call it here using pricingScopes :=
modelcatalog.PricingLookupScopesFromContext(ctx, string(entry.Provider)) and if
found { entry.Cost = &cost } (so a legitimate 0.0 is preserved) otherwise leave
entry.Cost nil; keep any downstream callers updated or add a compatibility
wrapper if you cannot change all call sites at once.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 50-64: The handler currently allows NewGovernanceHandler to be
constructed with a nil modelCatalog which causes create/update/delete routes to
skip the in-memory pricing override sync; change NewGovernanceHandler to require
modelCatalog (return an error if modelCatalog == nil) and update any route
handlers in GovernanceHandler that perform DB updates so they attempt the
in-memory reload via modelCatalog and, if the DB write succeeds but the
in-memory reload fails, return an HTTP 500 error (do not return success); refer
to NewGovernanceHandler, GovernanceHandler, modelCatalog, governanceManager and
configStore to locate and update constructor and the create/update/delete route
code paths to enforce this behavior.
In `@transports/bifrost-http/handlers/providers.go`:
- Around line 177-188: The handler currently unmarshals ctx.PostBody() directly
into the payload struct which ignores unknown top-level fields so legacy clients
can send pricing_overrides silently; before calling json.Unmarshal into payload,
first unmarshal ctx.PostBody() into a map[string]json.RawMessage (or otherwise
parse the top-level keys), check for the presence of the "pricing_overrides" key
and if found call SendError(ctx, fasthttp.StatusBadRequest, ...) to reject it,
otherwise proceed to json.Unmarshal into the existing payload struct as
currently done; apply the same check for the other occurrence around the payload
handling at the 315-326 region.
In `@transports/bifrost-http/lib/config.go`:
- Around line 1498-1519: The pricing override creation loop that iterates over
config.GovernanceConfig.PricingOverrides and calls
configstore.GeneratePricingOverrideHash and
config.ConfigStore.CreatePricingOverride should be moved to run after the
virtual key creation loop (the block that creates virtual keys and their
provider bindings) so foreign-key/scope validations succeed; locate the loop
referencing override.RequestTypesJSON and override.ConfigHash and relocate it
below the code that creates virtual keys/provider bindings (i.e., after the
virtual key creation logic and before committing the transaction) ensuring
RequestTypes JSON marshaling and hash generation still occur before calling
CreatePricingOverride.
In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx`:
- Around line 80-83: filteredFields currently only applies the text match when
isSearching, which bypasses the request-type filtering and re-exposes fields
hidden by selectedRequestTypes; update the useMemo for filteredFields to first
filter PRICING_FIELDS by the active selectedRequestTypes (respecting whatever
logic/utility is used elsewhere to check a field's request-type membership) and
then apply the text-match against trimmedSearch, so that searching narrows
within the already-request-type-filtered set (keep the dependency array
including isSearching and trimmedSearch and reference
filteredFields/PRICING_FIELDS/selectedRequestTypes/isSearching/trimmedSearch
accordingly).
- Around line 53-64: The effect currently only adds keys to activeFields; change
it to also remove stale keys when the backing values change by computing the set
of keys present in values (and the subset with non-empty strings) and then
calling setActiveFields to (a) preserve previous active keys only if that key
still exists in values, and (b) add any keys that now have non-empty values;
update the useEffect containing PRICING_FIELDS, setActiveFields, and values so
it filters prev by keys present in values and unions that with the keys that
have non-empty values.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 361-394: The effect currently re-runs whenever providerKeyOptions
changes and can overwrite unsaved user edits; modify the useEffect so it only
performs the initialization on the open transition (when open goes from false to
true) or when the user hasn't started editing. Concretely, add a persistent ref
(e.g., prevOpenRef or hasUserEditedRef) and: 1) detect the rising edge
(prevOpenRef.current === false && open === true) before running the init logic,
or 2) skip rehydration if a hasUserEditedRef (set when user changes any form
field or jsonEditingRef is true) indicates the user has modified the form; keep
all existing branches (editingOverride, scopeLock, defaultFormState) but only
call setForm when the guard allows it. Ensure to update prevOpenRef at the end
of the effect and set/reset hasUserEditedRef when opening/closing or when user
edits.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 52-58: The scopeLabel function currently ignores virtualKeyMap and
always returns "Virtual Key" for virtual-key-scoped overrides; update scopeLabel
(used for PricingOverride rows) to look up the concrete virtual key name from
virtualKeyMap using override.virtual_key_id and return that (e.g., the mapped
name or a sensible fallback like "Virtual Key (unknown)") when scopeKind
startsWith("virtual_key"); otherwise continue returning "Global".
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 586-609: The cache patch in createPricingOverride's onQueryStarted
is missing updates to the list counts; in the onQueryStarted handler (inside
createPricingOverride mutation) after unshifting data.pricing_override into
draft.pricing_overrides, increment draft.count and draft.total_count (if those
fields exist) the same way other create mutations do (see
createTeam/createCustomer logic) so the cached getPricingOverrides response
reflects the new item and updated totals.
---
Nitpick comments:
In `@docs/openapi/paths/management/governance.yaml`:
- Around line 1010-1031: The deletePricingOverride operation (operationId:
deletePricingOverride) is missing a 404 "not found" response; update its
responses block to include a 404 response for the not-found case using the same
response component used by other deletes (e.g., the project's
NotFound/NotFoundResponse component or the
../../openapi.yaml#/components/responses/NotFound reference) so it matches
deleteRoutingRule/deleteVirtualKey/deleteModelConfig conventions and returns the
same schema/message used elsewhere for 404s.
In `@framework/configstore/migrations.go`:
- Around line 4058-4072: The early return after
mgr.CreateTable(&tables.TablePricingOverride{}) skips the subsequent
reconciliation (tx.AutoMigrate and index creation), so newly-created
governance_pricing_overrides never get their indexes; remove the early return
(or explicitly run the same reconcile steps) so that after calling
mgr.CreateTable for TablePricingOverride you continue to call
tx.AutoMigrate(&tables.TablePricingOverride{}) and then loop over the index
names ("idx_pricing_override_scope", "idx_pricing_override_match") using
mgr.HasIndex and mgr.CreateIndex to ensure indexes are created.
In `@framework/modelcatalog/overrides_test.go`:
- Around line 395-487: The test TestApplyScopedPricingOverrides_ScopePrecedence
is missing the two hybrid highest-precedence cases (virtual_key_provider_key and
virtual_key_provider); add two table-driven test entries that pass
PricingLookupScopes with (1) VirtualKeyID set to virtualKeyScopeID and
SelectedKeyID set to providerKeyScopeID, and (2) VirtualKeyID set to
virtualKeyScopeID and Provider set to providerScopeID, then call
mc.applyPricingOverrides("gpt-5-nano", schemas.ChatCompletionRequest, base,
tc.scopes) and assert patched.InputCostPerToken == 5.0 for both to ensure the
virtual-key overrides win in those combined-scope branches.
In `@framework/modelcatalog/pricing_test.go`:
- Around line 1139-1235: Add table-driven tests in pricing_test.go that exercise
resolvePricing's override precedence (virtual_key_provider_key → provider_key →
provider_model_key → provider → global) and both exact and trailing-`*` pattern
matches; use testCatalogWithPricing and makeKey to inject pricing entries and
vary PricingLookupScopes and model/deployment inputs to assert which
TableModelPricing is selected, and include cases where a trailing-`*` entry
should match (e.g., "gpt-4*") versus an exact name to ensure exact wins over
wildcard and higher-precedence virtual/provider-specific keys win over
lower-precedence global entries.
In `@framework/modelcatalog/utils.go`:
- Around line 239-243: The unmarshal error in
convertTablePricingOverrideToPricingOverride currently returns the raw
sonic.Unmarshal error without context; change the error handling around
sonic.Unmarshal([]byte(override.PricingPatchJSON), &options) to wrap or annotate
the error with the override identity (at minimum override.ID, optionally
override.Name) so the returned error clearly states which TablePricingOverride
failed to decode (e.g., "failed to unmarshal PricingPatchJSON for override
ID=<id> Name=<name>: <err>"). Ensure the function still returns a non-nil error
on failure.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverridesEmptyState.tsx`:
- Line 27: Rename the inconsistent data-testid values to follow the repo
convention {entity}-{action}-btn; specifically replace
data-testid="pricing-overrides-button-read-more" with
data-testid="pricing-overrides-read-more-btn" and update the other sibling test
id in this file (the one at the other occurrence referenced in the review) to
the matching pattern (e.g., pricing-overrides-{action}-btn) so both selectors
use the same entity-action-element order and pluralization.
In `@ui/components/sidebar.tsx`:
- Around line 197-200: Two identical special-case checks for
"/workspace/custom-pricing" are duplicated; extract the logic into a shared
helper (e.g., add a function like isCustomPricingRoute or
normalizeCustomPricingMatch and use it inside isRouteMatch and the other
occurrence at the other location) so both active-state and auto-expand use the
same exact-match rule; update isRouteMatch to call this helper (using pathname
and the url argument) and replace the duplicate logic at the other spot
(currently performing the same exact-match) with a call to the new helper.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 570-584: getPricingOverrides query always emits scope_kind,
virtual_key_id, provider_id and provider_key_id even when undefined; update the
params construction in the getPricingOverrides builder.query so those four keys
are added conditionally (like limit/offset/search) — e.g. use conditional
spreads checking !== undefined on params?.scopeKind, params?.virtualKeyID,
params?.providerID and params?.providerKeyID inside the params object so
undefined values are not sent to the API.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Pratham-Mishra04
force-pushed
the
feat/scoped-pricing-overrides
branch
3 times, most recently
from
1d14371 to
9369f33
Compare
Contributor
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
♻️ Duplicate comments (11)
core/schemas/tracer.go (1)
71-71:⚠️ Potential issue | 🟠 MajorPublic
Tracerinterface became source-breaking by requiring*BifrostContext.Line 71 changes an exported interface method from a generic context contract to a concrete type, which breaks external tracer implementations and hard-couples the API to Bifrost internals. Please keep the public method on
context.Contextand add a separate additive hook if Bifrost-specific data is needed.Suggested compatibility-preserving shape
type Tracer interface { - PopulateLLMResponseAttributes(ctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError) + PopulateLLMResponseAttributes(ctx context.Context, handle SpanHandle, resp *BifrostResponse, err *BifrostError) } ... -func (n *NoOpTracer) PopulateLLMResponseAttributes(_ *BifrostContext, _ SpanHandle, _ *BifrostResponse, _ *BifrostError) { +func (n *NoOpTracer) PopulateLLMResponseAttributes(_ context.Context, _ SpanHandle, _ *BifrostResponse, _ *BifrostError) { }Also applies to: 147-147
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/schemas/tracer.go` at line 71, The Tracer interface was made breaking by changing the exported method PopulateLLMResponseAttributes to require *BifrostContext; revert the public method signature to use context.Context (PopulateLLMResponseAttributes(ctx context.Context, handle SpanHandle, resp *BifrostResponse, err *BifrostError)) so external implementations remain compatible, and add a new additive, Bifrost-specific hook (e.g. PopulateLLMResponseAttributesWithBifrost(ctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError)) for internal consumers that need Bifrost internals; update both occurrences of the signature in the Tracer interface and any internal callers to call the new Bifrost-specific method when they have a *BifrostContext.transports/bifrost-http/handlers/providers.go (1)
315-326:⚠️ Potential issue | 🟠 MajorReject legacy
pricing_overridesinstead of silently dropping it.Removing the field from the typed payload does not fail old clients here:
sonic.Unmarshalstill ignores unknown top-level keys, soPUT /api/providers/{provider}will return 200 and discard provider-level overrides. That makes the migration easy to miss and can silently disable custom billing. Add a raw-body check before unmarshaling, and reuse the same helper inaddProvider.Suggested fix
+func rejectLegacyProviderPricingOverrides(body []byte) error { + var raw map[string]json.RawMessage + if err := json.Unmarshal(body, &raw); err != nil { + return nil // keep the existing invalid-JSON handling below + } + if _, ok := raw["pricing_overrides"]; ok { + return fmt.Errorf("pricing_overrides is not a supported provider field; use /api/governance/pricing-overrides instead") + } + return nil +} + func (h *ProviderHandler) updateProvider(ctx *fasthttp.RequestCtx) { provider, err := getProviderFromCtx(ctx) if err != nil { @@ var payload = struct { Keys []schemas.Key `json:"keys"` NetworkConfig schemas.NetworkConfig `json:"network_config"` ConcurrencyAndBufferSize schemas.ConcurrencyAndBufferSize `json:"concurrency_and_buffer_size"` ProxyConfig *schemas.ProxyConfig `json:"proxy_config,omitempty"` SendBackRawRequest *bool `json:"send_back_raw_request,omitempty"` SendBackRawResponse *bool `json:"send_back_raw_response,omitempty"` CustomProviderConfig *schemas.CustomProviderConfig `json:"custom_provider_config,omitempty"` }{} + if err := rejectLegacyProviderPricingOverrides(ctx.PostBody()); err != nil { + SendError(ctx, fasthttp.StatusBadRequest, err.Error()) + return + } + if err := sonic.Unmarshal(ctx.PostBody(), &payload); err != nil { SendError(ctx, fasthttp.StatusBadRequest, fmt.Sprintf("Invalid JSON: %v", err)) return }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/providers.go` around lines 315 - 326, The handler currently unmarshals into a typed payload (variable payload) which ignores unknown top-level keys so legacy "pricing_overrides" gets silently dropped; before calling sonic.Unmarshal(ctx.PostBody(), &payload) add a raw-body validation that scans the JSON text for the top-level key "pricing_overrides" (or use a shared helper function reused by addProvider) and return a 400 via SendError if that key is present, so PUT /api/providers/{provider} rejects legacy overrides instead of discarding them; implement the same helper and call-site in addProvider to ensure consistent behavior across both endpoints.ui/lib/store/apis/governanceApi.ts (1)
643-660:⚠️ Potential issue | 🟠 MajorUse the full query-membership check when patching updates.
matchesQueryonly looks at scope fields, and theindex === -1early return means an updated override is never inserted into a cached list it now belongs to. Renaming or re-scoping an override can therefore leave filtered/search caches stale: lists keep rows that no longer match, while newly matching lists never get the row or atotal_countbump until refetch. This patch needs the same membership predicate asgetPricingOverrides(including search), and it needs to handle theindex === -1 && matchesQuerycase.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 643 - 660, The patch only uses matchesQuery (which checks a subset of fields) and returns early when index === -1, causing lists to fall out of sync; update the updateQueryData handler for getPricingOverrides to use the full membership predicate that getPricingOverrides uses (including the search text and all scope/provider/providerKey fields) instead of the current matchesQuery, and change the index handling so that if index === -1 and the full-membership predicate is true you insert the updated item into draft.pricing_overrides and increment draft.count and draft.total_count appropriately, while if index !== -1 and the item no longer matches you splice it out and decrement counts (preserving the existing splice/dec count logic).docs/openapi/schemas/management/governance.yaml (1)
1160-1207:⚠️ Potential issue | 🟠 MajorEncode the scope-specific IDs in the request schema.
virtual_key_id,provider_id, andprovider_key_idare only "required" in prose here, so client-generated requests can validate against OpenAPI and still fail server-side for non-global scopes. Please modelCreatePricingOverrideRequestwithoneOfbranches keyed byscope_kind(likeRoutingRule) so the relevant IDs are actually required by the schema itself.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/openapi/schemas/management/governance.yaml` around lines 1160 - 1207, The CreatePricingOverrideRequest schema currently lists virtual_key_id, provider_id, and provider_key_id only in prose; update the schema to use oneOf branches keyed by scope_kind so each scope variant enforces its required IDs: create oneOf alternatives for scope_kind values (e.g., global, provider, provider_key, virtual_key, virtual_key_provider, virtual_key_provider_key) mirroring RoutingRule's pattern, keep common properties (name, match_type, pattern, request_types, patch) in each branch, and add required arrays inside each branch to require virtual_key_id for virtual_key* scopes, provider_id for provider and virtual_key_provider scopes, and provider_key_id for provider_key and virtual_key_provider_key scopes so the OpenAPI validator enforces correct IDs.framework/configstore/rdb.go (1)
1320-1320:⚠️ Potential issue | 🟡 MinorAdd a deterministic tie-breaker to pricing override reads.
Line 1320 and Line 1366 still sort only by
created_at. If two overrides land in the same timestamp bucket, list order can drift between reloads/pages and the resolver can pick a different winner inside the same precedence level.📌 Stable ordering
- if err := q.Order("created_at ASC").Find(&overrides).Error; err != nil { + if err := q.Order("created_at ASC, id ASC").Find(&overrides).Error; err != nil { return nil, s.parseGormError(err) } @@ - Order("created_at ASC"). + Order("created_at ASC, id ASC"). Offset(offset). Limit(limit). Find(&overrides).Error; err != nil {Also applies to: 1365-1366
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/rdb.go` at line 1320, The query ordering currently uses Order("created_at ASC") when loading pricing overrides (see the call pattern q.Order("created_at ASC").Find(&overrides).Error) which can produce non-deterministic order for rows sharing the same timestamp; update the Order clause to include a deterministic tie-breaker (e.g., append the primary key column such as "id ASC" or the UUID column: Order("created_at ASC, id ASC")) so identical created_at values have stable ordering, and apply the same change to the other occurrence noted around the second call (the Order call near lines 1365-1366) so both reads use the deterministic compound sort.transports/config.schema.json (1)
3032-3103:⚠️ Potential issue | 🟠 MajorTighten
pricing_overridevalidation to match the new runtime rules.Line 3044 never requires the scope IDs implied by
scope_kind, Line 3066 still accepts empty or malformed patterns, and Line 3074 still allows arbitrary request type strings. Given this stack only supports exact matches or trailing*wildcards, configs likescope_kind: "provider_key"withoutprovider_key_id,request_types: ["foo"], ormatch_type: "wildcard"withpattern: "*bar"will still validate here and only fail later.🛡️ Suggested schema guards
"pattern": { "type": "string", + "minLength": 1, "description": "Model name pattern to match (exact name or wildcard prefix ending with *)" }, "request_types": { "type": "array", "description": "Request types this override applies to. At least one value is required.", "minItems": 1, "items": { - "type": "string" + "$ref": "#/$defs/pricing_override_request_type" } }, @@ - "required": ["id", "name", "scope_kind", "match_type", "pattern", "request_types"], + "required": ["id", "name", "scope_kind", "match_type", "pattern", "request_types"], + "allOf": [ + { + "if": { + "properties": { + "scope_kind": { + "enum": ["virtual_key", "virtual_key_provider", "virtual_key_provider_key"] + } + } + }, + "then": { + "required": ["virtual_key_id"] + } + }, + { + "if": { + "properties": { + "scope_kind": { + "enum": ["provider", "virtual_key_provider"] + } + } + }, + "then": { + "required": ["provider_id"] + } + }, + { + "if": { + "properties": { + "scope_kind": { + "enum": ["provider_key", "virtual_key_provider_key"] + } + } + }, + "then": { + "required": ["provider_key_id"] + } + }, + { + "if": { + "properties": { + "match_type": { + "const": "exact" + } + } + }, + "then": { + "properties": { + "pattern": { + "pattern": "^[^*]+$" + } + } + } + }, + { + "if": { + "properties": { + "match_type": { + "const": "wildcard" + } + } + }, + "then": { + "properties": { + "pattern": { + "pattern": "^[^*]+\\*$" + } + } + } + } + ], "additionalProperties": false🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/config.schema.json` around lines 3032 - 3103, The pricing_override schema allows invalid configs; tighten it by (1) adding conditional required fields on pricing_override.scope_kind so provider_id is required when scope_kind contains "provider", provider_key_id is required for "provider_key" or "virtual_key_provider_key", and virtual_key_id is required for any "virtual_key*" scope (use "if/then" based on scope_kind values referencing pricing_override.scope_kind and the specific id properties), (2) constrain pricing_override.pattern based on pricing_override.match_type so that when match_type == "exact" pattern cannot contain "*" and when match_type == "wildcard" pattern must be a non-empty string that ends with a single trailing "*" and contains no other "*" (use pattern/regex validation tied to match_type via conditional subschemas), and (3) tighten request_types to only allow the defined request type enum by referencing pricing_override_request_type for items instead of free strings; update the schema for pricing_override to include these conditional/regex rules and ensure existing required fields remain.ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx (3)
717-735:⚠️ Potential issue | 🟡 MinorDisable the provider-key selector when provider options are unavailable.
providerScopedKeyOptionsis derived fromproviders, but this select stays enabled while the providers query is loading or failed. In that state, a fetch failure looks the same as “this provider has no keys,” and edit mode can silently lose the current key context.♻️ Suggested fix
<Select + disabled={isProvidersLoading || !!providersError} value={form.providerKeyID || "__none__"} onValueChange={(value) => setForm((prev) => ({ ...prev, providerKeyID: value === "__none__" ? "" : value }))} >🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 717 - 735, The Provider key Select should be disabled when provider options are unavailable; update the JSX around the Select (the element using value={form.providerKeyID || "__none__"} and onValueChange={...}) to pass a disabled prop when providerScopedKeyOptions is empty or when the providers query is loading/errored (use the relevant loading/error flags from the providers query), and ensure setForm no-ops or preserves providerKeyID until options are present so edits don't silently clear the key; check symbols providerScopedKeyOptions, form.providerID, form.providerKeyID, Select, and setForm to implement this guard.
844-853:⚠️ Potential issue | 🟡 MinorAdd a
data-testidfor the JSON editor.This is still part of the editing surface, but it's the only new interactive control here without a stable selector.
♻️ Suggested fix
- <CodeEditor - lang="json" - code={jsonPatch} - onChange={handleJSONChange} - minHeight={40} - maxHeight={200} - autoResize - shouldAdjustInitialHeight - options={{ lineNumbers: "off", scrollBeyondLastLine: false }} - /> + <div data-testid="pricing-override-json-editor"> + <CodeEditor + lang="json" + code={jsonPatch} + onChange={handleJSONChange} + minHeight={40} + maxHeight={200} + autoResize + shouldAdjustInitialHeight + options={{ lineNumbers: "off", scrollBeyondLastLine: false }} + /> + </div>As per coding guidelines
ui/**/*.{ts,tsx}: UI interactive elements must have data-testid attributes following the patterndata-testid="<entity>-<element>-<qualifier>".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 844 - 853, The CodeEditor instance rendering the JSON patch (CodeEditor with props code={jsonPatch} and onChange={handleJSONChange}) is missing a stable test selector; add a data-testid prop to the CodeEditor component following the project pattern, e.g. data-testid="pricing-override-json-editor" (or similar entity-element-qualifier like pricing-override-json-editor-input) so tests can reliably target this interactive JSON editor.
363-399:⚠️ Potential issue | 🟡 MinorReopening the sheet can preserve stale JSON-only edits.
If the user closes after a JSON parse error,
formis still the shareddefaultFormState, so Line 398 becomes a no-op and the sync effect at Lines 445-452 never rebuildsjsonPatch. The next create flow opens with the previous JSON text still in the editor.♻️ Suggested fix
useEffect(() => { const wasOpen = prevOpenRef.current; prevOpenRef.current = open; if (!open || wasOpen) return; jsonEditingRef.current = false; setJSONError(undefined); + setJSONPatch(""); if (editingOverride) { const state = toFormState(editingOverride); // For provider_key scopes, provider_id is not stored in the DB (it's implicit from // the key). Derive it from providerKeyOptions so the provider selector renders and // the filtered key list shows the pre-selected key correctly. @@ - setForm(defaultFormState); + setForm({ ...defaultFormState, pricingValues: {} }); }, [open, editingOverride, scopeLock, shouldLockScope, providerKeyOptions]);Also applies to: 445-452
transports/bifrost-http/handlers/governance.go (2)
3277-3286:⚠️ Potential issue | 🟠 MajorPATCH scope changes still can't clear obsolete scope IDs.
virtual_key_id,provider_id, andprovider_key_idare plain*strings, so Go JSON decoding treats an omitted field and an explicitnullthe same way. A partial update that changes scope fromvirtual_key_provider_keytovirtual_keyorglobalhas no way to drop the old provider/provider-key identifiers beforemerged.IsValid()runs, so the request either fails or preserves stale scope data.In Go's encoding/json package, can a struct field of type *string distinguish an omitted JSON property from an explicit null during Unmarshal?Based on learnings: governance update handlers should support clearing individual fields by sending null/empty values; review input validation and JSON decoding (e.g., pointers vs values in Go structs) so partial updates can remove specific settings.
Also applies to: 3469-3504
3433-3443:⚠️ Potential issue | 🟠 MajorPricing-override writes can still leave runtime state stale.
POST/PUT persist before the runtime sync, and DELETE still swallows
DeletePricingOverridefailures. That leaves the config store and in-memory pricing resolution diverged, so callers can get a 5xx/2xx while requests keep using the old override set. At minimum, the delete path should fail with 500 too; ideally POST/PUT also add a compensating rollback or forced reconcile when the runtime update fails.🔧 Minimum fix for the DELETE path
if err := h.governanceManager.DeletePricingOverride(ctx, id); err != nil { - logger.Warn("failed to delete pricing override from memory: %v", err) + logger.Error("failed to delete pricing override from memory: %v", err) + SendError(ctx, fasthttp.StatusInternalServerError, "Pricing override deleted in database but failed to delete in-memory state") + return }Based on learnings: In transports/bifrost-http/handlers/governance.go, if the database update succeeds but the in-memory GovernanceManager reload fails, respond with HTTP 500 to the client rather than signaling success; DB and memory must stay in sync.
Also applies to: 3543-3553, 3562-3574
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3433 - 3443, The handler currently persists pricing overrides to the config store before updating in-memory state, which can leave DB and runtime diverged; update the POST/PUT and DELETE flows: in the POST/PUT handler that calls h.configStore.CreatePricingOverride (or Update) and then h.governanceManager.UpsertPricingOverride, if UpsertPricingOverride fails, perform a compensating rollback by calling h.configStore.DeletePricingOverride for the same override (log any rollback error) and return HTTP 500 to the client; in the DELETE handler, if h.configStore.DeletePricingOverride succeeds but h.governanceManager.DeletePricingOverride fails, treat it as an error—do not swallow it: return HTTP 500 and log the failure (consider attempting to re-create the DB record if needed), ensuring the functions CreatePricingOverride, UpsertPricingOverride, DeletePricingOverride and the handler methods in governance.go consistently return 500 when the in-memory sync fails.
🧹 Nitpick comments (5)
transports/bifrost-http/handlers/pricing_override_test.go (2)
100-109: Let GORM populate the audit timestamps in the seed row.
TablePricingOverridealready auto-managesCreatedAt/UpdatedAt, so setting them manually here makes the fixture less representative and keeps the extratimedependency around for no real gain. Based on learnings: In this codebase using GORM, models withCreatedAtandUpdatedAtfields tagged withgorm:"autoCreateTime"andgorm:"autoUpdateTime"are populated automatically on insert/update.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/pricing_override_test.go` around lines 100 - 109, Remove manual timestamp population from the test fixture: delete the now := time.Now().UTC() declaration and remove the CreatedAt and UpdatedAt fields from the TablePricingOverride instance named override in pricing_override_test.go so GORM can auto-populate those audit timestamps; also remove the unused time import if it becomes unused after this change.
114-122: Add a true partial-update case.This only exercises a full-body payload. Since the handler now preserves omitted top-level fields while replacing
patchwholesale, add a case that sends justnameand/orpatchand assertsscope_kind, IDs,pattern, andrequest_typesare preserved.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/pricing_override_test.go` around lines 114 - 122, Add a new test case in pricing_override_test.go that submits a true partial-update payload (e.g., send only "name" and/or only "patch" in the request body instead of the full object) to exercise preservation logic; call the same handler exercised by the existing test (reuse the same request path and test setup around the existing body variable) and assert that omitted top-level fields — scope_kind, id(s), pattern, and request_types — remain unchanged while the provided fields (name and/or patch) are updated, and verify that when patch is provided it replaces the previous patch wholesale. Ensure the test name clearly reflects "partial update" and add assertions checking both preserved fields and updated fields.framework/modelcatalog/pricing.go (1)
26-35:nilscopes still apply some overrides.
CalculateCostturnsnilinto a zero-valuePricingLookupScopes, andresolvePricing()later backfillsscopes.Providerbefore callingapplyPricingOverrides(). So the comment currently promises a stronger “no overrides” behavior than the implementation provides. Either bypass override application whenscopes == nil, or reword this to “zero-value scopes.”🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/pricing.go` around lines 26 - 35, The current CalculateCost coerces a nil scopes into a zero-value PricingLookupScopes which allows resolvePricing() to backfill and apply overrides; change CalculateCost to preserve the original nil intent by tracking whether the incoming scopes pointer was nil (e.g., originalNil := scopes == nil) and, if originalNil is true, skip calling resolvePricing()/applyPricingOverrides() (or otherwise bypass override application) so no scoped overrides are applied when the caller passed nil; update references to the local variable handling (keep using s or a pointer) accordingly to avoid altering other logic.ui/components/sidebar.tsx (1)
197-200: Centralize the custom-pricing route matcher.The
/workspace/custom-pricingexception now lives in three helpers. Pulling that into one shared matcher would reduce the chance of active-state and auto-expand behavior drifting the next time a pricing route changes.Also applies to: 804-807, 934-938
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/components/sidebar.tsx` around lines 197 - 200, The logic treating "/workspace/custom-pricing" as a special-case is duplicated; extract a single shared matcher function (e.g., export function matchWorkspaceRoute(pathname: string, url: string)) and replace the inline isRouteMatch implementation with a call to that function; the shared matcher should return exact equality for "/workspace/custom-pricing" and otherwise use startsWith, and then update the other route-active/auto-expand helpers that currently replicate this logic to call the new matchWorkspaceRoute to keep behavior consistent for pathname and "/workspace/custom-pricing".framework/modelcatalog/overrides_test.go (1)
412-448: Add the two combined scopes to this precedence table.This test stops at
virtual_key, sovirtual_key_providerandvirtual_key_provider_key—the two highest-precedence cases introduced by this feature—aren’t pinned here.Also applies to: 458-491
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides_test.go` around lines 412 - 448, The precedence table passed to mc.SetPricingOverrides stops at ScopeKindVirtualKey; add the two missing higher-precedence combined scopes (virtual_key_provider and virtual_key_provider_key) to the slice so tests cover the new cases: create TablePricingOverride entries with ScopeKind values matching the new constants (e.g., ScopeKindVirtualKeyProvider and ScopeKindVirtualKeyProviderKey), set the appropriate scope IDs using existing variables (virtualKeyScopeID + providerScopeID and virtualKeyScopeID + providerKeyScopeID), use MatchTypeExact and the same Pattern/RequestTypes as the other entries, and give them distinct PricingPatchJSON values so precedence behavior is asserted; ensure these new entries are added alongside the existing ones in the mc.SetPricingOverrides call used in the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/providers/custom-pricing.mdx`:
- Around line 11-15: The overview contradicts the documented contract for
request_types; update the "Request type filtering" bullet to match the required
`request_types` contract by stating that `request_types` is mandatory and must
contain at least one value (it cannot be left empty to match all), or
alternatively change the `request_types` contract elsewhere to make it
optional—choose one approach and make both the overview text and the
`request_types` documentation consistent (referencing the `request_types` field
name to locate and update the content).
- Line 313: The docs advertise output_cost_per_image_premium_image but the
engine's pricing path doesn't branch on a premium-image signal, so either
remove/mark this field as unsupported in docs or implement support: update the
billing/pricing logic (the engine's pricing path) to detect the premium-image
indicator (e.g., a request flag/metadata or a new field check) and, when
present, use output_cost_per_image_premium_image instead of the base image
billing path; ensure the docs for output_cost_per_image_premium_image reflect
the chosen approach (removed/marked-unavailable until implemented, or documented
with how the premium-image signal is provided).
In `@framework/configstore/migrations.go`:
- Around line 5137-5148: In migrationMakeBasePricingColumnsNullable, after
obtaining tx (and before/alongside altering the columns) add a backfill that
updates legacy zero sentinels to NULL on the TableModelPricing rows so old NOT
NULL zeros become “unset”; e.g. run updates via
tx.Model(&tables.TableModelPricing{}).Where("input_cost_per_token = ?",
0).Update("input_cost_per_token", nil) and similarly for "output_cost_per_token"
(and combine into one query if desired) so zeros are cleared before/when you
call m.AlterColumn for InputCostPerToken and OutputCostPerToken.
In `@framework/modelcatalog/overrides_test.go`:
- Around line 156-158: The test is comparing float64 to *float64 because
resolvePricing/applyPricingOverrides return pointer-valued cost fields; update
the assertions to dereference those pointers (e.g., assert.Equal(t, 7.0,
*pricing.InputCostPerToken)) and add nil checks where appropriate (e.g.,
require.NotNil(t, pricing.InputCostPerToken)) before dereferencing; apply the
same change to the other failing assertions that reference pointer cost fields
at the other locations (lines referenced for similar checks).
In `@framework/modelcatalog/pricing.go`:
- Around line 768-785: The current flow returns nil when getBasePricing(model,
provider, requestType) misses, preventing applyPricingOverrides from applying
override-only pricing; update the logic in the block around
getBasePricing/getBasePricing(deployment,...) so that if no base pricing exists
you still call mc.applyPricingOverrides(model, requestType, emptyBase, scopes)
where emptyBase is an empty TableModelPricing (or equivalent zero-value) to
allow overrides to resolve; keep the existing branch that tries the deployment
name first, but if both miss invoke applyPricingOverrides with an empty base
before returning nil so override-only entries can produce a result.
In `@transports/bifrost-http/lib/config.go`:
- Around line 450-457: The current startup silently ignores failures from
config.ModelCatalog.SetPricingOverrides, causing requests to run with incorrect
pricing when ConfigStore is nil; change the handling so failures are treated as
startup errors instead of a warning—replace the logger.Warn call after
config.ModelCatalog.SetPricingOverrides with code that returns or propagates the
error (or otherwise fails startup, e.g., logger.Error + return err) from the
enclosing initialization function so the caller can abort startup or handle the
failure; ensure you reference ConfigStore, ModelCatalog, GovernanceConfig and
SetPricingOverrides when locating and updating the code path.
In `@transports/bifrost-http/server/server.go`:
- Around line 812-825: The methods UpsertPricingOverride and
DeletePricingOverride on BifrostHTTPServer currently return nil when s.Config or
s.Config.ModelCatalog is nil, masking failures; update both functions to detect
a missing ModelCatalog and return a descriptive error instead of nil (e.g.,
"model catalog not initialized" or similar) so the caller knows the in-process
catalog wasn't updated; alternatively, initialize s.Config.ModelCatalog before
calling UpsertPricingOverrides/DeletePricingOverride if initialization logic
exists—ensure the error path references the BifrostHTTPServer, Config, and
ModelCatalog symbols so the caller can surface the split-brain condition.
In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx`:
- Around line 53-57: The current useEffect rebuilds activeFields from non-empty
values on every values change, which removes rows mid-edit; change the effect so
it only adds keys for non-empty values but does not remove existing active keys
(i.e., merge new non-empty keys into the existing activeFields set instead of
replacing it), or alternatively guard the sync to run only on external
loads/resets (use an isInitialLoad/valuesVersion flag). Update the effect that
references useEffect, setActiveFields, PRICING_FIELDS, and values to either: 1)
compute newKeys = PRICING_FIELDS.filter(...non-empty...).forEach(k =>
activeFields.add(k)) and call setActiveFields with the merged set, preserving
empty rows until a user-triggered remove; or 2) run the full replace logic only
when an explicit load/reset indicator changes. Ensure removal of empty rows
remains driven by the explicit remove button logic.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 76-80: The branch handling "provider_key" and
"virtual_key_provider_key" should fall back to the override's provider_id when
key metadata is missing: when computing the provider for an override (look for
provider_key_id via override.provider_key_id and keyProviderMap.get(keyID)), if
keyProviderMap.get(keyID) is undefined or providerMap.get(...) returns
undefined, try using override.provider_id and
providerMap.get(override.provider_id) before returning "-" — update the logic in
the switch case that references providerMap, keyProviderMap,
override.provider_key_id, and override.provider_id to prefer the resolved
provider from keyProviderMap but fall back to the explicit override.provider_id
(and its mapping in providerMap) if the key metadata is stale or unavailable.
In `@ui/lib/types/governance.ts`:
- Around line 447-456: CreatePricingOverrideRequest currently has an optional,
untyped request_types which lets invalid payloads through; make request_types a
required field typed as an explicit union array (declare or import
PricingOverrideRequestType and change request_types?: string[] to request_types:
PricingOverrideRequestType[]) so the compiler enforces valid request types at
compile time and aligns the UI contract with backend/docs; update any usages or
imports of CreatePricingOverrideRequest to satisfy the new required field.
---
Duplicate comments:
In `@core/schemas/tracer.go`:
- Line 71: The Tracer interface was made breaking by changing the exported
method PopulateLLMResponseAttributes to require *BifrostContext; revert the
public method signature to use context.Context
(PopulateLLMResponseAttributes(ctx context.Context, handle SpanHandle, resp
*BifrostResponse, err *BifrostError)) so external implementations remain
compatible, and add a new additive, Bifrost-specific hook (e.g.
PopulateLLMResponseAttributesWithBifrost(ctx *BifrostContext, handle SpanHandle,
resp *BifrostResponse, err *BifrostError)) for internal consumers that need
Bifrost internals; update both occurrences of the signature in the Tracer
interface and any internal callers to call the new Bifrost-specific method when
they have a *BifrostContext.
In `@docs/openapi/schemas/management/governance.yaml`:
- Around line 1160-1207: The CreatePricingOverrideRequest schema currently lists
virtual_key_id, provider_id, and provider_key_id only in prose; update the
schema to use oneOf branches keyed by scope_kind so each scope variant enforces
its required IDs: create oneOf alternatives for scope_kind values (e.g., global,
provider, provider_key, virtual_key, virtual_key_provider,
virtual_key_provider_key) mirroring RoutingRule's pattern, keep common
properties (name, match_type, pattern, request_types, patch) in each branch, and
add required arrays inside each branch to require virtual_key_id for
virtual_key* scopes, provider_id for provider and virtual_key_provider scopes,
and provider_key_id for provider_key and virtual_key_provider_key scopes so the
OpenAPI validator enforces correct IDs.
In `@framework/configstore/rdb.go`:
- Line 1320: The query ordering currently uses Order("created_at ASC") when
loading pricing overrides (see the call pattern q.Order("created_at
ASC").Find(&overrides).Error) which can produce non-deterministic order for rows
sharing the same timestamp; update the Order clause to include a deterministic
tie-breaker (e.g., append the primary key column such as "id ASC" or the UUID
column: Order("created_at ASC, id ASC")) so identical created_at values have
stable ordering, and apply the same change to the other occurrence noted around
the second call (the Order call near lines 1365-1366) so both reads use the
deterministic compound sort.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3433-3443: The handler currently persists pricing overrides to the
config store before updating in-memory state, which can leave DB and runtime
diverged; update the POST/PUT and DELETE flows: in the POST/PUT handler that
calls h.configStore.CreatePricingOverride (or Update) and then
h.governanceManager.UpsertPricingOverride, if UpsertPricingOverride fails,
perform a compensating rollback by calling h.configStore.DeletePricingOverride
for the same override (log any rollback error) and return HTTP 500 to the
client; in the DELETE handler, if h.configStore.DeletePricingOverride succeeds
but h.governanceManager.DeletePricingOverride fails, treat it as an error—do not
swallow it: return HTTP 500 and log the failure (consider attempting to
re-create the DB record if needed), ensuring the functions
CreatePricingOverride, UpsertPricingOverride, DeletePricingOverride and the
handler methods in governance.go consistently return 500 when the in-memory sync
fails.
In `@transports/bifrost-http/handlers/providers.go`:
- Around line 315-326: The handler currently unmarshals into a typed payload
(variable payload) which ignores unknown top-level keys so legacy
"pricing_overrides" gets silently dropped; before calling
sonic.Unmarshal(ctx.PostBody(), &payload) add a raw-body validation that scans
the JSON text for the top-level key "pricing_overrides" (or use a shared helper
function reused by addProvider) and return a 400 via SendError if that key is
present, so PUT /api/providers/{provider} rejects legacy overrides instead of
discarding them; implement the same helper and call-site in addProvider to
ensure consistent behavior across both endpoints.
In `@transports/config.schema.json`:
- Around line 3032-3103: The pricing_override schema allows invalid configs;
tighten it by (1) adding conditional required fields on
pricing_override.scope_kind so provider_id is required when scope_kind contains
"provider", provider_key_id is required for "provider_key" or
"virtual_key_provider_key", and virtual_key_id is required for any
"virtual_key*" scope (use "if/then" based on scope_kind values referencing
pricing_override.scope_kind and the specific id properties), (2) constrain
pricing_override.pattern based on pricing_override.match_type so that when
match_type == "exact" pattern cannot contain "*" and when match_type ==
"wildcard" pattern must be a non-empty string that ends with a single trailing
"*" and contains no other "*" (use pattern/regex validation tied to match_type
via conditional subschemas), and (3) tighten request_types to only allow the
defined request type enum by referencing pricing_override_request_type for items
instead of free strings; update the schema for pricing_override to include these
conditional/regex rules and ensure existing required fields remain.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 717-735: The Provider key Select should be disabled when provider
options are unavailable; update the JSX around the Select (the element using
value={form.providerKeyID || "__none__"} and onValueChange={...}) to pass a
disabled prop when providerScopedKeyOptions is empty or when the providers query
is loading/errored (use the relevant loading/error flags from the providers
query), and ensure setForm no-ops or preserves providerKeyID until options are
present so edits don't silently clear the key; check symbols
providerScopedKeyOptions, form.providerID, form.providerKeyID, Select, and
setForm to implement this guard.
- Around line 844-853: The CodeEditor instance rendering the JSON patch
(CodeEditor with props code={jsonPatch} and onChange={handleJSONChange}) is
missing a stable test selector; add a data-testid prop to the CodeEditor
component following the project pattern, e.g.
data-testid="pricing-override-json-editor" (or similar entity-element-qualifier
like pricing-override-json-editor-input) so tests can reliably target this
interactive JSON editor.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 643-660: The patch only uses matchesQuery (which checks a subset
of fields) and returns early when index === -1, causing lists to fall out of
sync; update the updateQueryData handler for getPricingOverrides to use the full
membership predicate that getPricingOverrides uses (including the search text
and all scope/provider/providerKey fields) instead of the current matchesQuery,
and change the index handling so that if index === -1 and the full-membership
predicate is true you insert the updated item into draft.pricing_overrides and
increment draft.count and draft.total_count appropriately, while if index !== -1
and the item no longer matches you splice it out and decrement counts
(preserving the existing splice/dec count logic).
---
Nitpick comments:
In `@framework/modelcatalog/overrides_test.go`:
- Around line 412-448: The precedence table passed to mc.SetPricingOverrides
stops at ScopeKindVirtualKey; add the two missing higher-precedence combined
scopes (virtual_key_provider and virtual_key_provider_key) to the slice so tests
cover the new cases: create TablePricingOverride entries with ScopeKind values
matching the new constants (e.g., ScopeKindVirtualKeyProvider and
ScopeKindVirtualKeyProviderKey), set the appropriate scope IDs using existing
variables (virtualKeyScopeID + providerScopeID and virtualKeyScopeID +
providerKeyScopeID), use MatchTypeExact and the same Pattern/RequestTypes as the
other entries, and give them distinct PricingPatchJSON values so precedence
behavior is asserted; ensure these new entries are added alongside the existing
ones in the mc.SetPricingOverrides call used in the test.
In `@framework/modelcatalog/pricing.go`:
- Around line 26-35: The current CalculateCost coerces a nil scopes into a
zero-value PricingLookupScopes which allows resolvePricing() to backfill and
apply overrides; change CalculateCost to preserve the original nil intent by
tracking whether the incoming scopes pointer was nil (e.g., originalNil :=
scopes == nil) and, if originalNil is true, skip calling
resolvePricing()/applyPricingOverrides() (or otherwise bypass override
application) so no scoped overrides are applied when the caller passed nil;
update references to the local variable handling (keep using s or a pointer)
accordingly to avoid altering other logic.
In `@transports/bifrost-http/handlers/pricing_override_test.go`:
- Around line 100-109: Remove manual timestamp population from the test fixture:
delete the now := time.Now().UTC() declaration and remove the CreatedAt and
UpdatedAt fields from the TablePricingOverride instance named override in
pricing_override_test.go so GORM can auto-populate those audit timestamps; also
remove the unused time import if it becomes unused after this change.
- Around line 114-122: Add a new test case in pricing_override_test.go that
submits a true partial-update payload (e.g., send only "name" and/or only
"patch" in the request body instead of the full object) to exercise preservation
logic; call the same handler exercised by the existing test (reuse the same
request path and test setup around the existing body variable) and assert that
omitted top-level fields — scope_kind, id(s), pattern, and request_types —
remain unchanged while the provided fields (name and/or patch) are updated, and
verify that when patch is provided it replaces the previous patch wholesale.
Ensure the test name clearly reflects "partial update" and add assertions
checking both preserved fields and updated fields.
In `@ui/components/sidebar.tsx`:
- Around line 197-200: The logic treating "/workspace/custom-pricing" as a
special-case is duplicated; extract a single shared matcher function (e.g.,
export function matchWorkspaceRoute(pathname: string, url: string)) and replace
the inline isRouteMatch implementation with a call to that function; the shared
matcher should return exact equality for "/workspace/custom-pricing" and
otherwise use startsWith, and then update the other route-active/auto-expand
helpers that currently replicate this logic to call the new matchWorkspaceRoute
to keep behavior consistent for pathname and "/workspace/custom-pricing".
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ec07bd69-3741-4aaa-bdd5-da3e4dbfc45f
⛔ Files ignored due to path filters (3)
cli/go.sumis excluded by!**/*.sumdocs/media/ui-custom-pricing-form.pngis excluded by!**/*.pngdocs/media/ui-custom-pricing-table.pngis excluded by!**/*.png
📒 Files selected for processing (61)
cli/go.modcore/bifrost.gocore/providers/utils/utils.gocore/schemas/provider.gocore/schemas/tracer.godocs/architecture/framework/model-catalog.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/openapi.yamldocs/openapi/paths/management/governance.yamldocs/openapi/schemas/management/governance.yamldocs/providers/custom-pricing.mdxexamples/configs/withpricingoverridesnostore/config.jsonexamples/configs/withpricingoverridessqlite/config.jsonframework/configstore/clientconfig.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/store.goframework/configstore/tables/modelpricing.goframework/configstore/tables/pricingoverride.goframework/configstore/tables/provider.goframework/logstore/tables.goframework/modelcatalog/main.goframework/modelcatalog/main_test.goframework/modelcatalog/overrides.goframework/modelcatalog/overrides_test.goframework/modelcatalog/pricing.goframework/modelcatalog/pricing_test.goframework/modelcatalog/utils.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/images.goframework/streaming/responses.goframework/streaming/transcription.goframework/tracing/tracer.goplugins/governance/main.goplugins/logging/main.goplugins/logging/operations.goplugins/telemetry/main.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/pricing_override_test.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/server/server.gotransports/config.schema.jsonui/app/workspace/custom-pricing/overrides/page.tsxui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsxui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsxui/app/workspace/custom-pricing/overrides/pricingOverridesEmptyState.tsxui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsxui/app/workspace/providers/fragments/index.tsui/app/workspace/providers/fragments/pricingOverridesFormFragment.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/sidebar.tsxui/lib/store/apis/baseApi.tsui/lib/store/apis/governanceApi.tsui/lib/types/config.tsui/lib/types/governance.tsui/lib/types/schemas.ts
💤 Files with no reviewable changes (4)
- ui/app/workspace/providers/fragments/index.ts
- ui/app/workspace/providers/fragments/pricingOverridesFormFragment.tsx
- ui/lib/types/config.ts
- ui/lib/types/schemas.ts
Pratham-Mishra04
force-pushed
the
feat/scoped-pricing-overrides
branch
from
9369f33 to
27abeb6
Compare
Collaborator
|
@coderabbitai full-review |
Contributor
|
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
core/schemas/provider.go (1)
401-410:⚠️ Potential issue | 🟠 MajorAdd explicit migration or rejection for legacy provider-level
pricing_overridesconfigs.Removing
pricing_overridesfromProviderConfigis correct for the new governance-scoped model, but the config loader uses standardjson.Unmarshal, which silently ignores unknown JSON fields. This means prerelease configs withproviders[].pricing_overrideswill deserialize without error, but those overrides will be dropped—breaking billing silently.Either configure the loader to reject unknown fields on the provider block, or add an explicit migration step that maps legacy
providers[].pricing_overridesinto the new governance scope.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/schemas/provider.go` around lines 401 - 410, The ProviderConfig struct no longer includes pricing_overrides so legacy JSON with providers[].pricing_overrides will be silently dropped; update the config loading path to either (A) reject unknown fields for provider blocks by decoding provider JSON with json.Decoder{DisallowUnknownFields:true} when populating ProviderConfig (e.g., in your config loader / UnmarshalConfig function) and return a clear error mentioning providers[].pricing_overrides, or (B) add an explicit migration step run after unmarshalling that detects a legacy providers[].pricing_overrides key and maps its values into the new governance-scoped structure (e.g., GovernanceConfig.PricingOverrides) and logs the migration; implement one of these in the code paths that create or validate ProviderConfig so legacy overrides are not silently dropped.framework/modelcatalog/main.go (2)
221-225:⚠️ Potential issue | 🟠 MajorStart
syncModelParametersonly after init can no longer fail.Line 259 now makes override loading fatal, but both branches launch the async model-parameter sync earlier. If override loading fails,
Initreturns an error while that goroutine keeps running against a catalog the caller never received.Also applies to: 243-247, 259-261
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/main.go` around lines 221 - 225, The background goroutine calling mc.syncModelParameters(ctx) is started before Init can fail (when override loading became fatal), so it may run against an uninitialized catalog if Init returns an error; fix by starting that goroutine only after Init has completed successfully (i.e., after the fatal override-loading path returns nil) or by tying it to a cancellation that is triggered when Init fails. Concretely, move the go func that calls mc.syncModelParameters(ctx) to a point after Init finishes the override loading check (the Init function) or ensure Init cancels the passed ctx before returning an error so the launched goroutine exits; reference the Init function and the syncModelParameters method and adjust where the mc.logger.Warn/logging and goroutine launch occur accordingly.
149-176:⚠️ Potential issue | 🟠 MajorDecoder does not handle plain float64 format for
search_context_cost_per_query.The UnmarshalJSON comment states the method should handle both "a plain float64 or a tiered object," but the implementation only supports the tiered object form. The
rawstruct declaresSearchContextCostPerQueryas*struct{Low, Medium, High}, so unmarshaling a plain JSON number will fail instead of populatingp.SearchContextCostPerQuery. Add fallback logic to unmarshal as a plain float64 when the tiered object structure is not present.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/main.go` around lines 149 - 176, The decoder for PricingEntry.UnmarshalJSON only handles the tiered object form and ignores a plain float64; modify the method to also try decoding search_context_cost_per_query as a plain number when raw.SearchContextCostPerQuery is nil: after the existing sonic.Unmarshal into raw (the struct with SearchContextCostPerQuery *struct{Low,Medium,High}), if raw.SearchContextCostPerQuery == nil then unmarshal the original data (or the raw JSON) into a tiny auxiliary struct with SearchContextCostPerQuery *float64 and, if that yields a non-nil value, set p.SearchContextCostPerQuery to that value (assign to p.SearchContextCostPerQuery). Ensure you reference and set p.SearchContextCostPerQuery and leave the existing tiered-selection logic intact for the tiered case.
♻️ Duplicate comments (16)
ui/lib/types/governance.ts (2)
402-417:⚠️ Potential issue | 🟠 Major
PricingOverridePatchdrifted from the supported image-pricing contract.This type still exposes premium-image variants that the docs for this stack no longer advertise, and it omits the documented
output_cost_per_image_above_2048_and_2048_pixelsandoutput_cost_per_image_above_4096_and_4096_pixelsfields. That leaves the UI able to type unsupported payloads while lacking some supported ones.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/types/governance.ts` around lines 402 - 417, The PricingOverridePatch type has drifted from the supported image-pricing contract: remove the deprecated premium-image fields (e.g., output_cost_per_image_premium_image and the combined premium variants output_cost_per_image_above_512_and_512_pixels_and_premium_image and output_cost_per_image_above_1024_and_1024_pixels_and_premium_image) and add the missing documented fields output_cost_per_image_above_2048_and_2048_pixels and output_cost_per_image_above_4096_and_4096_pixels so the PricingOverridePatch type accurately matches the current contract; update the type definition where PricingOverridePatch is declared in ui/lib/types/governance.ts and ensure any related optional input_cost/output_cost properties follow the same naming convention.
431-468:⚠️ Potential issue | 🟡 MinorKeep
request_typesstrongly typed on read/update models too.
CreatePricingOverrideRequestusesRequestType[], butPricingOverrideandUpdatePricingOverrideRequestfall back to rawstring[]and the read model even makes the field optional. That drops compile-time validation as soon as an override is loaded back into an edit flow.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/types/governance.ts` around lines 431 - 468, The read/update models use raw string[] for request_types which loses compile-time safety; update the PricingOverride interface and the UpdatePricingOverrideRequest type to use the RequestType union instead of string[] (i.e., change PricingOverride.request_types to RequestType[] (preserving optionality if intended) and UpdatePricingOverrideRequest.request_types to RequestType[]), ensuring all references to request_types in those interfaces match the CreatePricingOverrideRequest’s RequestType[] usage so loaded overrides retain strong typing.core/schemas/tracer.go (1)
69-71:⚠️ Potential issue | 🟠 MajorKeep the exported tracer hook source-compatible.
Changing
Tracer.PopulateLLMResponseAttributesto require*BifrostContextbreaks every external tracer implementation and couples the public interface to a concrete Bifrost type. If scoped pricing needs extra state, keep the old signature and add a Bifrost-specific escape hatch instead.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/schemas/tracer.go` around lines 69 - 71, The change made Tracer.PopulateLLMResponseAttributes to take *BifrostContext which breaks source compatibility; revert PopulateLLMResponseAttributes to its original public signature PopulateLLMResponseAttributes(handle SpanHandle, resp *BifrostResponse, err *BifrostError) so existing tracers continue to compile, and add a Bifrost-specific escape hatch (for example a new method PopulateLLMResponseAttributesWithContext(ctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError) or a new TracerWithContext interface) that callers inside Bifrost can use when context is required; update internal callers to use the new Bifrost-specific method while leaving the public Tracer API unchanged.transports/bifrost-http/handlers/providers.go (1)
177-190:⚠️ Potential issue | 🟠 MajorReject legacy
pricing_overridesinstead of silently ignoring it.Both handlers now unmarshal into payload structs that do not contain this field, and both decoders ignore unknown top-level keys. In this stack provider-scoped overrides are gone, so a client can still send the legacy payload and get a 200 while the override is discarded. Please fail fast with a 400 that points callers at the governance pricing-overrides API.
🐛 Suggested guard
func (h *ProviderHandler) addProvider(ctx *fasthttp.RequestCtx) { + if err := rejectUnsupportedProviderFields(ctx.PostBody()); err != nil { + SendError(ctx, fasthttp.StatusBadRequest, err.Error()) + return + } if err := json.Unmarshal(ctx.PostBody(), &payload); err != nil { SendError(ctx, fasthttp.StatusBadRequest, fmt.Sprintf("Invalid JSON: %v", err)) return } @@ - if err := sonic.Unmarshal(ctx.PostBody(), &payload); err != nil { + if err := rejectUnsupportedProviderFields(ctx.PostBody()); err != nil { + SendError(ctx, fasthttp.StatusBadRequest, err.Error()) + return + } + if err := sonic.Unmarshal(ctx.PostBody(), &payload); err != nil { SendError(ctx, fasthttp.StatusBadRequest, fmt.Sprintf("Invalid JSON: %v", err)) return }func rejectUnsupportedProviderFields(body []byte) error { var raw map[string]json.RawMessage if err := json.Unmarshal(body, &raw); err != nil { return nil } if _, ok := raw["pricing_overrides"]; ok { return fmt.Errorf("pricing_overrides is not a supported provider field; use /api/governance/pricing-overrides instead") } return nil }Also applies to: 315-328
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/providers.go` around lines 177 - 190, The handlers currently unmarshal request bodies into the payload struct and silently ignore unknown top-level keys like pricing_overrides; add a pre-unmarshal guard function (e.g., rejectUnsupportedProviderFields) that inspects the raw request body for the "pricing_overrides" key and, if present, returns an error; call this guard at the start of the handler (before json.Unmarshal of payload) and, on error, SendError with fasthttp.StatusBadRequest and a message directing the caller to /api/governance/pricing-overrides; ensure you apply the same change to both handler sites (the existing payload-unmarshal blocks around the payload struct and the other occurrence mentioned).framework/configstore/clientconfig.go (1)
970-983:⚠️ Potential issue | 🟠 MajorHash request types from the parsed slice, not
RequestTypesJSON.
RequestTypesJSONis only materialized during save and preserves raw ordering. Any reconcile/hash path that operates on an in-memoryTablePricingOverridecan therefore miss realrequest_typesedits entirely, while semantically equivalent reorderings hash differently.💡 Minimal fix
func GeneratePricingOverrideHash(p tables.TablePricingOverride) (string, error) { hash := sha256.New() hash.Write([]byte(p.ID)) hash.Write([]byte(p.Name)) hash.Write([]byte(p.ScopeKind)) hash.Write([]byte(derefStr(p.VirtualKeyID))) hash.Write([]byte(derefStr(p.ProviderID))) hash.Write([]byte(derefStr(p.ProviderKeyID))) hash.Write([]byte(p.MatchType)) hash.Write([]byte(p.Pattern)) - hash.Write([]byte(p.RequestTypesJSON)) + requestTypes := append([]schemas.RequestType{}, p.RequestTypes...) + if len(requestTypes) == 0 && p.RequestTypesJSON != "" { + if err := json.Unmarshal([]byte(p.RequestTypesJSON), &requestTypes); err != nil { + return "", err + } + } + sort.Slice(requestTypes, func(i, j int) bool { return requestTypes[i] < requestTypes[j] }) + data, err := sonic.Marshal(requestTypes) + if err != nil { + return "", err + } + hash.Write(data) hash.Write([]byte(p.PricingPatchJSON)) return hex.EncodeToString(hash.Sum(nil)), nil }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/clientconfig.go` around lines 970 - 983, GeneratePricingOverrideHash is currently hashing the raw RequestTypesJSON which can miss in-memory edits and yields order-sensitive hashes; replace the RequestTypesJSON usage with the parsed slice on the struct (e.g., p.RequestTypes or whatever parsed field holds []string) and hash each element instead; to make hashing order-insensitive, make a local copy, sort it (using sort.Strings), then iterate and hash each entry (handling nil/empty safely) so equivalent sets produce the same hash. Reference: GeneratePricingOverrideHash, TablePricingOverride, RequestTypesJSON -> RequestTypes.framework/modelcatalog/overrides_test.go (1)
497-502:⚠️ Potential issue | 🟠 MajorDereference the patched rate before asserting.
patched.InputCostPerTokenis a*float64, so Line 501 currently compares different types and this test will fail once it runs.✅ Minimal fix
t.Run(tc.name, func(t *testing.T) { patched, applied := mc.applyPricingOverrides("gpt-5-nano", schemas.ChatCompletionRequest, base, tc.scopes) require.True(t, applied) - assert.Equal(t, tc.expected, patched.InputCostPerToken) + require.NotNil(t, patched.InputCostPerToken) + assert.Equal(t, tc.expected, *patched.InputCostPerToken) })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides_test.go` around lines 497 - 502, The test currently compares tc.expected to patched.InputCostPerToken (a *float64); change the assertion to compare tc.expected to the dereferenced value (e.g., assert.Equal(t, tc.expected, *patched.InputCostPerToken)) and ensure patched and patched.InputCostPerToken are non-nil before dereferencing in the test around applyPricingOverrides.docs/openapi/schemas/management/governance.yaml (1)
1160-1207:⚠️ Potential issue | 🟠 MajorMake
CreatePricingOverrideRequestself-validating.
virtual_key_id,provider_id, andprovider_key_idare only “required” in descriptions here. Generated clients can submit payloads that pass OpenAPI validation and still get rejected by the handler. MirrorRoutingRuleand model this withoneOfbranches keyed byscope_kind.🧩 Example shape
CreatePricingOverrideRequest: type: object - required: - - name - - scope_kind - - match_type - - pattern - - request_types + oneOf: + - type: object + properties: + scope_kind: + type: string + enum: [global] + required: [name, scope_kind, match_type, pattern, request_types] + - type: object + properties: + scope_kind: + type: string + enum: [provider] + provider_id: + type: string + required: [name, scope_kind, provider_id, match_type, pattern, request_types] + # ...additional branches for provider_key, virtual_key, virtual_key_provider, virtual_key_provider_key🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/openapi/schemas/management/governance.yaml` around lines 1160 - 1207, CreatePricingOverrideRequest must enforce required fields per scope_kind: refactor the schema for CreatePricingOverrideRequest to use oneOf (modeled like RoutingRule) with separate branch schemas for each scope_kind value (global, provider, provider_key, virtual_key, virtual_key_provider, virtual_key_provider_key) where each branch includes the common properties (name, match_type, pattern, request_types, patch, scope_kind) and declares virtual_key_id, provider_id, and provider_key_id as required only on the branches that need them (e.g., provider branch requires provider_id, provider_key branch requires provider_key_id, virtual_key_provider_key branch requires virtual_key_id, provider_id and provider_key_id as appropriate); ensure scope_kind is fixed in each branch (enum with a single value) so generated clients must include the correct IDs for the selected scope.ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
76-80:⚠️ Potential issue | 🟡 MinorFallback to
override.provider_idwhen key metadata is missing.
provider_keyandvirtual_key_provider_keyrows already carryprovider_id, but this branch only resolves the provider throughprovider_key_id. If the key lookup is stale or unavailable, the table renders-even though the scoped provider is still known.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 76 - 80, The provider lookup for the "provider_key" and "virtual_key_provider_key" cases only uses override.provider_key_id and keyProviderMap, causing a "-" when key metadata is missing; update the return to fall back to override.provider_id: compute keyID = override.provider_key_id || "" as before, then try providerMap.get(keyProviderMap.get(keyID) || override.provider_id || "") and if that fails fall back to keyProviderMap.get(keyID) || override.provider_id || "-" so the scoped provider_id is used when key metadata is stale or absent.transports/bifrost-http/handlers/governance.go (1)
3433-3443:⚠️ Potential issue | 🟠 MajorDon't return success while pricing overrides are diverged between DB and memory.
Create/update persist the row before the in-memory upsert runs, and delete still treats the in-memory removal as non-fatal. If any of those sync steps fail, runtime pricing keeps using stale overrides while the persisted state says otherwise. Make the DB mutation and governance sync one failure boundary, or add compensating rollback/reload before acknowledging success.
Based on learnings: if the database update succeeds but the in-memory GovernanceManager reload fails, respond with HTTP 500 rather than signaling success; the system relies on in-memory state for internal operations, so DB and memory must stay in sync.
Also applies to: 3543-3553, 3562-3574
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3433 - 3443, The DB write (configStore.CreatePricingOverride / UpdatePricingOverride / DeletePricingOverride) and the in-memory sync (governanceManager.UpsertPricingOverride / RemovePricingOverride) must be atomic from the client perspective: if the in-memory upsert/reload fails, do not return success. Modify the handler so that after a successful DB mutation you call governanceManager.UpsertPricingOverride (or reload the manager) and if that call fails you either (a) rollback the DB change (call the corresponding configStore.DeletePricingOverride or restore previous row) or (b) trigger a full governanceManager reload from the DB before returning; in either case return HTTP 500 (use SendError) and log the error instead of returning success. Ensure this same pattern is applied to the create, update and delete flows handled by CreatePricingOverride/UpsertPricingOverride and their counterparts so DB and in-memory state never diverge.ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx (1)
53-57:⚠️ Potential issue | 🟡 MinorKeep cleared rows mounted until the explicit remove action.
Rebuilding
activeFieldsfrom non-emptyvalueson every change removes the row as soon as the last character is deleted. That makes “clear, then type the new value” brittle even though removal already has its own button. Limit this full reset to external loads/resets, or preserve active empty rows untildeactivateField()runs.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx` around lines 53 - 57, The current useEffect that calls setActiveFields(rebuild) removes rows as soon as a value becomes empty; change it to only add non-empty keys (union with the existing activeFields) so empty-but-still-active rows remain mounted until deactivateField() is explicitly called, e.g. replace the rebuild logic with setActiveFields(prev => new Set([...prev, ...nonEmptyKeys])), and keep a separate effect (or trigger) that fully resets activeFields when an external load/reset occurs (listen to whatever “external load” flag/prop you have) so that external loads still replace the set.transports/config.schema.json (1)
3032-3103:⚠️ Potential issue | 🟠 MajorThe schema still accepts invalid scoped overrides.
Line 3066 still allows an empty
pattern, Line 3074 accepts arbitrary strings instead of the constrained request-type enum, and there are still noscope_kindconditionals to require the right IDs. Invalid configs will pass schema validation and fail much later during bootstrap/runtime.Suggested fix
"pricing_override": { "type": "object", "properties": { ... "pattern": { "type": "string", + "minLength": 1, "description": "Model name pattern to match (exact name or wildcard prefix ending with *)" }, "request_types": { "type": "array", "minItems": 1, "items": { - "type": "string" + "$ref": "#/$defs/pricing_override_request_type" } }, ... }, + "allOf": [ + { + "if": { "properties": { "scope_kind": { "const": "provider" } } }, + "then": { "required": ["provider_id"] } + }, + { + "if": { "properties": { "scope_kind": { "const": "provider_key" } } }, + "then": { "required": ["provider_key_id"] } + }, + { + "if": { "properties": { "scope_kind": { "const": "virtual_key" } } }, + "then": { "required": ["virtual_key_id"] } + }, + { + "if": { "properties": { "scope_kind": { "const": "virtual_key_provider" } } }, + "then": { "required": ["virtual_key_id", "provider_id"] } + }, + { + "if": { "properties": { "scope_kind": { "const": "virtual_key_provider_key" } } }, + "then": { "required": ["virtual_key_id", "provider_key_id"] } + }, + { + "if": { "properties": { "match_type": { "const": "wildcard" } } }, + "then": { "properties": { "pattern": { "pattern": "^[^*]*\\*$" } } } + } + ], "required": ["id", "name", "scope_kind", "match_type", "pattern", "request_types"], "additionalProperties": false }You'll also want an
exactbranch that rejects*entirely.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/config.schema.json` around lines 3032 - 3103, The pricing_override schema allows invalid configs: make pattern non-empty and enforce match rules, constrain request_types to the defined enum, and add scope_kind conditionals to require the correct ID fields. Specifically: change pricing_override.pattern to require minLength:1 and for match_type "exact" reject a lone "*" (use an if: {properties:{match_type:{const:"exact"}}} then: {properties:{pattern:{not:{enum:["*"]}}}}); make pricing_override.request_types.items a $ref to pricing_override_request_type instead of free string; and add if/then conditionals keyed on pricing_override.scope_kind values (e.g., if scope_kind == "virtual_key" or "virtual_key_provider" require virtual_key_id, if scope_kind starts with "provider" require provider_id, if scope_kind includes "provider_key" or "virtual_key_provider_key" require provider_key_id). Ensure additionalProperties remains false.framework/modelcatalog/overrides.go (1)
332-336:⚠️ Potential issue | 🟡 MinorUse a stable sort for equal-length wildcard prefixes.
sort.Slicecan reorder equal-length patterns, so duplicate wildcard prefixes lose the store/load-order tie-breaker and the winning override can flip across rebuilds.sort.SliceStablepreserves the existing deterministic order.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides.go` around lines 332 - 336, The sort of wildcard patterns currently uses sort.Slice which can reorder entries with equal-length patterns; change the call to sort.SliceStable to preserve the existing store/load order for equal-length prefixes so the tie-breaker remains deterministic—update the sort invocation around data.wildcard (the anonymous comparator that compares len(data.wildcard[i].pattern)) to use sort.SliceStable with the same comparator.framework/configstore/rdb.go (1)
1320-1320:⚠️ Potential issue | 🟡 MinorAdd a stable secondary sort key for override loads.
ModelCataloguses load order as part of override precedence, but both queries sort only bycreated_at. If two rows share the same timestamp, reloads/pages can return them in different orders and a different override can win. Please add a unique tie-breaker such asid ASC.Minimal fix
- if err := q.Order("created_at ASC").Find(&overrides).Error; err != nil { + if err := q.Order("created_at ASC, id ASC").Find(&overrides).Error; err != nil { return nil, s.parseGormError(err) } @@ - Order("created_at ASC"). + Order("created_at ASC, id ASC"). Offset(offset). Limit(limit). Find(&overrides).Error; err != nil {Also applies to: 1365-1367
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/rdb.go` at line 1320, ModelCatalog's override loads currently order only by created_at which can yield nondeterministic precedence when timestamps tie; update the queries that call q.Order("created_at ASC").Find(&overrides).Error (and the other similar query around lines noted) to add a stable secondary sort, e.g. include "id ASC" as a tie-breaker so the Order becomes created_at ASC, id ASC (or call Order("id ASC") after the existing Order) for the q.Find(&overrides) calls to ensure deterministic override ordering.ui/lib/store/apis/governanceApi.ts (1)
636-664:⚠️ Potential issue | 🟠 MajorInsert updated overrides into queries they newly match.
This handler only patches lists that already contain the row. If an edit changes scope or name so the override should move into another cached
getPricingOverridesresult,index === -1returns early and that view stays stale until the next poll/refetch. Please handle the "now matches but wasn't present" case the same way as create, and includeargs.searchinmatchesQuery.
Based on learnings: inui/lib/store/apis/,onQueryStarted+updateQueryDatais the expected pattern specifically to avoid stale UI across replicas.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 636 - 664, The onQueryStarted handler currently only updates cached getPricingOverrides entries when the edited override already exists (index !== -1); update it to also insert the updated override into any cached query where it now matches but was absent: inside the governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, ...) callback, when index === -1 and matchesQuery is true, push (or unshift based on sort) the updated item into draft.pricing_overrides and increment draft.count and draft.total_count accordingly; also include args.search in the matchesQuery predicate (alongside scopeKind/virtualKeyID/providerID/providerKeyID) so search-filtered lists receive the new row. Ensure you reference onQueryStarted, getPricingOverrides, governanceApi.util.updateQueryData, matchesQuery, and handle adjusting counts when inserting.ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx (2)
844-853:⚠️ Potential issue | 🟡 MinorThe JSON editor still needs a stable e2e selector.
Every other primary control in this sheet now exposes a deterministic selector, but the JSON editor path still does not. If
CodeEditordoes not forward DOM props, attach the selector to the wrapper so JSON patch flows stay testable.🧪 Example fix
- <div className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")}> + <div + data-testid="pricing-override-json-editor" + className={cn("bg-muted/50 overflow-hidden rounded-md border", jsonError && "border-destructive")} + > <CodeEditorAs per coding guidelines
ui/**/*.{ts,tsx}: "UI interactive elements must have data-testid attributes following the pattern 'data-testid="--"'".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 844 - 853, The JSON CodeEditor lacks a stable e2e selector; either make CodeEditor forward DOM props so callers can pass data-testid or wrap the editor in a wrapper element and attach a deterministic data-testid like "pricingoverride-jsoneditor-editor" to it; update the JSX around the CodeEditor instance (the CodeEditor component usage where props include lang, code={jsonPatch}, onChange={handleJSONChange}) to ensure the data-testid is present on the DOM element that contains the editable JSON so tests can reliably target it.
363-399:⚠️ Potential issue | 🟡 MinorThe JSON-only reopen edge case is still leaving stale editor state behind.
The
wasOpenguard fixed the refetch-reset problem, but if the user only types invalid JSON,handleJSONChangeupdatesjsonPatchwithout changingform. On the next create open, Line 398 is a no-op because state is still the shareddefaultFormState, so Lines 445-452 never rebuild the editor contents and the sheet reopens with stale JSON whilejsonErrorhas already been cleared.🩹 Minimal fix
jsonEditingRef.current = false; setJSONError(undefined); + setJSONPatch(""); if (editingOverride) {Also applies to: 445-452, 454-485
🧹 Nitpick comments (5)
ui/lib/types/governance.ts (1)
3-3: Use the@/libalias here for consistency.Please switch this import to
@/lib/types/config; this file sits underui/lib, and the UI codebase prefers alias imports over relative ones.♻️ Suggested cleanup
-import { ModelProviderName, RequestType } from "./config"; +import { ModelProviderName, RequestType } from "@/lib/types/config";Based on learnings, prefer using the
@/libpath alias for imports instead of relative paths from within the ui/lib directory.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/types/governance.ts` at line 3, Replace the relative import of ModelProviderName and RequestType in governance.ts with the project alias import; update the import statement that currently references "./config" to use "@/lib/types/config" so the symbols ModelProviderName and RequestType are imported via the alias and match the UI codebase convention.transports/bifrost-http/handlers/inference.go (1)
748-764: Only attachpricingwhen at least one field is populated.With nullable base rates, this block can now emit
"pricing": {}for models whose catalog entry exists but none of the surfaced fields are set. KeepingPricingnil in that case avoids a small but user-visible API shape change.♻️ Possible cleanup
- pricing := &schemas.Pricing{} + pricing := &schemas.Pricing{} + hasPricing := false if pricingEntry.InputCostPerToken != nil { pricing.Prompt = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.InputCostPerToken)) + hasPricing = true } if pricingEntry.OutputCostPerToken != nil { pricing.Completion = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.OutputCostPerToken)) + hasPricing = true } if pricingEntry.InputCostPerImage != nil { pricing.Image = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.InputCostPerImage)) + hasPricing = true } if pricingEntry.CacheReadInputTokenCost != nil { pricing.InputCacheRead = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.CacheReadInputTokenCost)) + hasPricing = true } if pricingEntry.CacheCreationInputTokenCost != nil { pricing.InputCacheWrite = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.CacheCreationInputTokenCost)) + hasPricing = true } - resp.Data[i].Pricing = pricing + if hasPricing { + resp.Data[i].Pricing = pricing + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/inference.go` around lines 748 - 764, The current code always assigns an empty Pricing struct to resp.Data[i].Pricing even when no fields on pricingEntry are set; change the logic in the block that builds pricing (the pricing variable created with &schemas.Pricing{}) to only set resp.Data[i].Pricing when at least one of the pricing fields (InputCostPerToken, OutputCostPerToken, InputCostPerImage, CacheReadInputTokenCost, CacheCreationInputTokenCost) produced a non-nil value — e.g., track a boolean flag (or check that at least one of pricing.Prompt, pricing.Completion, pricing.Image, pricing.InputCacheRead, pricing.InputCacheWrite is non-nil) and only assign resp.Data[i].Pricing = pricing when that condition is true.plugins/logging/operations.go (1)
1023-1024: Return a scope pointer directly to match the repo’s pointer style.The scope resolution here looks right; this is just a small consistency cleanup so the new pointer construction does not introduce another
&valuepattern.♻️ Possible cleanup
-func pricingScopesForLog(logEntry *logstore.Log) modelcatalog.PricingLookupScopes { +func pricingScopesForLog(logEntry *logstore.Log) *modelcatalog.PricingLookupScopes { if logEntry == nil { - return modelcatalog.PricingLookupScopes{} + return &modelcatalog.PricingLookupScopes{} } @@ - return modelcatalog.PricingLookupScopes{ + return &modelcatalog.PricingLookupScopes{ Provider: logEntry.Provider, SelectedKeyID: logEntry.SelectedKeyID, VirtualKeyID: virtualKeyID, } } @@ - scopes := pricingScopesForLog(logEntry) - return p.pricingManager.CalculateCost(resp, &scopes), nil + return p.pricingManager.CalculateCost(resp, pricingScopesForLog(logEntry)), nilBased on learnings, prefer using bifrost.Ptr() to create pointers instead of the address operator (&) even when & would be valid syntactically.
Also applies to: 1157-1171
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@plugins/logging/operations.go` around lines 1023 - 1024, pricingScopesForLog currently returns a value which is then address-taken with & when calling p.pricingManager.CalculateCost; change pricingScopesForLog to return a pointer type instead and update its callers to use that pointer directly (remove the &scopes pattern). Where you need to construct a pointer for a literal or temporary scope, use bifrost.Ptr(...) to create the pointer (rather than &), and update the call site in this file (the CalculateCost call) and the similar block around lines 1157-1171 to accept the pointer return from pricingScopesForLog.plugins/governance/main.go (1)
1425-1433: Drop the staleselectedKeyIDdoc entry.The signature no longer carries
selectedKeyID, so this comment now points readers at a parameter that doesn't exist.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@plugins/governance/main.go` around lines 1425 - 1433, The doc comment above the function postHookWorker references a stale parameter selectedKeyID that no longer exists; update the comment to remove the "- selectedKeyID: The selected provider key ID used for scoped pricing overrides" line and ensure remaining parameter docs (virtualKey, requestID, userID, isCacheRead, isBatch, isFinalChunk, pricingScopes) match the current signature of postHookWorker to keep docs accurate.transports/bifrost-http/handlers/pricing_override_test.go (1)
97-146: Add a partial-update case here.This only proves the handler accepts a full-body replacement payload. The new contract is merge-on-omit, so a regression that clears omitted fields would still pass. Please add a second request that updates just
nameor a singlepatchfield and assert the stored scope, match type, and request types stay unchanged.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/pricing_override_test.go` around lines 97 - 146, Add a partial-update test to ensure merge-on-omit behavior: after creating the initial override in TestUpdatePricingOverride_ReplacesFullBody (or in a new TestUpdatePricingOverride_PartialMerge), send a second request using handler.updatePricingOverride with a body that only updates a single field (e.g., {"name":"PartiallyUpdated"} or {"patch":{"output_cost_per_token":4.0}}) created via newTestRequestCtx and ctx.SetUserValue(override.ID), assert StatusOK, then call store.GetPricingOverrideByID and verify that the omitted fields (ScopeKind, MatchType, RequestTypes and any unchanged fields in PricingPatchJSON) remain exactly as before while the intended field changed; use json.Unmarshal on stored.PricingPatchJSON and require/ assert on individual PricingOptions fields to confirm merge behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/providers/custom-pricing.mdx`:
- Around line 320-328: The table mistakenly lists the field name
`output_cost_per_audio_per_second`; update it to the correct contract field
`output_cost_per_second` (and ensure the description remains "Cost per second of
audio output" or similar) so the key matches the rest of the contract; verify no
other table rows use the incorrect `output_cost_per_audio_per_second` symbol.
- Around line 180-193: The example uses PUT against
/api/governance/pricing-overrides/{id} but the service now requires PATCH-only
semantics; update the example to call PATCH on
/api/governance/pricing-overrides/{id} and demonstrate a sparse update payload
(e.g., only include "patch": { "input_cost_per_token": ... } and other minimal
fields like "request_types" if needed) to show that partial updates are
supported; reference the patch-only behavior and the "input_cost_per_token"
field so readers know to send a partial JSON body rather than a full replace
payload.
In `@examples/configs/withpricingoverridessqlite/config.json`:
- Around line 7-14: The two "path" entries currently point outside and back into
the example directory; update the database paths so they use a single consistent
example-relative convention (e.g., "config.db" and "logs.db" or "./config.db"
and "./logs.db") instead of
"../../examples/configs/withpricingoverridessqlite/config.db" and the matching
logs path; change the top-level "path" (the config DB) and the "logs_store" ->
"config" -> "path" (the logs DB) to the chosen simple relative filenames so both
resolve correctly from the example directory.
In `@framework/modelcatalog/overrides.go`:
- Around line 386-437: patchPricing is skipping two image tiers:
OutputCostPerImageAbove2048x2048Pixels and
OutputCostPerImageAbove4096x4096Pixels on PricingOptions are not copied into
patched, so overrides for those tiers are ignored; update the loop in
patchPricing (the slice of struct{dst **float64; src *float64}) to include
entries mapping {dst: &patched.OutputCostPerImageAbove2048x2048Pixels, src:
override.OutputCostPerImageAbove2048x2048Pixels} and {dst:
&patched.OutputCostPerImageAbove4096x4096Pixels, src:
override.OutputCostPerImageAbove4096x4096Pixels} so those override values are
correctly applied.
In `@framework/modelcatalog/pricing.go`:
- Around line 26-43: The comment on CalculateCost is misleading: passing nil
currently does not fully disable scoped overrides because resolvePricing still
derives/uses scopes.Provider; change CalculateCost to treat a nil scopes as a
true "no overrides" sentinel by short‑circuiting any resolution that would
derive or apply provider/global overrides (i.e., when scopes == nil, call
calculateBaseCost or calculateCostWithCache without calling resolvePricing or
any function that reads/sets PricingLookupScopes fields), or alternatively
update the comment to accurately describe that resolvePricing will fill in
missing scope fields; update references around CalculateCost,
PricingLookupScopes, resolvePricing, calculateCostWithCache and
calculateBaseCost accordingly.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3289-3347: In getPricingOverrides, validate the incoming
scope_kind before building query params and before calling configStore: when
ctx.QueryArgs().Peek("scope_kind") is non-empty, run it through the same
validation/parsing helper used by create/update (reuse the existing scope
validation function—e.g., validateScopeKind or whatever helper is used
elsewhere) and if validation fails return SendError(ctx, 400, "<clear
message>"); if it succeeds use the canonical/validated value for scopeKind when
constructing configstore.PricingOverridesQueryParams so invalid scope_kind
yields a 4xx instead of hitting GetPricingOverridesPaginated.
- Around line 3277-3286: The UpdatePricingOverrideRequest struct currently uses
pointer fields with `omitempty` (e.g., VirtualKeyID, ProviderID, ProviderKeyID)
so JSON unmarshaling cannot distinguish omitted vs explicit null and your merge
logic (the block that checks `if req.VirtualKeyID != nil` etc.) cannot clear
scoped IDs; fix this by adding presence-tracking for these fields (either
implement a custom UnmarshalJSON on UpdatePricingOverrideRequest, overlay with
json.RawMessage to detect field presence, or replace those pointer fields with a
small nullable wrapper type that carries both a value and an explicit "set"
flag), update the merge logic to check the wrapper's "set" flag (or presence
map) rather than nil, and apply the same change for ProviderID and ProviderKeyID
so explicit nulls clear values while omitted fields leave them untouched.
In `@transports/bifrost-http/lib/config.go`:
- Around line 1541-1564: The loop that creates pricing overrides currently
swallows errors from configstore.GeneratePricingOverrideHash and only logs a
warning; change this so generation or serialization failures return an error up
the call chain instead of using logger.Warn: in the block handling
config.GovernanceConfig.PricingOverrides (the loop referencing override,
override.RequestTypesJSON, configstore.GeneratePricingOverrideHash and
override.ConfigHash), return a formatted error when json.Marshal or
GeneratePricingOverrideHash fails, and ensure those returned errors are
propagated through loadGovernanceConfigFromFile / loadConfigFromFile so startup
aborts rather than continuing with stale pricing.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 626-629: Each visible Label lacks an htmlFor/id pairing with its
corresponding control in pricingOverrideSheet.tsx; update each Label and
matching Input/Select/Button to use a unique id (you can derive from existing
data-testid values like "pricing-override-name-input") and set Label's htmlFor
to that id so clicks and screen readers correctly associate the label with the
control; apply the same change for the other label/control groups referenced
(lines 638-653, 656-678, 681-735, 746-769, 774-823) ensuring every
Label(htmlFor="...") matches the control id attribute.
In `@ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx`:
- Around line 14-18: The import block has inconsistent formatting: the Input
import (Input) and the Select import group (Select, SelectContent, SelectItem,
SelectTrigger, SelectValue) are missing trailing semicolons; update the import
statements for Input and the Select group in virtualKeysTable.tsx to include
trailing semicolons to match the surrounding imports (Badge, Button, Table,
etc.) so Prettier consistency is preserved.
---
Outside diff comments:
In `@core/schemas/provider.go`:
- Around line 401-410: The ProviderConfig struct no longer includes
pricing_overrides so legacy JSON with providers[].pricing_overrides will be
silently dropped; update the config loading path to either (A) reject unknown
fields for provider blocks by decoding provider JSON with
json.Decoder{DisallowUnknownFields:true} when populating ProviderConfig (e.g.,
in your config loader / UnmarshalConfig function) and return a clear error
mentioning providers[].pricing_overrides, or (B) add an explicit migration step
run after unmarshalling that detects a legacy providers[].pricing_overrides key
and maps its values into the new governance-scoped structure (e.g.,
GovernanceConfig.PricingOverrides) and logs the migration; implement one of
these in the code paths that create or validate ProviderConfig so legacy
overrides are not silently dropped.
In `@framework/modelcatalog/main.go`:
- Around line 221-225: The background goroutine calling
mc.syncModelParameters(ctx) is started before Init can fail (when override
loading became fatal), so it may run against an uninitialized catalog if Init
returns an error; fix by starting that goroutine only after Init has completed
successfully (i.e., after the fatal override-loading path returns nil) or by
tying it to a cancellation that is triggered when Init fails. Concretely, move
the go func that calls mc.syncModelParameters(ctx) to a point after Init
finishes the override loading check (the Init function) or ensure Init cancels
the passed ctx before returning an error so the launched goroutine exits;
reference the Init function and the syncModelParameters method and adjust where
the mc.logger.Warn/logging and goroutine launch occur accordingly.
- Around line 149-176: The decoder for PricingEntry.UnmarshalJSON only handles
the tiered object form and ignores a plain float64; modify the method to also
try decoding search_context_cost_per_query as a plain number when
raw.SearchContextCostPerQuery is nil: after the existing sonic.Unmarshal into
raw (the struct with SearchContextCostPerQuery *struct{Low,Medium,High}), if
raw.SearchContextCostPerQuery == nil then unmarshal the original data (or the
raw JSON) into a tiny auxiliary struct with SearchContextCostPerQuery *float64
and, if that yields a non-nil value, set p.SearchContextCostPerQuery to that
value (assign to p.SearchContextCostPerQuery). Ensure you reference and set
p.SearchContextCostPerQuery and leave the existing tiered-selection logic intact
for the tiered case.
---
Duplicate comments:
In `@core/schemas/tracer.go`:
- Around line 69-71: The change made Tracer.PopulateLLMResponseAttributes to
take *BifrostContext which breaks source compatibility; revert
PopulateLLMResponseAttributes to its original public signature
PopulateLLMResponseAttributes(handle SpanHandle, resp *BifrostResponse, err
*BifrostError) so existing tracers continue to compile, and add a
Bifrost-specific escape hatch (for example a new method
PopulateLLMResponseAttributesWithContext(ctx *BifrostContext, handle SpanHandle,
resp *BifrostResponse, err *BifrostError) or a new TracerWithContext interface)
that callers inside Bifrost can use when context is required; update internal
callers to use the new Bifrost-specific method while leaving the public Tracer
API unchanged.
In `@docs/openapi/schemas/management/governance.yaml`:
- Around line 1160-1207: CreatePricingOverrideRequest must enforce required
fields per scope_kind: refactor the schema for CreatePricingOverrideRequest to
use oneOf (modeled like RoutingRule) with separate branch schemas for each
scope_kind value (global, provider, provider_key, virtual_key,
virtual_key_provider, virtual_key_provider_key) where each branch includes the
common properties (name, match_type, pattern, request_types, patch, scope_kind)
and declares virtual_key_id, provider_id, and provider_key_id as required only
on the branches that need them (e.g., provider branch requires provider_id,
provider_key branch requires provider_key_id, virtual_key_provider_key branch
requires virtual_key_id, provider_id and provider_key_id as appropriate); ensure
scope_kind is fixed in each branch (enum with a single value) so generated
clients must include the correct IDs for the selected scope.
In `@framework/configstore/clientconfig.go`:
- Around line 970-983: GeneratePricingOverrideHash is currently hashing the raw
RequestTypesJSON which can miss in-memory edits and yields order-sensitive
hashes; replace the RequestTypesJSON usage with the parsed slice on the struct
(e.g., p.RequestTypes or whatever parsed field holds []string) and hash each
element instead; to make hashing order-insensitive, make a local copy, sort it
(using sort.Strings), then iterate and hash each entry (handling nil/empty
safely) so equivalent sets produce the same hash. Reference:
GeneratePricingOverrideHash, TablePricingOverride, RequestTypesJSON ->
RequestTypes.
In `@framework/configstore/rdb.go`:
- Line 1320: ModelCatalog's override loads currently order only by created_at
which can yield nondeterministic precedence when timestamps tie; update the
queries that call q.Order("created_at ASC").Find(&overrides).Error (and the
other similar query around lines noted) to add a stable secondary sort, e.g.
include "id ASC" as a tie-breaker so the Order becomes created_at ASC, id ASC
(or call Order("id ASC") after the existing Order) for the q.Find(&overrides)
calls to ensure deterministic override ordering.
In `@framework/modelcatalog/overrides_test.go`:
- Around line 497-502: The test currently compares tc.expected to
patched.InputCostPerToken (a *float64); change the assertion to compare
tc.expected to the dereferenced value (e.g., assert.Equal(t, tc.expected,
*patched.InputCostPerToken)) and ensure patched and patched.InputCostPerToken
are non-nil before dereferencing in the test around applyPricingOverrides.
In `@framework/modelcatalog/overrides.go`:
- Around line 332-336: The sort of wildcard patterns currently uses sort.Slice
which can reorder entries with equal-length patterns; change the call to
sort.SliceStable to preserve the existing store/load order for equal-length
prefixes so the tie-breaker remains deterministic—update the sort invocation
around data.wildcard (the anonymous comparator that compares
len(data.wildcard[i].pattern)) to use sort.SliceStable with the same comparator.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3433-3443: The DB write (configStore.CreatePricingOverride /
UpdatePricingOverride / DeletePricingOverride) and the in-memory sync
(governanceManager.UpsertPricingOverride / RemovePricingOverride) must be atomic
from the client perspective: if the in-memory upsert/reload fails, do not return
success. Modify the handler so that after a successful DB mutation you call
governanceManager.UpsertPricingOverride (or reload the manager) and if that call
fails you either (a) rollback the DB change (call the corresponding
configStore.DeletePricingOverride or restore previous row) or (b) trigger a full
governanceManager reload from the DB before returning; in either case return
HTTP 500 (use SendError) and log the error instead of returning success. Ensure
this same pattern is applied to the create, update and delete flows handled by
CreatePricingOverride/UpsertPricingOverride and their counterparts so DB and
in-memory state never diverge.
In `@transports/bifrost-http/handlers/providers.go`:
- Around line 177-190: The handlers currently unmarshal request bodies into the
payload struct and silently ignore unknown top-level keys like
pricing_overrides; add a pre-unmarshal guard function (e.g.,
rejectUnsupportedProviderFields) that inspects the raw request body for the
"pricing_overrides" key and, if present, returns an error; call this guard at
the start of the handler (before json.Unmarshal of payload) and, on error,
SendError with fasthttp.StatusBadRequest and a message directing the caller to
/api/governance/pricing-overrides; ensure you apply the same change to both
handler sites (the existing payload-unmarshal blocks around the payload struct
and the other occurrence mentioned).
In `@transports/config.schema.json`:
- Around line 3032-3103: The pricing_override schema allows invalid configs:
make pattern non-empty and enforce match rules, constrain request_types to the
defined enum, and add scope_kind conditionals to require the correct ID fields.
Specifically: change pricing_override.pattern to require minLength:1 and for
match_type "exact" reject a lone "*" (use an if:
{properties:{match_type:{const:"exact"}}} then:
{properties:{pattern:{not:{enum:["*"]}}}}); make
pricing_override.request_types.items a $ref to pricing_override_request_type
instead of free string; and add if/then conditionals keyed on
pricing_override.scope_kind values (e.g., if scope_kind == "virtual_key" or
"virtual_key_provider" require virtual_key_id, if scope_kind starts with
"provider" require provider_id, if scope_kind includes "provider_key" or
"virtual_key_provider_key" require provider_key_id). Ensure additionalProperties
remains false.
In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx`:
- Around line 53-57: The current useEffect that calls setActiveFields(rebuild)
removes rows as soon as a value becomes empty; change it to only add non-empty
keys (union with the existing activeFields) so empty-but-still-active rows
remain mounted until deactivateField() is explicitly called, e.g. replace the
rebuild logic with setActiveFields(prev => new Set([...prev, ...nonEmptyKeys])),
and keep a separate effect (or trigger) that fully resets activeFields when an
external load/reset occurs (listen to whatever “external load” flag/prop you
have) so that external loads still replace the set.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 844-853: The JSON CodeEditor lacks a stable e2e selector; either
make CodeEditor forward DOM props so callers can pass data-testid or wrap the
editor in a wrapper element and attach a deterministic data-testid like
"pricingoverride-jsoneditor-editor" to it; update the JSX around the CodeEditor
instance (the CodeEditor component usage where props include lang,
code={jsonPatch}, onChange={handleJSONChange}) to ensure the data-testid is
present on the DOM element that contains the editable JSON so tests can reliably
target it.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 76-80: The provider lookup for the "provider_key" and
"virtual_key_provider_key" cases only uses override.provider_key_id and
keyProviderMap, causing a "-" when key metadata is missing; update the return to
fall back to override.provider_id: compute keyID = override.provider_key_id ||
"" as before, then try providerMap.get(keyProviderMap.get(keyID) ||
override.provider_id || "") and if that fails fall back to
keyProviderMap.get(keyID) || override.provider_id || "-" so the scoped
provider_id is used when key metadata is stale or absent.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 636-664: The onQueryStarted handler currently only updates cached
getPricingOverrides entries when the edited override already exists (index !==
-1); update it to also insert the updated override into any cached query where
it now matches but was absent: inside the
governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs,
...) callback, when index === -1 and matchesQuery is true, push (or unshift
based on sort) the updated item into draft.pricing_overrides and increment
draft.count and draft.total_count accordingly; also include args.search in the
matchesQuery predicate (alongside
scopeKind/virtualKeyID/providerID/providerKeyID) so search-filtered lists
receive the new row. Ensure you reference onQueryStarted, getPricingOverrides,
governanceApi.util.updateQueryData, matchesQuery, and handle adjusting counts
when inserting.
In `@ui/lib/types/governance.ts`:
- Around line 402-417: The PricingOverridePatch type has drifted from the
supported image-pricing contract: remove the deprecated premium-image fields
(e.g., output_cost_per_image_premium_image and the combined premium variants
output_cost_per_image_above_512_and_512_pixels_and_premium_image and
output_cost_per_image_above_1024_and_1024_pixels_and_premium_image) and add the
missing documented fields output_cost_per_image_above_2048_and_2048_pixels and
output_cost_per_image_above_4096_and_4096_pixels so the PricingOverridePatch
type accurately matches the current contract; update the type definition where
PricingOverridePatch is declared in ui/lib/types/governance.ts and ensure any
related optional input_cost/output_cost properties follow the same naming
convention.
- Around line 431-468: The read/update models use raw string[] for request_types
which loses compile-time safety; update the PricingOverride interface and the
UpdatePricingOverrideRequest type to use the RequestType union instead of
string[] (i.e., change PricingOverride.request_types to RequestType[]
(preserving optionality if intended) and
UpdatePricingOverrideRequest.request_types to RequestType[]), ensuring all
references to request_types in those interfaces match the
CreatePricingOverrideRequest’s RequestType[] usage so loaded overrides retain
strong typing.
---
Nitpick comments:
In `@plugins/governance/main.go`:
- Around line 1425-1433: The doc comment above the function postHookWorker
references a stale parameter selectedKeyID that no longer exists; update the
comment to remove the "- selectedKeyID: The selected provider key ID used for
scoped pricing overrides" line and ensure remaining parameter docs (virtualKey,
requestID, userID, isCacheRead, isBatch, isFinalChunk, pricingScopes) match the
current signature of postHookWorker to keep docs accurate.
In `@plugins/logging/operations.go`:
- Around line 1023-1024: pricingScopesForLog currently returns a value which is
then address-taken with & when calling p.pricingManager.CalculateCost; change
pricingScopesForLog to return a pointer type instead and update its callers to
use that pointer directly (remove the &scopes pattern). Where you need to
construct a pointer for a literal or temporary scope, use bifrost.Ptr(...) to
create the pointer (rather than &), and update the call site in this file (the
CalculateCost call) and the similar block around lines 1157-1171 to accept the
pointer return from pricingScopesForLog.
In `@transports/bifrost-http/handlers/inference.go`:
- Around line 748-764: The current code always assigns an empty Pricing struct
to resp.Data[i].Pricing even when no fields on pricingEntry are set; change the
logic in the block that builds pricing (the pricing variable created with
&schemas.Pricing{}) to only set resp.Data[i].Pricing when at least one of the
pricing fields (InputCostPerToken, OutputCostPerToken, InputCostPerImage,
CacheReadInputTokenCost, CacheCreationInputTokenCost) produced a non-nil value —
e.g., track a boolean flag (or check that at least one of pricing.Prompt,
pricing.Completion, pricing.Image, pricing.InputCacheRead,
pricing.InputCacheWrite is non-nil) and only assign resp.Data[i].Pricing =
pricing when that condition is true.
In `@transports/bifrost-http/handlers/pricing_override_test.go`:
- Around line 97-146: Add a partial-update test to ensure merge-on-omit
behavior: after creating the initial override in
TestUpdatePricingOverride_ReplacesFullBody (or in a new
TestUpdatePricingOverride_PartialMerge), send a second request using
handler.updatePricingOverride with a body that only updates a single field
(e.g., {"name":"PartiallyUpdated"} or {"patch":{"output_cost_per_token":4.0}})
created via newTestRequestCtx and ctx.SetUserValue(override.ID), assert
StatusOK, then call store.GetPricingOverrideByID and verify that the omitted
fields (ScopeKind, MatchType, RequestTypes and any unchanged fields in
PricingPatchJSON) remain exactly as before while the intended field changed; use
json.Unmarshal on stored.PricingPatchJSON and require/ assert on individual
PricingOptions fields to confirm merge behavior.
In `@ui/lib/types/governance.ts`:
- Line 3: Replace the relative import of ModelProviderName and RequestType in
governance.ts with the project alias import; update the import statement that
currently references "./config" to use "@/lib/types/config" so the symbols
ModelProviderName and RequestType are imported via the alias and match the UI
codebase convention.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
Pratham-Mishra04
force-pushed
the
feat/scoped-pricing-overrides
branch
from
27abeb6 to
33d9edb
Compare
Collaborator
|
@coderabbitai full-review |
Contributor
|
✅ Actions performedFull review triggered. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
plugins/logging/operations.go (1)
950-1024:⚠️ Potential issue | 🟠 MajorDeserialize modality-specific outputs before recalculating cost.
This gate only calls
DeserializeFields()whentoken_usageorcache_debugis present. Speech/video entries can carry the billable data only inspeech_output/video_generation_output, so the patching block below never restores those fields andCalculateCostwill silently under-price those logs during recalculation.🔧 Suggested fix
- if (logEntry.TokenUsageParsed == nil && logEntry.TokenUsage != "") || - (logEntry.CacheDebugParsed == nil && logEntry.CacheDebug != "") { + if (logEntry.TokenUsageParsed == nil && logEntry.TokenUsage != "") || + (logEntry.CacheDebugParsed == nil && logEntry.CacheDebug != "") || + (logEntry.SpeechOutputParsed == nil && logEntry.SpeechOutput != "") || + (logEntry.TranscriptionOutputParsed == nil && logEntry.TranscriptionOutput != "") || + (logEntry.ImageGenerationOutputParsed == nil && logEntry.ImageGenerationOutput != "") || + (logEntry.VideoGenerationOutputParsed == nil && logEntry.VideoGenerationOutput != "") {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@plugins/logging/operations.go` around lines 950 - 1024, The DeserializeFields() call is currently gated only on TokenUsage/CacheDebug, so modality-specific parsed fields (TranscriptionOutputParsed, ImageGenerationOutputParsed, VideoGenerationOutputParsed, SpeechOutputParsed) may remain nil and never get restored; update the conditional before calling DeserializeFields() to also check for raw modality fields (e.g., logEntry.TranscriptionOutput != "", logEntry.ImageGenerationOutput != "", logEntry.VideoGenerationOutput != "", logEntry.SpeechOutput != "") or for their parsed counterparts being nil, and call logEntry.DeserializeFields() whenever any modality raw data exists but the corresponding parsed field (TranscriptionOutputParsed, ImageGenerationOutputParsed, VideoGenerationOutputParsed, SpeechOutputParsed) is nil so the later patching blocks can restore usage before p.pricingManager.CalculateCost(resp, &scopes).framework/modelcatalog/pricing.go (1)
83-111:⚠️ Potential issue | 🟠 MajorProvider-reported cost still bypasses scoped overrides.
calculateBaseCostreturnsusage.Cost.TotalCostbefore the newresolvePricing(..., scopes)path runs.responsesUsageToBifrostUsagein this file carriesu.Costthrough, so any Responses provider that reports cost will skip governance pricing overrides entirely and keep the raw provider price in downstream cost reporting. Use provider cost only as a fallback when catalog/override resolution misses.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/pricing.go` around lines 83 - 111, calculateBaseCost currently returns input.usage.Cost.TotalCost before resolvePricing is applied, letting provider-reported cost bypass governance/override pricing; change the logic in calculateBaseCost (which uses extractCostInput and later calls resolvePricing) so that you first call resolvePricing(provider, model, deployment, requestType, scopes) and use that pricing if present, and only if resolvePricing yields no pricing entry fall back to using input.usage.Cost.TotalCost (the provider-reported cost carried via responsesUsageToBifrostUsage). Ensure the provider cost is used as a last-resort fallback rather than an early return.
♻️ Duplicate comments (14)
ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx (1)
16-17:⚠️ Potential issue | 🟡 MinorAdd trailing semicolons to complete the formatting pass.
Lines 16 and 17 are still missing trailing semicolons while the surrounding imports in this updated block now use semicolons. This breaks Prettier consistency.
Suggested diff
-import { Input } from "@/components/ui/input" -import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select" +import { Input } from "@/components/ui/input"; +import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select";As per coding guidelines: TypeScript/React code must be formatted with Prettier.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx` around lines 16 - 17, The two import statements importing Input and the Select components (symbols: Input, Select, SelectContent, SelectItem, SelectTrigger, SelectValue) in virtualKeysTable.tsx are missing trailing semicolons; update those import lines to end with semicolons so they match surrounding imports and satisfy Prettier formatting rules.core/schemas/tracer.go (1)
69-71:⚠️ Potential issue | 🟠 MajorPublic
Tracerinterface change is source-breaking.Line 71 changes an exported method contract and will break external implementations that currently satisfy
schemas.Tracer. Please keepcontext.Contextin the interface and make Bifrost-specific context access additive.🛠️ Suggested direction
- PopulateLLMResponseAttributes(ctx *BifrostContext, handle SpanHandle, resp *BifrostResponse, err *BifrostError) + PopulateLLMResponseAttributes(ctx context.Context, handle SpanHandle, resp *BifrostResponse, err *BifrostError)-func (n *NoOpTracer) PopulateLLMResponseAttributes(_ *BifrostContext, _ SpanHandle, _ *BifrostResponse, _ *BifrostError) { +func (n *NoOpTracer) PopulateLLMResponseAttributes(_ context.Context, _ SpanHandle, _ *BifrostResponse, _ *BifrostError) { }If Bifrost-specific data is needed, pass it via typed context values or add a non-breaking optional extension path.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/schemas/tracer.go` around lines 69 - 71, The Tracer interface change removed the standard context.Context and is source-breaking; restore the original signature of PopulateLLMResponseAttributes to accept a context.Context (e.g., PopulateLLMResponseAttributes(ctx context.Context, handle SpanHandle, resp *BifrostResponse, err *BifrostError)) and keep BifrostContext usage additive by reading Bifrost-specific data from the context via typed context values or by adding a separate optional method (e.g., PopulateLLMResponseAttributesWithBifrost) rather than replacing the context parameter; update references to PopulateLLMResponseAttributes and any implementations to use context.Context and extract BifrostContext safely via context.Value when needed.transports/bifrost-http/handlers/providers.go (1)
315-326:⚠️ Potential issue | 🟠 MajorReject legacy
pricing_overridespayloads instead of silently dropping them.Removing the field from the struct is not enough here: old provider payloads can still be accepted and reach the success path while the override data is discarded. In this migration, that turns legacy clients into silent no-ops; please fail fast with
400here, and reuse the same check inProviderHandler.addProvider, so callers know they must use the governance pricing-override API.Do Go's `encoding/json.Unmarshal` and bytedance `sonic.Unmarshal` ignore unknown JSON object fields by default?🛑 Suggested guard
+ var rawFields map[string]json.RawMessage + if err := json.Unmarshal(ctx.PostBody(), &rawFields); err == nil { + if _, ok := rawFields["pricing_overrides"]; ok { + SendError(ctx, fasthttp.StatusBadRequest, "pricing_overrides is not a supported provider field; use /api/governance/pricing-overrides instead") + return + } + } if err := sonic.Unmarshal(ctx.PostBody(), &payload); err != nil {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/providers.go` around lines 315 - 326, The handler currently unmarshals into a struct that omits the legacy "pricing_overrides" field, which causes old payloads to be silently accepted and their overrides dropped; change the request handling around sonic.Unmarshal in this function and in ProviderHandler.addProvider to first inspect the raw JSON (e.g., unmarshal into a map[string]json.RawMessage or check for the "pricing_overrides" key using a fast search) and if the "pricing_overrides" key is present immediately call SendError(ctx, fasthttp.StatusBadRequest, ...) to reject with 400; keep the existing struct unmarshal afterward for valid payloads so the rest of the code (payload, NetworkConfig, CustomProviderConfig, ConcurrencyAndBufferSize, etc.) continues to work.framework/configstore/clientconfig.go (1)
970-985:⚠️ Potential issue | 🟠 MajorHash
request_typesfrom the parsed slice, notRequestTypesJSON.
RequestTypesJSONis a storage field populated later in the save path, so config-origin overrides can hash identically even afterrequest_typeschanges. It also turns reorder-only diffs into false updates.💡 Minimal fix
func GeneratePricingOverrideHash(p tables.TablePricingOverride) (string, error) { hash := sha256.New() hash.Write([]byte(p.ID)) hash.Write([]byte(p.Name)) hash.Write([]byte(p.ScopeKind)) hash.Write([]byte(derefStr(p.VirtualKeyID))) hash.Write([]byte(derefStr(p.ProviderID))) hash.Write([]byte(derefStr(p.ProviderKeyID))) hash.Write([]byte(p.MatchType)) hash.Write([]byte(p.Pattern)) - hash.Write([]byte(p.RequestTypesJSON)) + requestTypes := make([]schemas.RequestType, len(p.RequestTypes)) + copy(requestTypes, p.RequestTypes) + if len(requestTypes) == 0 && p.RequestTypesJSON != "" { + if err := sonic.Unmarshal([]byte(p.RequestTypesJSON), &requestTypes); err != nil { + return "", err + } + } + sort.Slice(requestTypes, func(i, j int) bool { + return requestTypes[i] < requestTypes[j] + }) + data, err := sonic.Marshal(requestTypes) + if err != nil { + return "", err + } + hash.Write(data) hash.Write([]byte(p.PricingPatchJSON)) return hex.EncodeToString(hash.Sum(nil)), nil }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/configstore/clientconfig.go` around lines 970 - 985, The GeneratePricingOverrideHash function currently hashes RequestTypesJSON (storage field) which can change independently; instead, hash the parsed request types slice used by the config (e.g., use p.RequestTypes or the in-memory parsed []string) in a stable order to avoid reorder-only diffs—serialize or iterate the slice deterministically (sorted or joined with a delimiter) and include that in the hash rather than RequestTypesJSON; update GeneratePricingOverrideHash to reference the parsed slice symbol and remove RequestTypesJSON from the hashed inputs.ui/lib/types/governance.ts (1)
440-440:⚠️ Potential issue | 🟡 MinorKeep
request_typestyped from fetch through update.
CreatePricingOverrideRequestis strict, butPricingOverride.request_typesandUpdatePricingOverrideRequest.request_typeswiden back tostring[]. That lets invalid values survive the edit flow and only fail at submit time.♻️ Suggested diff
export interface PricingOverride { id: string; name: string; scope_kind: PricingOverrideScopeKind; virtual_key_id?: string; provider_id?: string; provider_key_id?: string; match_type: PricingOverrideMatchType; pattern: string; - request_types?: string[]; + request_types?: RequestType[]; pricing_patch: string; config_hash?: string; created_at: string; updated_at: string; } @@ export interface UpdatePricingOverrideRequest { name?: string; scope_kind?: PricingOverrideScopeKind; virtual_key_id?: string; provider_id?: string; provider_key_id?: string; match_type?: PricingOverrideMatchType; pattern?: string; - request_types?: string[]; + request_types?: RequestType[]; patch?: PricingOverridePatch; }Also applies to: 467-467
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/types/governance.ts` at line 440, The request_types property is widened to string[] on PricingOverride and UpdatePricingOverrideRequest, allowing invalid values to pass; change both PricingOverride.request_types and UpdatePricingOverrideRequest.request_types to reuse the stricter type used by CreatePricingOverrideRequest (e.g., Replace the loose string[] with CreatePricingOverrideRequest['request_types'] or extract a shared RequestType union and reference it) so the same exact type flows from fetch through edit to submit, updating any related uses or casts of request_types to match.framework/modelcatalog/overrides_test.go (1)
497-501:⚠️ Potential issue | 🟠 MajorDereference the patched cost pointer in this active test.
patched.InputCostPerTokenis pointer-valued, so this assertion comparesfloat64to*float64and the precedence test will fail when it runs.Minimal fix
t.Run(tc.name, func(t *testing.T) { patched, applied := mc.applyPricingOverrides("gpt-5-nano", schemas.ChatCompletionRequest, base, tc.scopes) require.True(t, applied) - assert.Equal(t, tc.expected, patched.InputCostPerToken) + require.NotNil(t, patched.InputCostPerToken) + assert.Equal(t, tc.expected, *patched.InputCostPerToken) }) } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@framework/modelcatalog/overrides_test.go` around lines 497 - 501, The test currently compares tc.expected (float64) to patched.InputCostPerToken (a *float64); update the assertion to dereference the pointer and guard for nil: first assert.NotNil(t, patched.InputCostPerToken) (or require.NotNil if test must fail immediately), then assert.Equal(t, tc.expected, *patched.InputCostPerToken). Reference mc.applyPricingOverrides and the patched.InputCostPerToken field when making this change.ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx (1)
55-57:⚠️ Potential issue | 🟡 MinorDon't collapse a field row while the user is clearing it.
This effect rebuilds
activeFieldson everyvalueschange. When a user deletes the last character to replace a number, the row disappears before they can type the new value, even though removal already has an explicit X button. Limit the reset to external loads/resets, or preserve active empty rows until explicit removal.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx` around lines 55 - 57, The effect currently collapses rows whenever a value becomes an empty string, causing a row to disappear while the user is typing; update the useEffect that calls setActiveFields so it treats empty strings as still active (only remove a field from activeFields when its value is null/undefined or when the explicit "remove/X" action is used). Concretely, change the PRICING_FIELDS filter in the effect to keep fields where values[f.key] is an empty string (i.e., only exclude when values[f.key] == null) or add a short-lived ref/flag to distinguish external resets from user edits and use that to decide when to rebuild activeFields; reference the useEffect, setActiveFields, PRICING_FIELDS, and values symbols when making the change.ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
76-80:⚠️ Potential issue | 🟡 MinorFall back to
override.provider_idwhen key metadata is missing.If
provider_key_idpoints to stale or unloaded key metadata, this branch renders-even when the override itself still carries a validprovider_id. That drops scope information forprovider_keyandvirtual_key_provider_keyrows.Suggested fix
case "provider_key": case "virtual_key_provider_key": { const keyID = override.provider_key_id || ""; - return providerMap.get(keyProviderMap.get(keyID) || "") || keyProviderMap.get(keyID) || "-"; + const providerID = keyProviderMap.get(keyID) || override.provider_id || ""; + return providerMap.get(providerID) || providerID || "-"; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 76 - 80, The "provider_key"/"virtual_key_provider_key" branch uses provider_key_id to look up key metadata and currently falls back to "-" when that lookup is stale; update the logic in scopedPricingOverridesView.tsx so that after computing keyID (override.provider_key_id || ""), you derive providerId = keyProviderMap.get(keyID) || override.provider_id || "" and then return providerMap.get(providerId) || providerId || "-". This ensures you display the override.provider_id when key metadata is missing, while still preferring mapped names from keyProviderMap and providerMap for functions/variables referenced in the diff (keyID, keyProviderMap, providerMap, override.provider_id).docs/openapi/schemas/management/governance.yaml (1)
1160-1251:⚠️ Potential issue | 🟠 MajorEncode the scope-dependent IDs in the schema, not just the descriptions.
These request schemas still allow bodies like
scope_kind: providerwithoutprovider_id, so generated clients can pass OpenAPI validation and only fail at runtime. Please model the scope combinations withoneOf/conditional requirements here, the same wayRoutingRuledoes above.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@docs/openapi/schemas/management/governance.yaml` around lines 1160 - 1251, The CreatePricingOverrideRequest and UpdatePricingOverrideRequest schemas allow invalid combinations (e.g., scope_kind: provider without provider_id); update both schemas to encode scope-dependent required fields using oneOf with discriminators or conditional required blocks that match scope_kind values (e.g., one schema for global, one for provider requiring provider_id, one for provider_key requiring provider_key_id, and analogous variants for virtual_key* requiring virtual_key_id and/or provider_id/provider_key_id), mirroring how RoutingRule models scope combos so generated clients will fail OpenAPI validation rather than only runtime.transports/bifrost-http/handlers/governance.go (2)
3589-3601:⚠️ Potential issue | 🟠 MajorDon’t return success when the in-memory delete fails.
If
h.governanceManager.DeletePricingOverridefails after the DB delete, runtime pricing can keep using the stale override even though this endpoint returns 200. Surface that as a 500 and add a compensating sync/rollback path instead of log-and-continue.Based on learnings: if a governance DB write succeeds but the in-memory reload/delete fails, the handler must return HTTP 500 because DB and memory need to stay in sync.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3589 - 3601, The handler currently deletes the pricing override in storage via configStore.DeletePricingOverride then logs and continues if governanceManager.DeletePricingOverride fails; change this so that if governanceManager.DeletePricingOverride(ctx, id) returns an error you return a 500 (use SendError with fasthttp.StatusInternalServerError) instead of returning success, and implement a compensating path: attempt to re-create or re-sync the deleted DB record (or call a configStore.RollbackPricingOverride/PutPricingOverride if available) or call a governanceManager.SyncFromStore/reload method to reconcile state before responding; update the code around configStore.DeletePricingOverride and governanceManager.DeletePricingOverride to bail with an error response when in-memory deletion/sync fails so DB and memory remain consistent.
3314-3316:⚠️ Potential issue | 🟡 MinorValidate
scope_kindbefore calling the store.
scope_kindis still forwarded as arbitrary text here, so?scope_kind=foocan fall through toGetPricingOverrides*instead of producing the 4xx this endpoint is supposed to return for invalid scopes. Parse/validate it up front and reuse the validated value in both query paths.Also applies to: 3333-3338, 3383-3388
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3314 - 3316, The handler currently assigns scopeKind from the raw query string (scope_kind) and forwards it to store calls, allowing invalid values (e.g., ?scope_kind=foo) to reach GetPricingOverrides*; instead, parse and validate scope_kind immediately (e.g., map/enum check or parser used elsewhere) and if invalid return a 4xx before any store call, then reuse the validated value (the parsed enum/typed variable) in both query branches that call GetPricingOverrides*; update the code paths around the scopeKind variable assignment and the two other occurrences referenced (the blocks at the other GetPricingOverrides* call sites) to use the validated/typed value rather than the raw string.transports/bifrost-http/lib/config.go (1)
1197-1210:⚠️ Potential issue | 🟠 MajorDon't fail open on pricing-override merge errors.
createGovernanceConfigInStorenow aborts startup on override hash/serialization failures, but the existing-store merge path still just warns and skips the file row. After aconfig.jsonchange, that can leave the previous DB override active and serve stale pricing instead of honoring file precedence. Please makemergeGovernanceConfigfail closed here and propagate the error throughloadGovernanceConfigFromFile.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/lib/config.go` around lines 1197 - 1210, The mergeGovernanceConfig path currently swallows pricing-override serialization/hash errors (in the block using json.Marshal on newOverride.RequestTypes and configstore.GeneratePricingOverrideHash) by logging Warn and continuing, which can leave stale DB overrides active; change those logger.Warn+continue occurrences in mergeGovernanceConfig to return a descriptive error instead (including newOverride.ID and the underlying err) so the merge fails closed, and ensure loadGovernanceConfigFromFile propagates that error up (matching createGovernanceConfigInStore behavior) so startup aborts on failure to serialize/hash pricing overrides.ui/lib/store/apis/governanceApi.ts (1)
643-660:⚠️ Potential issue | 🟠 MajorHandle rows that start or stop matching a filtered query.
matchesQueryignoresargs.search, and theindex === -1fast-return means an edit that moves an override into a cached list never gets inserted there. Renaming or re-scoping an override can still leavegetPricingOverrideswrong until the next refetch.🛠️ Suggested fix
const args: PricingOverrideQueryArgs = entry.originalArgs ?? {}; const matchesQuery = (!args.scopeKind || args.scopeKind === updated.scope_kind) && (!args.virtualKeyID || args.virtualKeyID === updated.virtual_key_id) && (!args.providerID || args.providerID === updated.provider_id) && - (!args.providerKeyID || args.providerKeyID === updated.provider_key_id); + (!args.providerKeyID || args.providerKeyID === updated.provider_key_id) && + (!args.search || (updated.name ?? "").toLowerCase().includes(args.search.toLowerCase())); dispatch( governanceApi.util.updateQueryData("getPricingOverrides", entry.originalArgs, (draft) => { - if (!draft.pricing_overrides) return; + if (!draft.pricing_overrides) draft.pricing_overrides = []; const index = draft.pricing_overrides.findIndex((o) => o.id === id); - if (index === -1) return; - if (matchesQuery) { + if (index !== -1 && matchesQuery) { draft.pricing_overrides[index] = updated; - } else { + } else if (index !== -1) { // Override no longer belongs in this filtered list draft.pricing_overrides.splice(index, 1); draft.count = Math.max(0, (draft.count || 0) - 1); draft.total_count = Math.max(0, (draft.total_count || 0) - 1); + } else if (matchesQuery) { + draft.total_count = (draft.total_count || 0) + 1; + if (!args.offset || args.offset === 0) { + draft.pricing_overrides.unshift(updated); + draft.count = (draft.count || 0) + 1; + } } }), );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/store/apis/governanceApi.ts` around lines 643 - 660, The patch must ensure edits that start or stop matching a cached filtered query are handled: update the matchesQuery predicate in the update block (the PricingOverrideQueryArgs / matchesQuery logic) to include args.search (apply the same search filter used by getPricingOverrides), and inside the governanceApi.util.updateQueryData("getPricingOverrides", ...) callback, when index === -1 but matchesQuery is true, insert the updated item into draft.pricing_overrides (e.g., unshift or splice at 0) and increment draft.count and draft.total_count; keep the existing branch that splices out and decrements counts when the item exists but no longer matches. Reference symbols: PricingOverrideQueryArgs, matchesQuery, governanceApi.util.updateQueryData("getPricingOverrides", ...), draft.pricing_overrides, index.ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx (1)
282-296:⚠️ Potential issue | 🟡 MinorAssociate labels with inputs in
renderFieldsfor accessibility.The dynamically rendered fields lack
htmlFor/idpairing, which breaks screen reader announcements and prevents label-click focus behavior.♿ Proposed fix
export function renderFields( fields: ReadonlyArray<{ key: PricingFieldKey; label: string }>, form: FormState, setForm: Dispatch<SetStateAction<FormState>>, errors: FieldErrors, onFieldChange?: () => void, ) { return ( <div className="grid grid-cols-1 gap-4 md:grid-cols-2"> {fields.map((field) => ( <div key={field.key} className="space-y-2 pb-1"> - <Label>{field.label}</Label> + <Label htmlFor={`pricing-override-field-input-${field.key}`}>{field.label}</Label> <Input + id={`pricing-override-field-input-${field.key}`} data-testid={`pricing-override-field-input-${field.key}`} type="text"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx` around lines 282 - 296, The Label and Input rendered in renderFields lack an id/htmlFor pair; update the JSX for Label and Input to associate them by adding a unique id (e.g., `id={`pricing-override-field-${field.key}`}`) on the Input and `htmlFor` with the same value on the Label so clicking the label focuses the input and screen readers announce correctly; ensure you use the same field.key-derived identifier and keep existing props like data-testid, value, onChange, and className unchanged when adding the id/htmlFor attributes.
🧹 Nitpick comments (5)
transports/bifrost-http/handlers/inference.go (1)
748-764: Avoid emitting emptypricingobjects when no mapped fields are present.Line 748 always allocates
pricing, so models can returnpricing: {}even when all relevant costs are nil. Consider assigning only when at least one field is set.♻️ Suggested refinement
- pricing := &schemas.Pricing{} + pricing := &schemas.Pricing{} + hasPricingField := false if pricingEntry.InputCostPerToken != nil { pricing.Prompt = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.InputCostPerToken)) + hasPricingField = true } if pricingEntry.OutputCostPerToken != nil { pricing.Completion = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.OutputCostPerToken)) + hasPricingField = true } if pricingEntry.InputCostPerImage != nil { pricing.Image = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.InputCostPerImage)) + hasPricingField = true } if pricingEntry.CacheReadInputTokenCost != nil { pricing.InputCacheRead = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.CacheReadInputTokenCost)) + hasPricingField = true } if pricingEntry.CacheCreationInputTokenCost != nil { pricing.InputCacheWrite = bifrost.Ptr(fmt.Sprintf("%.10f", *pricingEntry.CacheCreationInputTokenCost)) + hasPricingField = true } - resp.Data[i].Pricing = pricing + if hasPricingField { + resp.Data[i].Pricing = pricing + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/inference.go` around lines 748 - 764, The code always allocates pricing := &schemas.Pricing{} which causes empty pricing objects to be emitted; change the logic in the block that maps pricingEntry -> pricing so you only create and assign a *schemas.Pricing when at least one of the mapped fields on pricingEntry is non-nil (check InputCostPerToken, OutputCostPerToken, InputCostPerImage, CacheReadInputTokenCost, CacheCreationInputTokenCost), and only then set resp.Data[i].Pricing = pricing; leave resp.Data[i].Pricing nil otherwise.plugins/governance/main.go (1)
1425-1433: Remove the staleselectedKeyIDparameter from this doc comment.
postHookWorkerno longer acceptsselectedKeyIDdirectly; that identifier now comes throughpricingScopes, so the current comment is misleading.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@plugins/governance/main.go` around lines 1425 - 1433, The doc comment for postHookWorker lists a stale parameter selectedKeyID; update the comment to remove that bullet and any mention of selectedKeyID and instead reflect that the selected provider key ID is obtained via pricingScopes (or omit entirely), ensuring the parameter list in the comment matches the actual function signature for postHookWorker and retains accurate descriptions for virtualKey, requestID, userID, isCacheRead/isBatch/isFinalChunk, and pricingScopes.ui/lib/types/governance.ts (1)
3-3: Use the UI path alias here.Keep this consistent with the rest of
ui/liband import from@/lib/types/configinstead of a relative path.♻️ Suggested diff
-import { ModelProviderName, RequestType } from "./config"; +import { ModelProviderName, RequestType } from "@/lib/types/config";Based on learnings, "In the Bifrost codebase, prefer using the
@/libpath alias for imports instead of relative paths from within the ui/lib directory."🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/lib/types/governance.ts` at line 3, The import in governance.ts uses a relative path; update it to use the UI path alias by importing ModelProviderName and RequestType from "@/lib/types/config" instead of "./config" so it matches other files in ui/lib and stays consistent with the project's path-alias convention (look for the import line referencing ModelProviderName and RequestType in governance.ts).ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx (1)
287-306: Don’t style the row as clickable unless it actually does something.
cursor-pointerplusstopPropagation()suggests row-level navigation/editing, but the row has no click handler. Right now users get a click affordance with no action.Smallest cleanup if the row is not meant to open the editor
- <TableRow key={row.id} className="hover:bg-muted/50 cursor-pointer transition-colors"> + <TableRow key={row.id} className="hover:bg-muted/50 transition-colors"> ... - <TableCell className="text-right" onClick={(e) => e.stopPropagation()}> + <TableCell className="text-right">🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx` around lines 287 - 306, The TableRow is styled as clickable but has no click handler; remove the misleading affordance by deleting the "cursor-pointer" class on the TableRow (and any related "hover:bg-muted/50" if you want no hover effect) and remove the onClick={(e) => e.stopPropagation()} from the TableCell, or alternatively wire an actual row click handler (e.g., add a TableRow onClick like onClick={() => openEditor(row)}) if the row should open an editor; update the JSX around TableRow and the TableCell with className and onClick changes accordingly (look for TableRow and the TableCell that currently contains onClick={(e) => e.stopPropagation()}).transports/bifrost-http/handlers/governance.go (1)
3418-3428: Normalize first, then validate.Both write paths call
IsValid()beforenormalizeOptionalString()/TrimSpace(), but persist the normalized values. Building the normalized shape first keeps validation aligned with what actually reaches storage and makes whitespace handling deterministic.Also applies to: 3438-3448, 3491-3529, 3554-3563
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@transports/bifrost-http/handlers/governance.go` around lines 3418 - 3428, The code constructs a modelcatalog.PricingOverride named shape and calls shape.IsValid() before normalizing string fields; change the sequence so you first normalize/trim optional string fields (use normalizeOptionalString and strings.TrimSpace on req.Pattern, req.ProviderKeyID, etc.) and assign those normalized values into shape, then call shape.IsValid(); if IsValid returns an error, SendError as before. Apply the same change to the other create/update blocks that build PricingOverride (the blocks around the other ranges called out) so validation runs against the normalized shape that will be persisted.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@framework/modelcatalog/main.go`:
- Around line 259-261: The background sync worker never reloads scoped pricing
overrides, so non-writing replicas keep stale overrides; update the periodic
sync logic to call mc.loadPricingOverridesFromStore(ctx) on a steady-state
interval (or add a dedicated steady-state refresh) similar to
ForceReloadPricing, and handle errors the same way it does in Init (wrap and
log/return as appropriate). Reference mc.loadPricingOverridesFromStore,
UpsertPricingOverrides, DeletePricingOverride and ForceReloadPricing when adding
this call so the worker will refresh in-memory overrides after remote changes;
ensure the reload runs only in steady-state (non-blocking/debounced) to avoid
contention with writers.
In `@transports/bifrost-http/handlers/pricing_override_test.go`:
- Around line 119-145: The test sends every stored field and both patch keys so
it can't tell full replacement from merge; in the pricing override update test
(handler.updatePricingOverride, using override and stored from
store.GetPricingOverrideByID and stored.PricingPatchJSON) change the request
body to omit at least one persisted field or one patch key (e.g., omit
"patch.output_cost_per_token" or omit "name") and then assert the stored value
for that omitted field remains the previous value while the explicitly updated
keys changed (e.g., check patch.InputCostPerToken changed to 1.0 and
patch.OutputCostPerToken stayed the original value if you omitted it), so the
test actually verifies merge semantics.
In `@ui/lib/types/governance.ts`:
- Line 413: The TypeScript type in ui/lib/types/governance.ts declares the wrong
field name output_cost_per_image_above_1024_and_1024_pixels_and_premium_image
which does not match the pricing schema key
output_cost_per_image_above_1024x1024_pixels_premium used in
framework/configstore/migrations.go; update the property name in the governance
type to exactly output_cost_per_image_above_1024x1024_pixels_premium so the
override key sent in patches matches the server-side pricing schema (ensure any
usages of the old property name in functions, serializers, or tests are also
renamed to the new identifier).
---
Outside diff comments:
In `@framework/modelcatalog/pricing.go`:
- Around line 83-111: calculateBaseCost currently returns
input.usage.Cost.TotalCost before resolvePricing is applied, letting
provider-reported cost bypass governance/override pricing; change the logic in
calculateBaseCost (which uses extractCostInput and later calls resolvePricing)
so that you first call resolvePricing(provider, model, deployment, requestType,
scopes) and use that pricing if present, and only if resolvePricing yields no
pricing entry fall back to using input.usage.Cost.TotalCost (the
provider-reported cost carried via responsesUsageToBifrostUsage). Ensure the
provider cost is used as a last-resort fallback rather than an early return.
In `@plugins/logging/operations.go`:
- Around line 950-1024: The DeserializeFields() call is currently gated only on
TokenUsage/CacheDebug, so modality-specific parsed fields
(TranscriptionOutputParsed, ImageGenerationOutputParsed,
VideoGenerationOutputParsed, SpeechOutputParsed) may remain nil and never get
restored; update the conditional before calling DeserializeFields() to also
check for raw modality fields (e.g., logEntry.TranscriptionOutput != "",
logEntry.ImageGenerationOutput != "", logEntry.VideoGenerationOutput != "",
logEntry.SpeechOutput != "") or for their parsed counterparts being nil, and
call logEntry.DeserializeFields() whenever any modality raw data exists but the
corresponding parsed field (TranscriptionOutputParsed,
ImageGenerationOutputParsed, VideoGenerationOutputParsed, SpeechOutputParsed) is
nil so the later patching blocks can restore usage before
p.pricingManager.CalculateCost(resp, &scopes).
---
Duplicate comments:
In `@core/schemas/tracer.go`:
- Around line 69-71: The Tracer interface change removed the standard
context.Context and is source-breaking; restore the original signature of
PopulateLLMResponseAttributes to accept a context.Context (e.g.,
PopulateLLMResponseAttributes(ctx context.Context, handle SpanHandle, resp
*BifrostResponse, err *BifrostError)) and keep BifrostContext usage additive by
reading Bifrost-specific data from the context via typed context values or by
adding a separate optional method (e.g.,
PopulateLLMResponseAttributesWithBifrost) rather than replacing the context
parameter; update references to PopulateLLMResponseAttributes and any
implementations to use context.Context and extract BifrostContext safely via
context.Value when needed.
In `@docs/openapi/schemas/management/governance.yaml`:
- Around line 1160-1251: The CreatePricingOverrideRequest and
UpdatePricingOverrideRequest schemas allow invalid combinations (e.g.,
scope_kind: provider without provider_id); update both schemas to encode
scope-dependent required fields using oneOf with discriminators or conditional
required blocks that match scope_kind values (e.g., one schema for global, one
for provider requiring provider_id, one for provider_key requiring
provider_key_id, and analogous variants for virtual_key* requiring
virtual_key_id and/or provider_id/provider_key_id), mirroring how RoutingRule
models scope combos so generated clients will fail OpenAPI validation rather
than only runtime.
In `@framework/configstore/clientconfig.go`:
- Around line 970-985: The GeneratePricingOverrideHash function currently hashes
RequestTypesJSON (storage field) which can change independently; instead, hash
the parsed request types slice used by the config (e.g., use p.RequestTypes or
the in-memory parsed []string) in a stable order to avoid reorder-only
diffs—serialize or iterate the slice deterministically (sorted or joined with a
delimiter) and include that in the hash rather than RequestTypesJSON; update
GeneratePricingOverrideHash to reference the parsed slice symbol and remove
RequestTypesJSON from the hashed inputs.
In `@framework/modelcatalog/overrides_test.go`:
- Around line 497-501: The test currently compares tc.expected (float64) to
patched.InputCostPerToken (a *float64); update the assertion to dereference the
pointer and guard for nil: first assert.NotNil(t, patched.InputCostPerToken) (or
require.NotNil if test must fail immediately), then assert.Equal(t, tc.expected,
*patched.InputCostPerToken). Reference mc.applyPricingOverrides and the
patched.InputCostPerToken field when making this change.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3589-3601: The handler currently deletes the pricing override in
storage via configStore.DeletePricingOverride then logs and continues if
governanceManager.DeletePricingOverride fails; change this so that if
governanceManager.DeletePricingOverride(ctx, id) returns an error you return a
500 (use SendError with fasthttp.StatusInternalServerError) instead of returning
success, and implement a compensating path: attempt to re-create or re-sync the
deleted DB record (or call a
configStore.RollbackPricingOverride/PutPricingOverride if available) or call a
governanceManager.SyncFromStore/reload method to reconcile state before
responding; update the code around configStore.DeletePricingOverride and
governanceManager.DeletePricingOverride to bail with an error response when
in-memory deletion/sync fails so DB and memory remain consistent.
- Around line 3314-3316: The handler currently assigns scopeKind from the raw
query string (scope_kind) and forwards it to store calls, allowing invalid
values (e.g., ?scope_kind=foo) to reach GetPricingOverrides*; instead, parse and
validate scope_kind immediately (e.g., map/enum check or parser used elsewhere)
and if invalid return a 4xx before any store call, then reuse the validated
value (the parsed enum/typed variable) in both query branches that call
GetPricingOverrides*; update the code paths around the scopeKind variable
assignment and the two other occurrences referenced (the blocks at the other
GetPricingOverrides* call sites) to use the validated/typed value rather than
the raw string.
In `@transports/bifrost-http/handlers/providers.go`:
- Around line 315-326: The handler currently unmarshals into a struct that omits
the legacy "pricing_overrides" field, which causes old payloads to be silently
accepted and their overrides dropped; change the request handling around
sonic.Unmarshal in this function and in ProviderHandler.addProvider to first
inspect the raw JSON (e.g., unmarshal into a map[string]json.RawMessage or check
for the "pricing_overrides" key using a fast search) and if the
"pricing_overrides" key is present immediately call SendError(ctx,
fasthttp.StatusBadRequest, ...) to reject with 400; keep the existing struct
unmarshal afterward for valid payloads so the rest of the code (payload,
NetworkConfig, CustomProviderConfig, ConcurrencyAndBufferSize, etc.) continues
to work.
In `@transports/bifrost-http/lib/config.go`:
- Around line 1197-1210: The mergeGovernanceConfig path currently swallows
pricing-override serialization/hash errors (in the block using json.Marshal on
newOverride.RequestTypes and configstore.GeneratePricingOverrideHash) by logging
Warn and continuing, which can leave stale DB overrides active; change those
logger.Warn+continue occurrences in mergeGovernanceConfig to return a
descriptive error instead (including newOverride.ID and the underlying err) so
the merge fails closed, and ensure loadGovernanceConfigFromFile propagates that
error up (matching createGovernanceConfigInStore behavior) so startup aborts on
failure to serialize/hash pricing overrides.
In `@ui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsx`:
- Around line 55-57: The effect currently collapses rows whenever a value
becomes an empty string, causing a row to disappear while the user is typing;
update the useEffect that calls setActiveFields so it treats empty strings as
still active (only remove a field from activeFields when its value is
null/undefined or when the explicit "remove/X" action is used). Concretely,
change the PRICING_FIELDS filter in the effect to keep fields where
values[f.key] is an empty string (i.e., only exclude when values[f.key] == null)
or add a short-lived ref/flag to distinguish external resets from user edits and
use that to decide when to rebuild activeFields; reference the useEffect,
setActiveFields, PRICING_FIELDS, and values symbols when making the change.
In `@ui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsx`:
- Around line 282-296: The Label and Input rendered in renderFields lack an
id/htmlFor pair; update the JSX for Label and Input to associate them by adding
a unique id (e.g., `id={`pricing-override-field-${field.key}`}`) on the Input
and `htmlFor` with the same value on the Label so clicking the label focuses the
input and screen readers announce correctly; ensure you use the same
field.key-derived identifier and keep existing props like data-testid, value,
onChange, and className unchanged when adding the id/htmlFor attributes.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 76-80: The "provider_key"/"virtual_key_provider_key" branch uses
provider_key_id to look up key metadata and currently falls back to "-" when
that lookup is stale; update the logic in scopedPricingOverridesView.tsx so that
after computing keyID (override.provider_key_id || ""), you derive providerId =
keyProviderMap.get(keyID) || override.provider_id || "" and then return
providerMap.get(providerId) || providerId || "-". This ensures you display the
override.provider_id when key metadata is missing, while still preferring mapped
names from keyProviderMap and providerMap for functions/variables referenced in
the diff (keyID, keyProviderMap, providerMap, override.provider_id).
In `@ui/app/workspace/virtual-keys/views/virtualKeysTable.tsx`:
- Around line 16-17: The two import statements importing Input and the Select
components (symbols: Input, Select, SelectContent, SelectItem, SelectTrigger,
SelectValue) in virtualKeysTable.tsx are missing trailing semicolons; update
those import lines to end with semicolons so they match surrounding imports and
satisfy Prettier formatting rules.
In `@ui/lib/store/apis/governanceApi.ts`:
- Around line 643-660: The patch must ensure edits that start or stop matching a
cached filtered query are handled: update the matchesQuery predicate in the
update block (the PricingOverrideQueryArgs / matchesQuery logic) to include
args.search (apply the same search filter used by getPricingOverrides), and
inside the governanceApi.util.updateQueryData("getPricingOverrides", ...)
callback, when index === -1 but matchesQuery is true, insert the updated item
into draft.pricing_overrides (e.g., unshift or splice at 0) and increment
draft.count and draft.total_count; keep the existing branch that splices out and
decrements counts when the item exists but no longer matches. Reference symbols:
PricingOverrideQueryArgs, matchesQuery,
governanceApi.util.updateQueryData("getPricingOverrides", ...),
draft.pricing_overrides, index.
In `@ui/lib/types/governance.ts`:
- Line 440: The request_types property is widened to string[] on PricingOverride
and UpdatePricingOverrideRequest, allowing invalid values to pass; change both
PricingOverride.request_types and UpdatePricingOverrideRequest.request_types to
reuse the stricter type used by CreatePricingOverrideRequest (e.g., Replace the
loose string[] with CreatePricingOverrideRequest['request_types'] or extract a
shared RequestType union and reference it) so the same exact type flows from
fetch through edit to submit, updating any related uses or casts of
request_types to match.
---
Nitpick comments:
In `@plugins/governance/main.go`:
- Around line 1425-1433: The doc comment for postHookWorker lists a stale
parameter selectedKeyID; update the comment to remove that bullet and any
mention of selectedKeyID and instead reflect that the selected provider key ID
is obtained via pricingScopes (or omit entirely), ensuring the parameter list in
the comment matches the actual function signature for postHookWorker and retains
accurate descriptions for virtualKey, requestID, userID,
isCacheRead/isBatch/isFinalChunk, and pricingScopes.
In `@transports/bifrost-http/handlers/governance.go`:
- Around line 3418-3428: The code constructs a modelcatalog.PricingOverride
named shape and calls shape.IsValid() before normalizing string fields; change
the sequence so you first normalize/trim optional string fields (use
normalizeOptionalString and strings.TrimSpace on req.Pattern, req.ProviderKeyID,
etc.) and assign those normalized values into shape, then call shape.IsValid();
if IsValid returns an error, SendError as before. Apply the same change to the
other create/update blocks that build PricingOverride (the blocks around the
other ranges called out) so validation runs against the normalized shape that
will be persisted.
In `@transports/bifrost-http/handlers/inference.go`:
- Around line 748-764: The code always allocates pricing := &schemas.Pricing{}
which causes empty pricing objects to be emitted; change the logic in the block
that maps pricingEntry -> pricing so you only create and assign a
*schemas.Pricing when at least one of the mapped fields on pricingEntry is
non-nil (check InputCostPerToken, OutputCostPerToken, InputCostPerImage,
CacheReadInputTokenCost, CacheCreationInputTokenCost), and only then set
resp.Data[i].Pricing = pricing; leave resp.Data[i].Pricing nil otherwise.
In `@ui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsx`:
- Around line 287-306: The TableRow is styled as clickable but has no click
handler; remove the misleading affordance by deleting the "cursor-pointer" class
on the TableRow (and any related "hover:bg-muted/50" if you want no hover
effect) and remove the onClick={(e) => e.stopPropagation()} from the TableCell,
or alternatively wire an actual row click handler (e.g., add a TableRow onClick
like onClick={() => openEditor(row)}) if the row should open an editor; update
the JSX around TableRow and the TableCell with className and onClick changes
accordingly (look for TableRow and the TableCell that currently contains
onClick={(e) => e.stopPropagation()}).
In `@ui/lib/types/governance.ts`:
- Line 3: The import in governance.ts uses a relative path; update it to use the
UI path alias by importing ModelProviderName and RequestType from
"@/lib/types/config" instead of "./config" so it matches other files in ui/lib
and stays consistent with the project's path-alias convention (look for the
import line referencing ModelProviderName and RequestType in governance.ts).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 6fb2b173-8e2e-408a-ba34-62bfc193eb81
⛔ Files ignored due to path filters (3)
cli/go.sumis excluded by!**/*.sumdocs/media/ui-custom-pricing-form.pngis excluded by!**/*.pngdocs/media/ui-custom-pricing-table.pngis excluded by!**/*.png
📒 Files selected for processing (61)
cli/go.modcore/bifrost.gocore/providers/utils/utils.gocore/schemas/provider.gocore/schemas/tracer.godocs/architecture/framework/model-catalog.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/openapi.yamldocs/openapi/paths/management/governance.yamldocs/openapi/schemas/management/governance.yamldocs/providers/custom-pricing.mdxexamples/configs/withpricingoverridesnostore/config.jsonexamples/configs/withpricingoverridessqlite/config.jsonframework/configstore/clientconfig.goframework/configstore/migrations.goframework/configstore/rdb.goframework/configstore/store.goframework/configstore/tables/modelpricing.goframework/configstore/tables/pricingoverride.goframework/configstore/tables/provider.goframework/logstore/tables.goframework/modelcatalog/main.goframework/modelcatalog/main_test.goframework/modelcatalog/overrides.goframework/modelcatalog/overrides_test.goframework/modelcatalog/pricing.goframework/modelcatalog/pricing_test.goframework/modelcatalog/utils.goframework/streaming/audio.goframework/streaming/chat.goframework/streaming/images.goframework/streaming/responses.goframework/streaming/transcription.goframework/tracing/tracer.goplugins/governance/main.goplugins/logging/main.goplugins/logging/operations.goplugins/telemetry/main.gotransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/handlers/pricing_override_test.gotransports/bifrost-http/handlers/providers.gotransports/bifrost-http/lib/config.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/server/server.gotransports/config.schema.jsonui/app/workspace/custom-pricing/overrides/page.tsxui/app/workspace/custom-pricing/overrides/pricingFieldSelector.tsxui/app/workspace/custom-pricing/overrides/pricingOverrideSheet.tsxui/app/workspace/custom-pricing/overrides/pricingOverridesEmptyState.tsxui/app/workspace/custom-pricing/overrides/scopedPricingOverridesView.tsxui/app/workspace/providers/fragments/index.tsui/app/workspace/providers/fragments/pricingOverridesFormFragment.tsxui/app/workspace/virtual-keys/views/virtualKeysTable.tsxui/components/sidebar.tsxui/lib/store/apis/baseApi.tsui/lib/store/apis/governanceApi.tsui/lib/types/config.tsui/lib/types/governance.tsui/lib/types/schemas.ts
💤 Files with no reviewable changes (4)
- ui/app/workspace/providers/fragments/index.ts
- ui/app/workspace/providers/fragments/pricingOverridesFormFragment.tsx
- ui/lib/types/config.ts
- ui/lib/types/schemas.ts
Pratham-Mishra04
force-pushed
the
feat/scoped-pricing-overrides
branch
from
33d9edb to
edb1b46
Compare
Collaborator
|
@coderabbitai full-review |
Contributor
|
✅ Actions performedFull review triggered. |
Collaborator
Merge activity
|
18 tasks
18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
virtual_key_provider_key > virtual_key_provider > virtual_key > provider_key > provider > globalType of change
Affected areas
How to test
Describe the steps to validate this change. Include commands and expected outcomes.
Core / Transports / Plugins
UI
cd ui npm i npx tsc --noEmit npx next build --no-lintFunctional validation (API + behavior)
List overrides
Create global override
Create provider_key override
Patch override
Delete override
Expected outcomes
No new configs or environment variables were added.
Screenshots / Recordings
Breaking changes
Impact and migration instructions
*wildcard.Related issues
Implements scoped pricing overrides feature branch and stacks on top of PR
#1800(02-26-refactor_pricing_module_refactor).Security considerations
Checklist
docs/contributing/README.mdand followed the guidelines