fix: evict stale VK-scoped model configs from in-memory store on ReloadVirtualKey - #4043
Merged
Merged
Conversation
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
📝 WalkthroughWalkthroughAdds GovernanceStore.ScopedModelConfigIDs and LocalGovernanceStore.BumpRateLimitUsageBy; ReloadVirtualKey now evicts stale VK-scoped in-memory model configs after fetching DB results; ConfigStore.UpdateRateLimitUsage accepts an optional variadic *gorm.DB and RDB uses it when present. ChangesScoped configs, rate-limit usage, and VK reload
Sequence Diagram(s)sequenceDiagram
participant Server
participant GovernanceStore
participant DB
Server->>DB: Fetch VK-scoped model configs
Server->>GovernanceStore: ScopedModelConfigIDs(scope, scopeID)
Server->>GovernanceStore: Upsert/update in-memory configs for fetched IDs
Server->>GovernanceStore: Delete scoped IDs not present in fetched set
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Comment |
BearTS
changed the title
ReloadVirtualKey
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
18 tasks
BearTS
marked this pull request as ready for review
June 4, 2026 02:09
Contributor
Author
|
@coderabbitai review |
Contributor
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@transports/bifrost-http/server/server.go`:
- Around line 399-414: The race occurs because staleIDs is built after the DB
fetch so an interleaving ReloadVirtualKey can cause an older run to delete newer
in-memory configs; fix by taking the snapshot of current in-memory VK-scoped
config IDs from
governancePlugin.GetGovernanceStore().ScopedModelConfigIDs(tables.ModelConfigScopeVirtualKey,
id) into staleIDs before performing the DB fetch that produces mcs, then proceed
to delete entries removed from that snapshot only after you
UpdateModelConfigInMemory(ctx, &mcs[i]) for each returned config and finally
call DeleteModelConfigInMemory(ctx, mcID) for any remaining IDs — ensure you
reference the existing functions/vars (store, mcs, id, ctx,
UpdateModelConfigInMemory, DeleteModelConfigInMemory) and move the snapshot code
earlier so eviction is based on the pre-fetch view.
- Around line 396-415: The current ReloadVirtualKey path swallows errors from
GetModelConfigsByScopeAndScopeIDs causing a silent partial reload; change the
branch so that if GetModelConfigsByScopeAndScopeIDs returns an error you
propagate/return that error (or wrap it) from ReloadVirtualKey instead of
proceeding, so VK-scoped model-config in-memory updates
(store.UpdateModelConfigInMemory / store.DeleteModelConfigInMemory) only run
when the DB fetch succeeds; reference the call to
GetModelConfigsByScopeAndScopeIDs and ensure ReloadVirtualKey returns an error
on err != nil rather than treating it as a no-op.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1433cef2-bf40-4937-b508-58fc48fc08f9
📒 Files selected for processing (2)
plugins/governance/store.gotransports/bifrost-http/server/server.go
Contributor
Confidence Score: 5/5The core fix is safe to merge; the eviction logic correctly diffs in-memory state against the DB result and the DB fetch now gates all in-memory mutation. The eviction logic in ReloadVirtualKey is straightforward and correct: snapshot, diff, upsert, delete. The DB fetch is moved before any mutation, so a DB error aborts cleanly. The only notable addition without a call site is BumpRateLimitUsageBy, which is unreachable via the GovernanceStore interface and has no callers, but it does not affect the correctness of existing paths. plugins/governance/store.go — BumpRateLimitUsageBy is dead code and not wired into the interface. Important Files Changed
Reviews (6): Last reviewed commit: "fix: remove stale in memory model config..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
BearTS
force-pushed
the
06-04-fix_remove_stale_in_memory_model_configs
branch
2 times, most recently
from
June 4, 2026 12:48
e8aa278 to
ac4efb8
Compare
BearTS
force-pushed
the
06-04-feat_add_txn_to_update_budget_usage
branch
from
June 4, 2026 15:59
1e6aa2d to
b53c920
Compare
BearTS
force-pushed
the
06-04-fix_remove_stale_in_memory_model_configs
branch
from
June 4, 2026 15:59
ac4efb8 to
2f3a3cf
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@framework/configstore/rdb.go`:
- Around line 3011-3035: The VK cleanup currently only collects legacy BudgetID
from scopedModelConfigs and misses budgets owned via the Budgets relation,
leaving orphaned governance_budgets rows; update the loop over
scopedModelConfigs (the scopedModelConfigs []tables.TableModelConfig iteration)
to also extract IDs from each mc.Budgets collection (and from any nested budget
field names used in tables.TableModelConfig) and add them to budgetIDs,
de-duplicating (e.g., via a map) before running the Delete on TableBudget; keep
the existing handling for mc.BudgetID and mc.RateLimitID and ensure budgetIDs is
populated with both legacy and relation-owned budget IDs prior to the
txDB.Delete call.
In `@plugins/governance/store.go`:
- Around line 461-481: The BumpRateLimitUsageBy method currently accepts
"arbitrary token and request deltas" but doesn't state whether negatives are
allowed; add explicit validation or documentation: update the
BumpRateLimitUsageBy function to either reject negative deltas (e.g., check
tokenDelta < 0 || requestDelta < 0 and return a descriptive error) or add a
clear comment/docstring on BumpRateLimitUsageBy indicating that negative deltas
are permitted for corrections and ensure any subsequent logic (the clone
adjustments and CompareAndSwap on gs.rateLimits) safely supports decreasing
usage; pick one approach and apply consistently in the function and its callers.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 82cd2c48-d1a9-4cbd-9973-ea0571713aaa
📒 Files selected for processing (5)
framework/configstore/rdb.goframework/configstore/store.goplugins/governance/store.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/server/server.go
💤 Files with no reviewable changes (2)
- transports/bifrost-http/server/server.go
- transports/bifrost-http/lib/config_test.go
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@framework/configstore/rdb.go`:
- Around line 3011-3035: The VK cleanup currently only collects legacy BudgetID
from scopedModelConfigs and misses budgets owned via the Budgets relation,
leaving orphaned governance_budgets rows; update the loop over
scopedModelConfigs (the scopedModelConfigs []tables.TableModelConfig iteration)
to also extract IDs from each mc.Budgets collection (and from any nested budget
field names used in tables.TableModelConfig) and add them to budgetIDs,
de-duplicating (e.g., via a map) before running the Delete on TableBudget; keep
the existing handling for mc.BudgetID and mc.RateLimitID and ensure budgetIDs is
populated with both legacy and relation-owned budget IDs prior to the
txDB.Delete call.
In `@plugins/governance/store.go`:
- Around line 461-481: The BumpRateLimitUsageBy method currently accepts
"arbitrary token and request deltas" but doesn't state whether negatives are
allowed; add explicit validation or documentation: update the
BumpRateLimitUsageBy function to either reject negative deltas (e.g., check
tokenDelta < 0 || requestDelta < 0 and return a descriptive error) or add a
clear comment/docstring on BumpRateLimitUsageBy indicating that negative deltas
are permitted for corrections and ensure any subsequent logic (the clone
adjustments and CompareAndSwap on gs.rateLimits) safely supports decreasing
usage; pick one approach and apply consistently in the function and its callers.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 82cd2c48-d1a9-4cbd-9973-ea0571713aaa
📒 Files selected for processing (5)
framework/configstore/rdb.goframework/configstore/store.goplugins/governance/store.gotransports/bifrost-http/lib/config_test.gotransports/bifrost-http/server/server.go
💤 Files with no reviewable changes (2)
- transports/bifrost-http/server/server.go
- transports/bifrost-http/lib/config_test.go
🛑 Comments failed to post (2)
framework/configstore/rdb.go (1)
3011-3035:
⚠️ Potential issue | 🟠 Major | ⚡ Quick winDeleteVirtualKey cleanup misses has-many model-config budgets.
This path deletes VK-scoped model configs but only collects legacy
BudgetID. Model configs that own budgets throughBudgetscan leave orphanedgovernance_budgetsrows after VK deletion.💡 Suggested fix
var scopedModelConfigs []tables.TableModelConfig if err := txDB.WithContext(ctx). + Preload("Budgets"). Where("scope = ? AND scope_id = ?", tables.ModelConfigScopeVirtualKey, id). Find(&scopedModelConfigs).Error; err != nil { return err } budgetIDs := make([]string, 0, len(scopedModelConfigs)) rateLimitIDs := make([]string, 0, len(scopedModelConfigs)) for _, mc := range scopedModelConfigs { + for i := range mc.Budgets { + budgetIDs = append(budgetIDs, mc.Budgets[i].ID) + } if mc.BudgetID != nil { budgetIDs = append(budgetIDs, *mc.BudgetID) } if mc.RateLimitID != nil { rateLimitIDs = append(rateLimitIDs, *mc.RateLimitID) } }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.var scopedModelConfigs []tables.TableModelConfig if err := txDB.WithContext(ctx). Preload("Budgets"). Where("scope = ? AND scope_id = ?", tables.ModelConfigScopeVirtualKey, id). Find(&scopedModelConfigs).Error; err != nil { return err } budgetIDs := make([]string, 0, len(scopedModelConfigs)) rateLimitIDs := make([]string, 0, len(scopedModelConfigs)) for _, mc := range scopedModelConfigs { for i := range mc.Budgets { budgetIDs = append(budgetIDs, mc.Budgets[i].ID) } if mc.BudgetID != nil { budgetIDs = append(budgetIDs, *mc.BudgetID) } if mc.RateLimitID != nil { rateLimitIDs = append(rateLimitIDs, *mc.RateLimitID) } } if err := txDB.WithContext(ctx). Where("scope = ? AND scope_id = ?", tables.ModelConfigScopeVirtualKey, id). Delete(&tables.TableModelConfig{}).Error; err != nil { return err } if len(budgetIDs) > 0 { if err := txDB.WithContext(ctx).Delete(&tables.TableBudget{}, "id IN ?", budgetIDs).Error; err != nil { return err } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@framework/configstore/rdb.go` around lines 3011 - 3035, The VK cleanup currently only collects legacy BudgetID from scopedModelConfigs and misses budgets owned via the Budgets relation, leaving orphaned governance_budgets rows; update the loop over scopedModelConfigs (the scopedModelConfigs []tables.TableModelConfig iteration) to also extract IDs from each mc.Budgets collection (and from any nested budget field names used in tables.TableModelConfig) and add them to budgetIDs, de-duplicating (e.g., via a map) before running the Delete on TableBudget; keep the existing handling for mc.BudgetID and mc.RateLimitID and ensure budgetIDs is populated with both legacy and relation-owned budget IDs prior to the txDB.Delete call.plugins/governance/store.go (1)
461-481:
⚠️ Potential issue | 🟡 Minor | ⚡ Quick winClarify whether negative deltas are allowed or add validation.
The function comment mentions "arbitrary token and request deltas" but also states it's "used to fold accumulated usage carried from another rate limit," which suggests positive values. Should negative deltas be allowed for usage corrections, or should they be rejected? Consider either:
- Adding validation:
if tokenDelta < 0 || requestDelta < 0 { return fmt.Errorf("deltas must be non-negative") }, or- Documenting explicitly that negative deltas are allowed for corrections.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/governance/store.go` around lines 461 - 481, The BumpRateLimitUsageBy method currently accepts "arbitrary token and request deltas" but doesn't state whether negatives are allowed; add explicit validation or documentation: update the BumpRateLimitUsageBy function to either reject negative deltas (e.g., check tokenDelta < 0 || requestDelta < 0 and return a descriptive error) or add a clear comment/docstring on BumpRateLimitUsageBy indicating that negative deltas are permitted for corrections and ensure any subsequent logic (the clone adjustments and CompareAndSwap on gs.rateLimits) safely supports decreasing usage; pick one approach and apply consistently in the function and its callers.
BearTS
force-pushed
the
06-04-fix_remove_stale_in_memory_model_configs
branch
from
June 4, 2026 16:17
2f3a3cf to
8d85c06
Compare
Contributor
akshaydeo
changed the base branch from
06-04-feat_add_txn_to_update_budget_usage
to
graphite-base/4043
June 4, 2026 16:31
akshaydeo
force-pushed
the
06-04-fix_remove_stale_in_memory_model_configs
branch
from
June 4, 2026 16:32
8d85c06 to
9976675
Compare
18 tasks
akshaydeo
added a commit
that referenced
this pull request
Jun 6, 2026
## Summary This PR bumps the Go toolchain version from `1.26.3` to `1.26.4` across all modules and CI workflows, and cuts a new release (`core` v1.5.17, `framework` v1.3.17, `transports` v1.5.9, `plugins/compat` v0.1.16, `plugins/governance` v1.5.17, and associated plugin versions) incorporating a large batch of features and fixes accumulated since the previous release. ## Changes - **Go 1.26.4** — Updated `go-version` in all GitHub Actions workflows (`e2e-tests`, `helm-release`, `pr-tests`, `release-cli`, `release-pipeline`, `snyk`) and all `go.mod` files (core, framework, transports, cli, all plugins, examples, and test modules). - **Core (v1.5.17)** — OpenAI compaction support, multi-customer logs and usage tracking, multiple team/business unit support, `request_headers` wildcard pattern capture for OTel and Maxim plugins, xAI `x_search` tool, fetch URL validation with SSRF hardening, `file://` pricing URL scheme, virtual key provider fan-out filtering, and a broad set of fixes including Anthropic prompt cache key, empty thinking block stripping, OpenAI stream usage event cleanup, Gemini numeric schema constraints, stale connection retries, Azure Claude diagnostic strip, and passthrough budget handling. - **Framework (v1.3.17)** — Scope-aware budgets and limits wired from model configs, provider-level governance, multiple customer budget support with `calendar_aligned` windows, paginated virtual key fetch, `config.json` source-of-truth flow, FTS index cap reduction, sync worker drift fix, cascade deletes for model configs, and high-scale virtual key flow improvements. - **Transports (v1.5.9)** — Full changelog covering all of the above plus UI improvements (log navigation, customer detail sheet, `BudgetDisplay` component, inline loading shell, materialized view alias), SCIM provisioning fields, Helm/config schema additions (`roles`, `per_user_oauth`), client IP resolution from forwarded headers, and dependency upgrades (`recharts` to 3.8.1, `golang.org/x` CVE remediation). - **Plugins** — `governance` v1.5.17 adds team budget/rate-limit exporters, ghost node reconciliation fix, and VK double usage counting fix; `logging` v1.5.17 adds wildcard header capture and file attachment rendering; `otel` v1.2.17 adds `disable_content_logging` and multiple collectors support; `maxim` v1.6.17 adds `request_headers` wildcard capture; `compat` v0.1.16 fixes `max_tokens` preservation during param filtering. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Verify Go version go version # should report go1.26.4 # Run core tests cd core && go test ./... # Run framework tests cd framework && go test ./... # Run transports tests cd transports && go test ./... # Run plugin tests cd plugins/governance && go test ./... cd plugins/logging && go test ./... cd plugins/otel && go test ./... # UI cd ui pnpm i pnpm build pnpm test ``` ## Breaking changes - [ ] Yes - [x] No ## Related issues #4053, #4066, #4041, #4012, #3976, #3947, #3991, #4045, #3957, #3938, #3937, #3939, #3981, #3998, #3997, #4092, #4091, #4079, #4080, #4086, #3929, #3994, #4028, #3970, #3919, #3861, #3664, #3999, #4088, #4070, #4051, #4043, #4057, #4023, #3941, #3955, #4024, #3956, #3967, #3925, #3992, #3900 ## Security considerations - Fetch URL validation hardened against SSRF by tightening IP checks for private networks and link-local addresses (#4092, #3947, #3991). - Transitive `golang.org/x` dependencies (crypto, net, sys, text) bumped to address Docker Scout CVEs (#3900). ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * OpenAI compaction, multi-customer/team logstore support, request-header wildcard capture, enhanced governance (provider-level & scope-aware limits), disable-content-logging option, support for multiple OpenTelemetry collectors, SSRF hardening and URL validation. * **Chores** * Bumped Go toolchain across modules and updated component/plugin version releases. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Merged
akshaydeo
pushed a commit
that referenced
this pull request
Jun 7, 2026
akshaydeo
added a commit
that referenced
this pull request
Jun 7, 2026
## ✨ Features - **OpenAI Compaction** — Added OpenAI conversation compaction support across core, framework, logging, and the API surface (#4053) - **Multi-Customer & Org Hierarchy** — Logs and usage tracking now support multiple customers, teams, and business units, including business unit CRUD, team assignment, and governance endpoints in the OpenAPI spec (#4066, #4041, #4082) - **Provider-Level Governance** — Budgets & limits are now scope-aware and can be applied at the virtual-key top level and per provider, wired from the model configs table, with UI filters for scope and providers (#3938, #3937, #3939, #3981, #3962) - **Customer Budgets** — Customers support multiple budgets and `calendar_aligned` budget windows (#3998, #3997) - **Virtual Key Attribution & Controls** — Added a `created_by` user attribution column and a `blacklisted_models` column for virtual key provider configs (#3672, #3653) - **Request Header Capture** — OTel and Maxim observability plugins capture `request_headers` by pattern, with wildcard support (e.g. `x-custom-*`); logging gained the same wildcard header capture (#4012, #3958) - **OTel Content Controls & Collectors** — New `disable_content_logging` option drops message/tool content from exported spans, plus support for multiple OTel collectors (#4064, #3894) - **xAI x_search** — Added xAI `x_search` tool support (#3976) - **URL Validation** — Added fetch URL validation with private-network configuration and link-local blocking (#3947, #3991) - **File Scheme Pricing URLs** — Pricing source URLs now accept the `file://` scheme for air-gapped and self-hosted deployments (#4045) - **Paginated Virtual Keys** — Virtual key fetching is paginated to handle deployments with very large numbers of keys (#3957) - **Client IP Resolution** — Resolve client IP from `X-Forwarded-For`/`X-Real-IP` headers - **SCIM Provisioning** — Added `attributeType`/`attributeValue` SCIM provisioning fields - **Helm/Config Schema** — Added `roles` RBAC governance config and `per_user_oauth` MCP auth to the Helm chart and config schema (#4004, #4009) - **Log Navigation UI** — Added a "View logs" menu item to customer, team, and virtual key tables, clickable links in log detail views, a customer detail sheet, and a reusable `BudgetDisplay` component (#4073, #4054, #4026, #4055) - **Faster First Paint** — Added an inline loading shell to `#root` before React mounts (#4063) - **Materialized View Alias** — Added an `alias` column to the materialized view with filter support (#4078) ## 🐞 Fixed - **Fetch URL IP Checks** — Hardened fetch URL IP checks against SSRF (#4092) - **Mantle Model Matching** — Broadened Mantle model matching to all `gpt` variants (#4091) - **Empty Thinking Blocks** — Strip thinking blocks when the signature is empty (#4079) - **OpenAI Stream Usage** — Removed usage from the `responses.created` event in the OpenAI stream (#4080) - **Prompt Cache Key** — Set the prompt cache key from the Anthropic integration (#4086) - **Upstream Failure Status** — Map upstream connection failures to 502 instead of 400 (#3929) (thanks [@chris-colinsky](https://github.com/chris-colinsky)!) - **Gemini Schema Constraints** — Accept numeric schema integer constraints for Gemini (#3994) (thanks [@yanhao98](https://github.com/yanhao98)!) - **Files Provider Param** — Accept the `?provider=` query param on `GET /v1/files` (#3971) (thanks [@alexef](https://github.com/alexef)!) - **Optional Batch Model** — Made the `model` field optional on `POST /v1/batches` (#3973) (thanks [@alexef](https://github.com/alexef)!) - **Helm Azure Config** — Added missing `azure_key_config` fields to the Helm schema (#3996) (thanks [@axelray-dev](https://github.com/axelray-dev)!) - **Text Completion Chunk Model** — Added the missing `Model` field to `TextCompletionChunkResponse` (#3970) (thanks [@kuishou68](https://github.com/kuishou68)!) - **MCP Inline stdio Env** — MCP stdio server configs accept inline environment variable assignments (#3861) (thanks [@Shushmitaaaa](https://github.com/Shushmitaaaa)!) - **Orphaned Tool Results** — Orphaned tool results in the OpenAI to Anthropic conversion flow are no longer rejected by the Anthropic API (#3919) - **Node Usage Reconciliation** — Added a monotonic `inc_number` log cursor so node usage reconciliation does not skip late async log writes (#3664) - **Bedrock Output Assessments** — Corrected the type of `outputAssessments` in Bedrock responses (#4028) - **Model Pool Pricing Reloads** — Preserve non-pricing model pool entries across pricing reloads (#3999) - **Ghost Node Reconciliation** — Replicate the VK hierarchy flow for ghost node reconciliation (#4088) - **VK Double Usage Counting** — Fixed double usage counting when creating a virtual key (#4070) - **Model Config Lifecycle** — Cascade deletes for model configs and removal of stale in-memory model configs (#4051, #4043) - **FTS Index Cap** — Reduced the FTS index `left()` cap from 800k to 250k chars to stay within the tsvector limit (#4057) - **Sync Worker Drift** — Reduced the sync worker ticker period to 5m to prevent threshold drift (#4023) - **Passthrough** — Fixed passthrough budgets, gated passthrough models per VK, model extraction for Azure passthrough, and restricted fallbacks/provider selection to the VK boundary (#3941, #3988, #3983, #3924) - **Provider Response Headers** — Strip provider response headers and add a content-type filter (#3955, #4024) - **Stream Handling** — Drain non-SSE stream readers and retry stale connections (#3956, #3967) - **Azure Claude** — Strip Azure diagnostic property for Claude models (#3925) - **Compat max_tokens** — Preserve chat `max_tokens` during param filtering (#3992) - **Raw Request Flag** — Removed the raw request flag from providers that don't support it (#4058) - **UI Fixes** — Standardized page container layout, virtual key model configs UI, and dashboard chart tooltips (#4046, #4052, #4044) ## 🔧 Maintenance - **Dependency Upgrades** — Bumped transitive `golang.org/x` dependencies (crypto, net, sys, text) for Docker Scout CVE remediation and `recharts` to 3.8.1; cascaded version bumps across all modules (#3900, #4003)
18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a virtual key's governance model configs are deleted from the DB (e.g. when a standalone VK is adopted into an access profile and its VK-scoped model configs are removed), the in-memory store was never evicted of those stale entries. This caused deleted budget/rate-limit configs to continue enforcing against requests.
Changes
ScopedModelConfigIDs(scope, scopeID string) []stringtoGovernanceStoreinterface and implemented it onLocalGovernanceStore. It iterates the in-memory model config map and returns the IDs of all entries matching the given scope and scope ID.ReloadVirtualKeyin the HTTP server to snapshot in-memory VK-scoped config IDs before upserting the DB results, then delete any IDs that were not present in the DB response, evicting stale entries.Type of change
Affected areas
How to test
ReloadVirtualKeyfor that VK.go test ./plugins/governance/... ./transports/bifrost-http/...Breaking changes
Related issues
Security considerations
None. This change only affects in-memory eviction of governance configs and does not touch auth, secrets, or PII.
Checklist
docs/contributing/README.mdand followed the guidelinesSummary by CodeRabbit
New Features
Bug Fixes
Tests