Repository navigation
fix(db): cache getPricingForModel() to stop /api/usage/history event-loop stall (#13891) - #14040
Merged
diegosouzapw merged 1 commit intoSep 18, 2026
Merged
Conversation
getPricingForModel() called the uncached getPricing() on every invocation instead of the existing getCachedPricing() helper (30s TTL, readCache.ts), so usageStats.getUsageStats() re-ran a 3-SELECT + JSON.parse + merge cycle against key_value once per GROUP BY row -- up to 531 times on a large usage_history table -- blocking the event loop for several seconds on /api/usage/history. Every known pricing writer (updatePricing, LiteLLM/models.dev sync) already invalidates this cache via touchPricing()/invalidateDbCache, so a write remains immediately visible; added a regression test that proves both the cache hit path and the invalidation path.
Collaborator
Author
|
Validated in local merge-train /tmp/mt-train3b.log on 192.168.0.113 @ train tip 408e2128791696a18966681953136fc96aeb99b0 (32 PRs boarded): static gates green; full test:unit 40694 tests, 17 failing — every one reproduces on the pure release tip (base-red sweep list), zero new reds. Merged --admin per merge-gates §4/§7. |
muhamadgalihsaputra
pushed a commit
to niyatna/NiyatnaRoute
that referenced
this pull request
Sep 27, 2026
…nference#13891) (cheaper-inference#14040) getPricingForModel() called the uncached getPricing() on every invocation instead of the existing getCachedPricing() helper (30s TTL, readCache.ts), so usageStats.getUsageStats() re-ran a 3-SELECT + JSON.parse + merge cycle against key_value once per GROUP BY row -- up to 531 times on a large usage_history table -- blocking the event loop for several seconds on /api/usage/history. Every known pricing writer (updatePricing, LiteLLM/models.dev sync) already invalidates this cache via touchPricing()/invalidateDbCache, so a write remains immediately visible; added a regression test that proves both the cache hit path and the invalidation path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #13891
Root cause
getPricingForModel()(src/lib/db/settings/pricing.ts) called the uncachedgetPricing()on every invocation instead of thegetCachedPricing()helperthat already exists in
src/lib/db/readCache.ts(30s TTL, invalidated onevery pricing write via
touchPricing()->invalidateDbCache("pricing")).usageStats.getUsageStats()callscalculateAggregateCost()->calculateCost()->
getPricingForModel()once per GROUP BY row — up to 531 times on thereporter's 45k-row
usage_historytable — so every/api/usage/historyrequestre-ran a synchronous 3-SELECT +
JSON.parse+ merge cycle againstkey_valuehundreds of times, blocking the event loop for ~5s.
Fix: swap the internal call inside
getPricingForModel()fromgetPricing()togetCachedPricing(). The exportedgetPricing()/getPricingWithSources()functions are untouched — callers that need a guaranteed-fresh read (pricing
admin route, sync jobs) keep using them directly.
I audited every pricing writer (
pricing.ts,pricingSync.ts,modelsDevSync.ts) — all three namespaces (pricing,pricing_synced,models_dev_pricing) already callinvalidateDbCache("pricing")on write, sothis does not introduce any new staleness window beyond the existing 30s TTL
that other callers of
getCachedPricing()already tolerate.Regression test
tests/unit/pricing-for-model-cached-13891.test.tsgetPricingForModel()triggered 60
db.prepare()calls (3 SELECTs × 20, zero caching).db.prepare()calls.updatePricing()returns the old value, and the very next lookup after the write returns the
fresh value — confirming a stale price cannot silently persist past a write.
Gates run
node scripts/check/check-file-size.mjs— no violation on touched filesnode scripts/check/check-complexity.mjs— OK (2857 ≤ baseline 3218)node scripts/check/check-cognitive-complexity.mjs— OK (1291 ≤ baseline 1437)npm run typecheck:core— exit 0npx eslint --suppressions-location config/quality/eslint-suppressions.json src/lib/db/settings/pricing.ts tests/unit/pricing-for-model-cached-13891.test.ts— exit 0pricing-sync-memoization.test.ts(3/3),pricing-route-sources.test.ts(1/1),pricing-sync-extended.test.ts(6/6),tier-pricing-cache.test.ts(4/4),db-read-cache.test.ts(10/10)node scripts/check/check-changelog-integrity.mjs— OK