Skip to content

fix(db): cache getPricingForModel() to stop /api/usage/history event-loop stall (#13891) - #14040

Merged
diegosouzapw merged 1 commit into
release/v3.8.51from
fix/13891-usage-history-pricing-cache
Sep 18, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.51from
fix/13891-usage-history-pricing-cache

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Closes #13891

Root cause

getPricingForModel() (src/lib/db/settings/pricing.ts) called the uncached
getPricing() on every invocation instead of the getCachedPricing() helper
that already exists in src/lib/db/readCache.ts (30s TTL, invalidated on
every pricing write via touchPricing() -> invalidateDbCache("pricing")).

usageStats.getUsageStats() calls calculateAggregateCost() -> calculateCost()
-> getPricingForModel() once per GROUP BY row — up to 531 times on the
reporter's 45k-row usage_history table — so every /api/usage/history request
re-ran a synchronous 3-SELECT + JSON.parse + merge cycle against key_value
hundreds of times, blocking the event loop for ~5s.

Fix: swap the internal call inside getPricingForModel() from getPricing() to
getCachedPricing(). The exported getPricing() / getPricingWithSources()
functions are untouched — callers that need a guaranteed-fresh read (pricing
admin route, sync jobs) keep using them directly.

I audited every pricing writer (pricing.ts, pricingSync.ts,
modelsDevSync.ts) — all three namespaces (pricing, pricing_synced,
models_dev_pricing) already call invalidateDbCache("pricing") on write, so
this does not introduce any new staleness window beyond the existing 30s TTL
that other callers of getCachedPricing() already tolerate.

Regression test

tests/unit/pricing-for-model-cached-13891.test.ts

  • RED (pre-fix, run against unfixed code): 20 calls to getPricingForModel()
    triggered 60 db.prepare() calls (3 SELECTs × 20, zero caching).
  • GREEN (post-fix): the same 20 calls trigger ≤3 db.prepare() calls.
  • A second test proves cache invalidation: a lookup before updatePricing()
    returns the old value, and the very next lookup after the write returns the
    fresh value — confirming a stale price cannot silently persist past a write.
node --import tsx/esm --test tests/unit/pricing-for-model-cached-13891.test.ts
# tests 2, pass 2, fail 0

Gates run

  • node scripts/check/check-file-size.mjs — no violation on touched files
  • node scripts/check/check-complexity.mjs — OK (2857 ≤ baseline 3218)
  • node scripts/check/check-cognitive-complexity.mjs — OK (1291 ≤ baseline 1437)
  • npm run typecheck:core — exit 0
  • npx eslint --suppressions-location config/quality/eslint-suppressions.json src/lib/db/settings/pricing.ts tests/unit/pricing-for-model-cached-13891.test.ts — exit 0
  • New test: 2/2 pass (RED→GREEN evidence above)
  • Existing pricing regression suite — all pass:
    pricing-sync-memoization.test.ts (3/3), pricing-route-sources.test.ts (1/1),
    pricing-sync-extended.test.ts (6/6), tier-pricing-cache.test.ts (4/4),
    db-read-cache.test.ts (10/10)
  • node scripts/check/check-changelog-integrity.mjs — OK

⚠️ base-red inherited: #14004 — docs env/docs contract (fixed separately in #14022), chatHelpers file-size drift

getPricingForModel() called the uncached getPricing() on every
invocation instead of the existing getCachedPricing() helper
(30s TTL, readCache.ts), so usageStats.getUsageStats() re-ran a
3-SELECT + JSON.parse + merge cycle against key_value once per
GROUP BY row -- up to 531 times on a large usage_history table --
blocking the event loop for several seconds on /api/usage/history.

Every known pricing writer (updatePricing, LiteLLM/models.dev sync)
already invalidates this cache via touchPricing()/invalidateDbCache,
so a write remains immediately visible; added a regression test that
proves both the cache hit path and the invalidation path.
@diegosouzapw

Copy link
Copy Markdown
Collaborator Author

Validated in local merge-train /tmp/mt-train3b.log on 192.168.0.113 @ train tip 408e2128791696a18966681953136fc96aeb99b0 (32 PRs boarded): static gates green; full test:unit 40694 tests, 17 failing — every one reproduces on the pure release tip (base-red sweep list), zero new reds. Merged --admin per merge-gates §4/§7.

@diegosouzapw
diegosouzapw merged commit a96c838 into release/v3.8.51 Sep 18, 2026
14 of 21 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…nference#13891) (cheaper-inference#14040)

getPricingForModel() called the uncached getPricing() on every
invocation instead of the existing getCachedPricing() helper
(30s TTL, readCache.ts), so usageStats.getUsageStats() re-ran a
3-SELECT + JSON.parse + merge cycle against key_value once per
GROUP BY row -- up to 531 times on a large usage_history table --
blocking the event loop for several seconds on /api/usage/history.

Every known pricing writer (updatePricing, LiteLLM/models.dev sync)
already invalidates this cache via touchPricing()/invalidateDbCache,
so a write remains immediately visible; added a regression test that
proves both the cache hit path and the invalidation path.
@diegosouzapw
diegosouzapw deleted the fix/13891-usage-history-pricing-cache branch October 18, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(backend): /api/usage/history blocks the event loop ~5s — uncached getPricing() rebuild per row

1 participant