fix(lib): memoize catalog pricing/capability lookups to fix cold /v1/models freeze (#8697)

[maxmad64bis]

⚠️ Availability impact

This fixes a server-wide freeze, not a slow endpoint. While a cold /v1/models catalog rebuild runs, the whole Node event loop is blocked synchronously — every other in-flight request stalls too (dashboard, other API keys, health checks), for the full duration. Reporter measured ~15s on a 9-API-key Docker deployment (#8697); profiling on a larger local catalog measured 41-54s. On any instance with the models.dev sync enabled (default-on catalog resolution path), the first /v1/models call after a cold start or after the ~60s catalog memo TTL expires pays this cost, and it recurs roughly every minute per distinct API key (#8219 capped the memo TTL at 60s; the memo is also keyed per-API-key per #6440, so multi-key deployments pay it multiple times over).

Root cause & how we got here

This is a nice example of a cost that's essentially invisible in isolation and only shows up once a project has grown a lot. A handful of small, individually reasonable helpers — case-insensitive lookups, reverse-alias maps — were added over the last few months as the model catalog and its feature set kept expanding (MODEL_SPECS alone grew from 5 entries to 56, and models.dev itself started at "109 providers, 4,146+ models"). Each one is cheap on its own; none of them were ever slow in a unit test with a handful of fixture models. It's only once the real catalog reached thousands of entries that the combined per-model cost of these lookups became visible as a multi-second stall instead of noise — nothing here is a classic regression (nothing that used to be fast got slower), just a cost curve that only bends upward once you're operating at real scale.

This also matches the reporter's own note on #8697 — "Subjectively this was much faster on 3.8.48 — I have not bisected" — there's no single commit to point to; it's a threshold crossed gradually as the project (happily) kept growing.

Summary

• Fixes the cold GET /v1/models catalog rebuild freezing the entire server for 41-54s (issue fix(api): cold GET /v1/models blocks ~15s per API key — memo should serve last snapshot + async refresh #8697) by fixing 6 instances of the same missing-memoization pattern — a per-model function rescanning a static or synced data structure (Object.entries()/Object.keys(), or a raw SQLite query) on every call instead of once per rebuild:
  1. getModelsDevPricing() (src/lib/modelsDevSync.ts) — re-ran a synchronous SQLite query and re-JSON.parse'd ~180 blobs on every call (up to ~6091x instead of once per request). Memoized via the existing modelCatalogCacheVersion invalidation signal (same pattern as getCachedRawProviderConnections/getCachedProviderNodes in db/readCache.ts). Dominant cost of the original 41-54s freeze.
  2. findInsensitive() (src/lib/modelMetadataRegistry.ts, resolveCatalogPricing) — rebuilt a full Object.entries() scan on every case-insensitive lookup miss, twice per model. Replaced with a lowercase-key index built once per distinct pricing object and cached by identity (WeakMap); warns once at index-build time if two keys collide case-insensitively instead of silently discarding the second value.
  3. getSyncedCapability() (src/lib/modelsDevSync.ts) — ran a per-model SQLite SELECT on a cold cache instead of self-warming the whole-table cache; no caller in the /v1/models build path ever primed it, so a cold rebuild ran one SQLite round-trip per model per call site. Now self-warms via the existing bulk getSyncedCapabilities() on first miss. ~70% of a full catalog sweep's remaining cost after fixes 1-2.
  4. getCanonicalModelSpecId() (src/shared/constants/modelSpecs.ts) — up to 3 separate linear scans over the static MODEL_SPECS table per call (exact case-insensitive, alias case-insensitive, prefix). Replaced with a lazy lowercase-key index built once (MODEL_SPECS never changes at runtime); prefix-match iteration order preserved exactly so resolution outcomes are unchanged.
  5. getStaticSpecCanonicalModelId() (src/lib/modelCapabilities.ts) — duplicated the same exact+alias scan as (4) in a second, independent rescan. Now reuses the shared index via a new exported helper (findModelSpecIdByExactOrAlias) instead of maintaining a second cache over the same static table. Also fixed reverseModelsDevProviders() in the same file — rescanned Object.entries(MODELS_DEV_PROVIDER_MAP) (also static) on every call; memoized by provider key, result frozen (readonly string[]) since it is now shared across calls instead of freshly allocated each time.
  6. resolveModelAlias() (src/shared/constants/modelSpecs.ts) — rescanned Object.entries(MODEL_SPECS) unconditionally once per model. Case-sensitive exact match (Array.includes(), no .toLowerCase()) — uses a dedicated exact-match index rather than the case-insensitive alias index from fix 4/5, which would have silently broadened matches and changed behavior.
• All fixes are local to these functions — no architecture change, no behavior change on any other code path (verified by re-running every pre-existing consumer test suite).
• Measured on a 1940-pair real-catalog sample (static PROVIDER_MODELS registry): cold sweep 828ms → 356ms after fixes 3-6 on top of 1-2, extrapolating to roughly ~1s on the real ~6091-model catalog — down from the original 41-54s freeze.
• Complementary to the stale-serve fix in [v3.8.50] fix(api): serve last /v1/models snapshot instead of blocking on cold rebuild (#8697) #8801 (upstream) — neither alone eliminates the freeze; this PR addresses the root computational cost, [v3.8.50] fix(api): serve last /v1/models snapshot instead of blocking on cold rebuild (#8697) #8801 addresses client-facing staleness during a rebuild.

Related Issues

• Related to fix(api): cold GET /v1/models blocks ~15s per API key — memo should serve last snapshot + async refresh #8697
• Related to [v3.8.50] fix(api): serve last /v1/models snapshot instead of blocking on cold rebuild (#8697) #8801

Validation

• Focused tests for the change: node --import tsx/esm --test tests/unit/*8697*.test.ts tests/integration/modelsDevSync.test.ts
• npm run lint (clean on touched files)
• npm run typecheck:core (clean)
• Production-code changes include a new or updated automated test in this PR
• SonarQube PR analysis — pending CI

Tests Added Or Updated

• tests/unit/models-dev-pricing-memoization-8697.test.ts (new) — getModelsDevPricing() hits the DB at most once across repeated reads within the same cache version; returns fresh data immediately after a write invalidates the cache.
• tests/unit/catalog-pricing-lookup-index-8697.test.ts (new) — case-insensitive pricing resolution stays correct; Object.entries() call count stays constant (not scaling with iteration count) across repeated catalog-scale lookups.
• tests/unit/synced-capability-warmup-8697.test.ts (new) — getSyncedCapability() runs at most 2 db.prepare() calls (bulk load + table guard) across 200 distinct model lookups, not one round-trip per lookup.
• tests/unit/model-spec-lookup-index-8697.test.ts (new) — getCanonicalModelSpecId() stays correct for case-insensitive/unknown ids; 0 Object.entries()/Object.keys() calls across 500 repeated lookups once the index is warm.
• tests/unit/reverse-models-dev-providers-8697.test.ts (new) — indirect correctness via getResolvedModelCapabilities(); 0 scans of MODELS_DEV_PROVIDER_MAP specifically across 300 repeated calls for the same provider.
• tests/unit/resolve-model-alias-index-8697.test.ts (new) — resolveModelAlias() stays correct (exact alias resolves, case-varied input does NOT resolve — preserving case-sensitive semantics, unknown input passes through unchanged); 0 Object.entries() calls across 500 repeated lookups once the index is warm.
• Re-ran every pre-existing consumer suite to confirm no regression: catalog-pricing-surface-8018, model-capabilities-specialty-8016, effort-thinking-standardization-6241, modelsDevSync-extended, xiaomi-mimo-provider, reasoning-token-buffer-6274, t31-t33-t34-t38-model-specs, combo-thinking-disabled-fable5-3554, minimax-m3-maxtokens, kimi-k2.7-code-registration, openai-gpt56-catalog, model-capabilities-mistral-vision-sync-4073, modelsdev-specialty-keying-8017, plus the modelsDevSync integration suite — 96 tests total across 19 files, all passing.

Coverage Notes

• Touches src/lib/modelMetadataRegistry.ts, src/lib/modelsDevSync.ts, src/lib/modelCapabilities.ts, src/shared/constants/modelSpecs.ts. Every changed function is exercised end-to-end by the new regression tests plus the pre-existing suites listed above; no coverage regression expected — pure memoization/indexing added around already-covered logic, no new branches beyond cache-hit/cache-miss paths the tests exercise.

Reviewer Notes

• getModelsDevPricing()'s only other caller (src/app/api/settings/models-dev/route.ts) is read-only, so memoizing the returned reference is safe (no mutation risk).
• findInsensitive()'s WeakMap index is invalidated implicitly: getModelsDevPricing() returns a brand-new object on real cache misses, so the old object (and its index entry) becomes unreachable and garbage-collected.
• getSyncedCapability()'s self-warm reuses getSyncedCapabilities()'s existing write-path invalidation (saveModelsDevCapabilities/clearModelsDevCapabilities both already set cachedCapabilitiesLoadedAll), so no new invalidation logic was needed.
• getCanonicalModelSpecId()/getStaticSpecCanonicalModelId()/reverseModelsDevProviders()/resolveModelAlias() index static, module-level constants (MODEL_SPECS, MODELS_DEV_PROVIDER_MAP) that are never mutated at runtime, so none of their caches need any invalidation at all.
• resolveModelAlias() deliberately does NOT reuse the case-insensitive aliasCi index built for fixes 4/5 — it has always been case-sensitive (Array.includes()), and reusing the ci index would silently broaden which inputs resolve. A dedicated case-sensitive aliasExact map was added to getModelSpecIndex() instead.
• reverseModelsDevProviders()'s return type changed from string[] to readonly string[] (its single call site only spreads the result into a Set, never mutates it) — the result is now Object.freeze()d since it is shared across calls via the cache instead of freshly allocated each time.

[maxmad64bis]

Runtime validation

Built this branch (webpack fallback — Turbopack's native memory isn't bounded by --max-old-space-size on this Next.js version, see #6409) and deployed it to a real running instance, replacing the previous local backport of #8801.

Measured on the same instance, same method, immediately before/after the deploy:

	Cold GET /v1/models (fresh restart, no snapshot)	Warm (2nd call)
Before (previous build)	37.18s	—
After (this branch)	3.29s	36ms

Response validated as a real, complete catalog (6079 models), not a fast error. PROC-0 sanity check green (404/405 on bogus ids), clean boot, no silent exceptions in the application log.

The real-world gain (~11x) is smaller than the synthetic sweep extrapolation in the PR description (~37-54x) — expected, since the synthetic benchmark only exercised the static registry lookup path in a tight loop, while the real endpoint also does live provider-connection resolution and full response formatting that this PR doesn't touch. Still a decisive improvement over the original 41-54s freeze.

[maxmad64bis]

Two days running with this patch and zero freeze, no side effect that I can tell.

[diegosouzapw]

Merged via local merge-train on 192.168.0.113 (32 cores) @ train tip 1efacead211f446aac3146c0599eca1ca190ac4c — log /srv/omniroute-train/.claude/worktrees/train-20260804-130726-suite.log.

Green: typecheck:core, check-complexity, check-cognitive-complexity, check-changelog-integrity, test:vitest.

test:unit: 15 failures, and every one of them reproduces identically on the clean release tip with zero PRs boarded (merge-gates §3) — no new failure was introduced by this train. Thirteen are the bare-model routing assertions left stale by #9275, which intentionally moved bare gpt-5.5/gpt-5.6-sol to codex without updating the tests that encoded the old destination; two are the trailing-period message change covered by #9392. A fix for those is in flight.

For reference, check-file-size is also a pre-existing base-red on the bare tip (src/sse/handlers/chat.ts 1846>1845, open-sse/executors/base.ts 1623>1578 — identical numbers with no PRs boarded): baseline drift for the release captain, not introduced here.