Skip to content

fix(db): stop backoff-reset from busting the model catalog cache (#13389) - #13783

Merged
diegosouzapw merged 2 commits into
release/v3.8.51from
fix/13389-v1-models-intermittent-75-120s
Sep 16, 2026
Merged

diegosouzapw merged 2 commits into
release/v3.8.51from
fix/13389-v1-models-intermittent-75-120s

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Refs #13389

Not covered here (follow-up, Refs #13389):

  • The should-fix half of the plan — bounding CATALOG_BUILD_TIMEOUT_MS for a genuinely
    slow-but-not-hung synchronous builder (currently only a never-resolving/hung builder is
    bounded). This is a materially larger change (worker-thread offload or finer-grained
    SQLite yield points) and is tracked as a separate follow-up.

Root cause (short)

resetConnectionBackoff() (src/lib/db/providers.ts) fires automatically whenever a
previously-cooled-down connection is auto-recovered during normal request routing
(src/sse/services/auth.ts). It called invalidateDbCache("connections"), which
unconditionally bumps modelCatalogCacheVersion — busting the entire /v1/models
response cache as a side effect, even though the catalog builder
(src/app/api/v1/models/catalog.ts) never reads backoff/error/cooldown state at all
(backoffLevel, testStatus, rateLimitedUntil, lastError*, errorCode) — only
structural fields such as excludedModels or enabled/disabled.

On a deployment routing many providers, this invalidated the catalog cache far more often
than its 60s TTL / 30s stale-while-revalidate window intends, purely as a side effect of
unrelated chat traffic, forcing frequent expensive cold rebuilds — the dominant trigger
behind the reported 75-120s/502 spread (independent of the 8s cold-build timeout added in
#12628, which itself only bounds a hung build, not a slow one — see the follow-up above).

Fix

Added a skipModelCatalog option to invalidateDbCache() (src/lib/db/readCache.ts).
resetConnectionBackoff now passes it, so it still busts the connections read cache
(needed for accurate routing) but no longer bumps modelCatalogCacheVersion.

Audited every other invalidateDbCache("connections", …) call site in providers.ts
(create, dedupe-merge update, general update) — all are structural connection writes that
genuinely change catalog-relevant fields, so they keep the default full invalidation.

Regression test (path + RED output excerpt on unfixed code + GREEN excerpt)

tests/unit/issue-13389-catalog-cache-backoff-reset.test.ts

RED (unfixed code):

✖ #13389 resetConnectionBackoff does NOT bust the model catalog cache (4088.727199ms)
  AssertionError [ERR_ASSERTION]: a pure backoff/error-state reset must not invalidate the model catalog cache — the catalog builder never reads backoffLevel/testStatus/rateLimitedUntil
  3 !== 2

GREEN (fixed code):

✔ #13389 resetConnectionBackoff does NOT bust the model catalog cache (5601.60286ms)
✔ #13389 a structural connection write still busts the model catalog cache (65.784542ms)
ℹ tests 2
ℹ pass 2
ℹ fail 0

Gates run

  • npx eslint --suppressions-location config/quality/eslint-suppressions.json src/lib/db/providers.ts src/lib/db/readCache.ts tests/unit/issue-13389-catalog-cache-backoff-reset.test.ts → clean, exit 0
  • npm run typecheck:core → clean, exit 0
  • node scripts/check/check-file-size.mjs → clean (trimmed the added comment to keep providers.ts at 1198 lines, under the frozen 1200-line cap)
  • node scripts/check/check-complexity.mjs → no violations for the touched files
  • node scripts/check/check-cognitive-complexity.mjs → no violations for the touched files
  • node scripts/check/check-test-discovery.mjs → OK, new test file discovered
  • Existing tests in the touched area (tests/unit/reset-connection-backoff.test.ts, tests/unit/db-read-cache.test.ts) → 14/14 pass, unchanged

Existing tests aligned

None needed changing — invalidateDbCache's new opts parameter is additive/optional, and no
existing test called resetConnectionBackoff while asserting on modelCatalogCacheVersion.

diegosouzapw and others added 2 commits September 15, 2026 15:51
)

resetConnectionBackoff() ran through invalidateDbCache("connections"),
which unconditionally bumps modelCatalogCacheVersion and clears the
entire /v1/models response cache — even though the catalog builder
(src/app/api/v1/models/catalog.ts) never reads backoff/error/cooldown
state, only structural fields (excludedModels, enabled/disabled, etc).
Since resetConnectionBackoff fires automatically on every routine
connection auto-recovery during request routing (src/sse/services/auth.ts),
this was busting the catalog cache far more often than its 60s TTL / 30s
stale-while-revalidate window intends, forcing frequent expensive cold
rebuilds — the dominant trigger behind the reported 75-120s/502 spread.

Adds a `skipModelCatalog` option to invalidateDbCache and passes it only
from resetConnectionBackoff; every other "connections" call site in
providers.ts (create/update/dedupe-merge) is structural and keeps the
default full invalidation.

Regression test: tests/unit/issue-13389-catalog-cache-backoff-reset.test.ts

Does not cover the should-fix half of #13389 (the CATALOG_BUILD_TIMEOUT_MS
abort not bounding a genuinely slow-but-not-hung synchronous builder) —
tracked as a separate follow-up.
@diegosouzapw
diegosouzapw merged commit 05b44fa into release/v3.8.51 Sep 16, 2026
18 of 21 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…gosouzapw#13389) (diegosouzapw#13783)

Merged in the 2026-09-16 sweep of the maintainer's own open PRs, at the owner's explicit instruction. No push was made to the PR branch: the merge took the head as the owning session left it (verified OPEN, non-draft and MERGEABLE against the release tip immediately before merging).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant