Skip to content

perf(ai-gateway): read provider index from replica - #4879

Merged
chrarnoldus merged 1 commit into
mainfrom
wave-cast
Jul 30, 2026
Merged

perf(ai-gateway): read provider index from replica#4879
chrarnoldus merged 1 commit into
mainfrom
wave-cast

Conversation

@chrarnoldus

Copy link
Copy Markdown
Contributor

Summary

  • read the cached models-by-provider snapshot from readDb instead of the primary
  • keep strongly consistent gateway reads, including balances, rate limits, BYOK credentials, and experiment routing, on the writer

Rationale

The provider index already has a 30-second in-process cache and has no read-after-write dependency, so it can tolerate normal replica lag while reducing primary traffic before OpenRouter requests.

Validation

  • pnpm exec oxfmt apps/web/src/lib/ai-gateway/providers/openrouter/models-by-provider-index.server.ts
  • pnpm exec oxlint --config .oxlintrc.json apps/web/src/lib/ai-gateway/providers/openrouter/models-by-provider-index.server.ts
  • pnpm --filter web test --runInBand src/app/api/organizations/[id]/defaults/route.test.ts src/routers/model-preferences-router.test.ts src/routers/organizations/organization-settings-router.test.ts
  • pnpm --filter web typecheck
  • git diff --check

@chrarnoldus chrarnoldus self-assigned this Jul 30, 2026
@kilo-code-bot

kilo-code-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Single-line data-source swap (dbreadDb) for a cache-backed, read-after-write-independent query; no issues found in the changed lines.

Files Reviewed (1 files)
  • apps/web/src/lib/ai-gateway/providers/openrouter/models-by-provider-index.server.ts

Reviewed by claude-sonnet-5 · Input: 20 · Output: 3.6K · Cached: 457.6K

Review guidance: REVIEW.md from base branch main

@chrarnoldus
chrarnoldus merged commit 1911150 into main Jul 30, 2026
16 checks passed
@chrarnoldus
chrarnoldus deleted the wave-cast branch July 30, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants