coderouter: serve stored usage in account status

[lawrencecchen]

Measured before: GET /api/coderouter/accounts took 1.4-3.9 s, of which provider (one KMS decrypt plus one chatgpt.com/backend-api/wham/usage read per active Codex account, 80 for cmux) was 1.2-2.1 s.

Readings are now stored on coderouter_accounts (usage, usage_error, usage_fetched_at, usage_refresh_claimed_at):

• under 30 s old: served as is, no provider call;
• 30 s to 10 min: served at once, then refreshed after the response (after()); a claim column means one instance refreshes each account, not every concurrent viewer;
• over 10 min or missing: read before answering, so an occasional cr never shows a long-stale quota.

usageAgeSeconds now reports the oldest reading served and cacheMaxAgeSeconds is 30; the CLI already prints both ("Usage cached Ns ago"). Routing does not read these numbers: it uses cooldown_until, set by the proxy on 429 and still set by a refresh that sees limit_reached.

Trade-off: cr can show quota up to 10 min old (labelled with its age); cr --watch stays within about 60 s.

Schema-dependent: the migration 20260923120000_coderouter_account_usage_cache (four nullable ADD COLUMNs, no backfill, no lock beyond the brief ALTER) must be rehearsed on the PlanetScale staging branch and applied to main before this merges, because merging deploys code that selects the new columns.

Third of three coderouter latency PRs (#13976, #13977).

Tests: new tests/coderouter-account-usage.test.ts (fresh, stale plus claim, missing and too-old, stored error and skipped accounts, failed store); bun test tests/coderouter-*.test.ts (428 pass); bun run typecheck; eslint; verify-migration-preflight.sh --schema-only (drizzle-kit check passes). The migration SQL was not run against a real Postgres.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

---

Summary by cubic

Caches Codex quota reads for coderouter status so GET /api/coderouter/accounts no longer waits on one KMS decrypt plus one ChatGPT usage call per active Codex account (up to ~80 for cmux); status dropped from 1.4–3.9s to well under a second.

Readings are now stored on coderouter_accounts and served by age:

• Under 30s: served as is, no provider call.
• 30s to 10min: served immediately, then refreshed after the response; a claim column ensures only one instance refreshes each account, not every concurrent viewer.
• Over 10min or missing: read before answering, so an occasional cr never shows a long-stale quota.

Routing still uses cooldown_until, which refreshes keep setting on limit_reached; the cache only feeds status display. usageAgeSeconds reports the oldest reading served and cacheMaxAgeSeconds is 30, so cr can show quota up to 10min old but cr --watch stays within about 60s.

Migration: the 20260923120000_coderouter_account_usage_cache migration (four nullable ADD COLUMNs, no backfill) must be applied to PlanetScale main before this merges, since the merged code selects the new columns.

^{Written for commit 0d4d682. Summary will update on new commits.}

Summary by CodeRabbit

• New Features
  • Account usage and quota readings are now cached, so status checks can show recent results without fetching each account’s usage every time.
  • Older readings are refreshed in the background when possible; missing or very old readings are fetched before the status is returned.
  • Status results include when the usage data was last fetched, and display provider errors when available.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: acfdbc5c-dfd9-4177-af34-a42af8af6f44

📥 Commits

Reviewing files that changed from the base of the PR and between 638aaa7 and 0d4d682.

📒 Files selected for processing (5)

• web/db/migrations/20260923120000_coderouter_account_usage_cache/migration.sql
• web/db/schema.ts
• web/services/coderouter/repository.ts
• web/services/coderouter/usage.ts
• web/tests/coderouter-account-usage.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds persistent account usage caching and refresh coordination. The usage service serves stored readings, refreshes missing or very old readings before responding, and defers refreshes for stale readings.

Changes

Account usage cache

Layer / File(s)	Summary
Usage cache storage web/db/migrations/*, web/db/schema.ts, web/services/coderouter/repository.ts	The account table gains usage data, fetch time, error, and refresh-claim columns. Repository functions read stored readings, claim eligible refreshes, and persist newer readings.
Cached usage serving and refresh web/services/coderouter/usage.ts, web/tests/coderouter-account-usage.test.ts	The service serves stored readings and refreshes missing or very old readings before responding. It defers refreshes for stale readings and includes usage timestamps and cache age in its result. Tests cover cache age, refresh claims, stored errors, inactive and non-Codex accounts, and store failures.

Priority: ⚪ Not assessed

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant accountsWithUsage
  participant deferAfterResponse
  participant claimAccountUsageRefresh
  participant readCodexUsage
  participant CODEX_USAGE_URL
  participant storeAccountUsage
  Caller->>accountsWithUsage: Request account usage
  accountsWithUsage->>deferAfterResponse: Schedule stale account refresh
  deferAfterResponse->>claimAccountUsageRefresh: Claim eligible account IDs
  deferAfterResponse->>readCodexUsage: Read claimed accounts
  readCodexUsage->>CODEX_USAGE_URL: Fetch provider usage
  readCodexUsage-->>deferAfterResponse: Return usage or error
  deferAfterResponse->>storeAccountUsage: Persist reading or error

Loading

Suggested reviewers: teamleaderleo

Merge Risk: ⚪ Minimal · up to 0d4d6

The accounts endpoint now serves stored usage readings, refreshes stale ones in the background, and still uses cooldowns for routing. No blocking issues were found, and the new migration only adds nullable columns in the project's standard format. As the author notes, apply and rehearse the migration on the database before deploying this code.

🚥 Pre-merge checks | ✅ 24 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 23.53% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 4 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (24 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the main change: serving stored coderouter usage data in account status.
Description check	✅ Passed	The description explains the problem, age-based caching behavior, migration dependency, testing performed, and known migration limitation. It does not include the template checklist, review-trigger bl…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input	✅ Passed	PASS: The authoritative PR diff changes only coderouter usage persistence, schema, repository logic, usage caching, and related tests. It introduces no Cloud terminal creation, cmux-tui client, physic…
Cmux Swift Actor Isolation	✅ Passed	The review-scoped diff changes only TypeScript, SQL, and test files. It contains no production Swift changes, so it cannot introduce or worsen Swift 6 actor-isolation issues.
Cmux Swift Blocking Runtime	✅ Passed	The pull request changes only TypeScript and SQL files. It introduces no production Swift files or Swift blocking/runtime synchronization primitives. The Swift-specific check is not applicable.
Cmux Browser Automation Off-Main	✅ Passed	PASS: The pull request changes only web SQL and TypeScript files. It does not modify Sources/TerminalController.swift or ControlCommandExecutionPolicy.swift, and the patch contains no `browser.*…
Cmux Expensive Synchronous Load	✅ Passed	PASS: This check applies only to production Swift changes. The pull-request diff changes five web TypeScript/SQL/test files and contains no Swift, Objective-C, or Objective-C++ paths. Therefore it doe…
Cmux Cache Substitution Correctness	✅ Passed	PASS: The cache substitution is limited to the GET /api/coderouter/accounts status response. It does not replace a read in a persistence, history, undo, or snapshot path. The changed code handles co…
Cmux No Hacky Sleeps	✅ Passed	The diff adds no sleep, setTimeout, setInterval, polling loop, or fixed backoff. The new after(settled) call defers stale-account refresh to Next.js response completion, which is an explicit l…
Cmux Algorithmic Complexity	✅ Passed	The changed production path remains linear in account count. load scans accounts once, uses Map.get for stored readings, and maps the result once. refresh builds one credential map and performs …
Cmux Swift Concurrency	✅ Passed	PASS: The reviewed diff changes only one SQL file and four TypeScript files. It contains no Swift files or Swift concurrency changes, so the cmux Swift concurrency check is not applicable.
Cmux Swift @Concurrent	✅ Passed	The pull request changes only SQL, TypeScript, and test files. It contains no Swift files or Swift concurrency changes, so the @concurrent rule does not apply.
Cmux Swift Package Boundaries	✅ Passed	The pull request changes only TypeScript and SQL files. The authoritative diff contains no Swift, SwiftPM, Xcode project, or workspace files. The Swift package-boundaries check is therefore not applic…
Cmux Swiftpm Lockfiles	✅ Passed	The PR changes only web database, service, and test files. The authoritative diff contains no Package.swift, Package.resolved, .gitignore, workflow, Xcode project, or SwiftPM dependency changes. There…
Cmux Swift Logging	✅ Passed	PASS — the pull request changes only SQL and TypeScript files. It introduces no production Swift changes or logging statements, so the Swift logging rules do not apply.
Cmux User-Facing Error Privacy	✅ Passed	No privacy violation is introduced. The concrete end-user path is cmux coderouter accounts --json, which consumes GET /api/coderouter/accounts. The changed usage flow exposes only the existing gen…
Cmux Full Internationalization	✅ Passed	PASS. The PR changes backend quota caching, database schema, and tests. It does not change web UI copy, locale routing, message files, Swift catalogs, metadata, markdown, or changelog text. The API st…
Cmux Swiftui State Layout	✅ Passed	PASS: The pull request changes only web TypeScript, SQL migration, and test files. The authoritative diff contains no Swift, SwiftUI, AppKit, or related UI files or symbols. The SwiftUI state-layout c…
Cmux Architecture Rethink	✅ Passed	The check is not applicable. The authoritative pull-request diff changes only four TypeScript files, one SQL migration, and one TypeScript test file. It introduces no Swift, SwiftUI, or AppKit code, s…
Cmux Swift Auxiliary Window Close Shortcuts	✅ Passed	PASS. The review-scoped diff contains only web TypeScript, SQL migration, and test files. It contains no Swift, storyboard, or XIB changes. Therefore, the auxiliary-window close-shortcut rule is not a…
Cmux Source Artifacts	✅ Passed	All five changed paths are intentional product or test sources: one database migration, the schema declaration, two coderouter service files, and a coderouter test. No local output, generated logs, sc…
Cmux No Test Or Debug Seam In Production Source	✅ Passed	PASS. The pull request changes only web TypeScript, SQL, and test files. It does not change any Swift file under a production Sources/ path, so the custom check is not applicable.

Full details: Docstring Coverage

Explanation

Docstring coverage is 23.53% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 4 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[teamleaderleo]

Status: its migration now ships on its own in #14073, because schema.ts must not reach production before the columns exist. The order is #14073, then the staging migration, then the production migration, then this PR. No rebase is needed; the identical migration file drops out of this diff once #14073 is on main.

— Ibex g1 🌿

[teamleaderleo]

Review status (Leo's maintainer sweep). No push from me.

CI is green and the branch merges cleanly with main. It also merges cleanly with #13977, which touches the same repository.ts.

Review:

• Correct as far as I can see. The tiers match the description: hasUsage accounts are split into blocking (missing, or older than 10 min), stale (30 s to 10 min, served now and refreshed through after()), and fresh. claimAccountUsageRefresh is a conditional UPDATE ... RETURNING, so only one instance refreshes each account. storeAccountUsage never overwrites a newer reading.
• web/tests/coderouter-account-usage.test.ts covers each tier, the claim window arguments, a stored provider error, inactive and non-Codex accounts, and a failed store.
• Minor: the blocking path (refresh(teamId, blocking) in usage.ts) does not claim. Concurrent first views from several instances will each read the provider once for missing readings. That is acceptable for a cold start.
• The migration folder has no snapshot.json, which matches the other recent migrations (20260917*, 20260921*).

Still blocking, and it needs a human: the description says the 20260923120000_coderouter_account_usage_cache migration must be rehearsed on the PlanetScale staging branch and applied to main before merge, because the deployed code selects the new columns. I can't verify that from here. Whoever merges should confirm it first.

[teamleaderleo]

Merge order status: the migration PR #14073 merged at 02:06Z. cloud-vm-migrate.yml run 35946090138 applied it to staging (success), and its migrate-production job is waiting on the production environment approval. This PR stays unmerged until that job succeeds; otherwise the reader deploys before its columns exist. Code and CI are ready, and it builds cleanly on #13977, which merged as c50cf5ab79.

— Ophelia g1 🍄
Run: run_cmux_main_red_triage_app_host_census_and_pr_review_20260923_07d8d17b

[teamleaderleo]

Migration rollout status:

• Staging: done. cloud-vm-migrate.yml run 35945984665 checked out a3da5bc (the db: add coderouter account usage cache columns ahead of their reader #14073 merge) and reported "staging PlanetScale migration applied".
• Production: waiting for approval. Run 35946090138 passed preflight and re-ran the staging job. Its migrate-production job is held by the cloud-vm-production environment, and only its reviewers can approve it: @lawrencecchen, @austinywang, @azooz2003-bit. The migration adds four nullable columns with no default or backfill, so the ALTER changes only the catalog. An independent review of db: add coderouter account usage cache columns ahead of their reader #14073 approved it.

#13978 must not merge until production has run, because its schema.ts makes every coderouter_accounts read select these columns.

— Ibex g1 🌿