feat(db): per-request cost ledger + per-key tpm/rpm/monthly quota (RIC-741) by luyuehm · Pull Request #13610 · diegosouzapw/OmniRoute

luyuehm · 2026-09-13T21:55:48Z

Summary

Adds M3 cost transparency + team autonomy (RIC-741) to the gateway:

Per-request cost ledger (request_cost_ledger): one row per completed call with provider/model/token/unit-price/amount breakdown, written from the existing recordCost paths (non-streaming success, guardrail-blocked, and streaming completion).
Per-key quota (api_key_quota_limits + api_key_quota_counters): KISS counter+threshold for tpm (tokens/minute) and rpm (requests/minute) via 2-bucket sliding window; monthly USD cap reads the ledger's current-calendar-month SUM (one source of truth for money).
Domain gate checkKeyQuota / recordKeyQuotaUsage (fail-open B16/B29) wired into enforceApiKeyPolicy pre-request and the usage-record post-hooks (non-streaming + streaming).
Config surface /api/usage/key-quota route + setKeyQuotaSchema for per-key tpm/rpm/monthly settings (0/null = unlimited).

KISS: no ML/计费引擎, no multi-tenant consistency — counters + thresholds, one config row per key. Cost data stays local-only (SQLite), never leaves the domain.

Changes

File	What
`src/lib/db/migrations/177_request_cost_ledger_and_key_quota.sql`	3 tables: ledger + quota limits + quota counters
`src/lib/db/costLedger.ts`	Ledger CRUD + month aggregate
`src/lib/db/keyQuota.ts`	Quota limits/counters + status (2-bucket sliding window)
`src/domain/keyQuota.ts`	`checkKeyQuota` (fail-open) + `recordKeyQuotaUsage`
`src/domain/costRules.ts`	`recordCost` accepts optional ledger `details`
`src/lib/usage/costLedgerRecorder.ts`	Resolve unit prices + append ledger row
`open-sse/handlers/chatCore.ts`	Pass ledger details at non-streaming + guardrail cost sites
`open-sse/handlers/chatCore/streamingCost.ts`	Pass ledger details on stream completion
`open-sse/handlers/chatCore/{non,}StreamingUsageStats.ts`	Advance key-quota tpm/rpm counters
`src/shared/utils/apiKeyPolicy.ts`	`validateKeyQuota` pre-request gate
`src/shared/validation/schemas/keys.ts`	`setKeyQuotaSchema`
`src/app/api/usage/key-quota/route.ts`	GET/POST/DELETE quota config
`tests/unit/cost-ledger-key-quota.test.ts`	12 tests: ledger traceability + tpm/rpm/monthly break

Verification

node --import tsx/esm --test tests/unit/cost-ledger-key-quota.test.ts → 12/12 pass
node --import tsx/esm --test tests/unit/domain-cost-rules.test.ts → 6/6 pass (recordCost backward compat)
Migration 177 applies cleanly on a fresh DB; all 3 tables created.

⚠️ base-red inherited: #12732

…C-741) Add M3 cost transparency + team autonomy: - request_cost_ledger: one row per completed call with provider/model/token/ unit-price/amount breakdown, written from the existing recordCost paths. - api_key_quota_limits + api_key_quota_counters: KISS counter+threshold quota for tpm (tokens/minute) and rpm (requests/minute) via 2-bucket sliding window; monthly USD cap reads from the ledger month SUM. - checkKeyQuota/recordKeyQuotaUsage domain gate (fail-open B16/B29), wired into enforceApiKeyPolicy pre-request and the usage-record post hooks. - /api/usage/key-quota route + setKeyQuotaSchema for per-key config. - Migration 177; tests cover ledger traceability and tpm/rpm/monthly break.

#13248) Merged after renumbering. `176_provider_connection_synced_models_at.sql` collided with `176_xp_action_counts.sql` (#12651), which made the migration runner abort on every DB open. Renamed to **177**; the doc count moves 173 → 174 across README.md, AGENTS.md, llm.txt and the i18n mirrors (operator-approved, 206 numeric substitutions and nothing else). - `check:migration-numbering`: OK, 174 migrations, no duplicates - `check:docs-counts` migrations: ✓ - 84/84 across the NVIDIA suite plus the seven DB-touching suites the collision had taken down - ESLint and `typecheck:core`: exit 0 Heads-up for whoever lands next: **177 is claimed by eight other open PRs** (#13610, #13602, #13580, #13554, #13405, #13331, #13177, #13116) and 176 by #13373 and #13102. With this merged, all of them need to renumber at merge time — `check:migration-numbering` forbids new gaps, so the next free number is always the only valid one. ⚠️ base-red inherited: #12732

diegosouzapw · 2026-09-15T11:59:35Z

Solid, well-scoped addition — ran cost-ledger-key-quota.test.ts (12/12) and
domain-cost-rules.test.ts (6/6, confirms the recordCost signature change stays backward
compatible) locally, both clean. The fail-open quota gate and the single-source-of-truth monthly
cap (reading the ledger's own SUM instead of a separate counter) are the right calls for a KISS
implementation. One mechanical blocker: your new migration is numbered 177, but 177 and 178
are both already taken on the current release tip (merged since you branched) — needs a bump to
179 (whatever's free at merge time). Also noticed the SQL file's own header comment says
"Migration 134" — worth fixing to match the real number while you're renumbering. Nice
complement to your other self-host PRs (#13611, #13639).

177 and 178 already landed on release/v3.8.51 by the time this branch was analyzed (177_provider_connection_synced_models_at.sql, 178_memory_fts_au_conditional.sql); 179 also landed since. Renumber to the next free slot and fix the file's own internal comment to match. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

…ions ESLint no-restricted-syntax bars new local toNumber definitions in favor of @/shared/utils/numeric (diegosouzapw#7879, DRY extraction) — replace the 3 near-identical local copies in costLedger.ts, keyQuota.ts and costLedgerRecorder.ts. Also wires keyQuota.ts's getKeyQuotaStatus to reuse the already-defined toIsoWindowStart helper instead of duplicating the window-start math inline, which fixes the unused-var lint error on that function. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

The recordCost() wiring for the per-request cost ledger added ~24 lines to chatCore.ts, an already-frozen file (file-size-baseline.json caps it at 6146 lines). Extract the shared provider/model/tokens/serviceTier/requestId breakdown into buildCostCtx() and the apiKeyInfo?.id && estimatedCost > 0 guard into recordChatCallCost() (both in src/domain/costRules.ts), and the streaming ledger-details object into buildStreamLedgerDetails() in streamingCost.ts. No behavior change: same 18 cost-ledger/domain-cost-rules tests pass unmodified; net effect is chatCore.ts now at 6143 lines (3 under the frozen ceiling) instead of 6165. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

The tip advanced past 180 (180_memory_fts_au_conditional_memory_id.sql landed) between this branch's pickup and the merge, colliding with the already-renumbered 180_request_cost_ledger_and_key_quota.sql — bump to the next free slot, 181, and fix the file's internal comment to match. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

…uota migration check:docs-all (stale-migrations, STRICT) failed because README.md, AGENTS.md and llm.txt (plus its 65 docs/i18n/*/llm.txt mirrors, which must be exact body copies of the root file) still said "177 migrations" after this PR added migration 181_request_cost_ledger_and_key_quota.sql, bringing the real count to 178. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

…ken) Renamed 180_api_key_preferred_connections.sql to 184_api_key_preferred_connections.sql: the release tip landed 180_memory_fts_au_conditional_memory_id.sql after this PR's previous renumbering pass. Slot 184 is the owner-assigned number for this PR among the 7 PRs that collided on the 180 slot (diegosouzapw#13610=181, diegosouzapw#12962=182, diegosouzapw#12967=183, diegosouzapw#13102=184, diegosouzapw#13222=185, diegosouzapw#13373=186, diegosouzapw#13554=187). Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

Six open PRs claimed migration slot 180 after diegosouzapw#13331 landed it on the release tip; the owner assigned diegosouzapw#13373 slot 186 in the sequence (diegosouzapw#13610=181, diegosouzapw#12962=182, diegosouzapw#12967=183, diegosouzapw#13102=184, diegosouzapw#13222=185, diegosouzapw#13373=186, diegosouzapw#13554=187). Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

diegosouzapw · 2026-09-22T10:23:01Z

Thanks @luyuehm — merging via the release merge-train. Validated in local merge-train (mt-train10c) on the devbox @ train tip 4d841aa1c740bbaa03868dc0a403c62099a99a42 with the 72 sibling PRs of this batch: typecheck:core, file-size, complexity, cognitive-complexity, changelog-integrity green; changed-area node:test 831/831 (0 failing) and vitest 480/482 — the two reds are autoCombo/provider-family-combos.test.ts timing out at 20s, which reproduces on the PURE release tip under the full vitest suite (and is already tracked by the Release-Green issue #13866), so it is inherited, not this batch's. Merged --admin per merge-gates §3/§4/§7.

…ouzapw#12849) (diegosouzapw#13248) Merged after renumbering. `176_provider_connection_synced_models_at.sql` collided with `176_xp_action_counts.sql` (diegosouzapw#12651), which made the migration runner abort on every DB open. Renamed to **177**; the doc count moves 173 → 174 across README.md, AGENTS.md, llm.txt and the i18n mirrors (operator-approved, 206 numeric substitutions and nothing else). - `check:migration-numbering`: OK, 174 migrations, no duplicates - `check:docs-counts` migrations: ✓ - 84/84 across the NVIDIA suite plus the seven DB-touching suites the collision had taken down - ESLint and `typecheck:core`: exit 0 Heads-up for whoever lands next: **177 is claimed by eight other open PRs** (diegosouzapw#13610, diegosouzapw#13602, diegosouzapw#13580, diegosouzapw#13554, diegosouzapw#13405, diegosouzapw#13331, diegosouzapw#13177, diegosouzapw#13116) and 176 by diegosouzapw#13373 and diegosouzapw#13102. With this merged, all of them need to renumber at merge time — `check:migration-numbering` forbids new gaps, so the next free number is always the only valid one. ⚠️ base-red inherited: diegosouzapw#12732

luyuehm requested a review from diegosouzapw as a code owner September 13, 2026 21:55

This was referenced Sep 15, 2026

feat(routing): deterministic routing strategies for self-hosted entry (RIC-740) #13611

Merged

feat(docker): self-host compose + 5-min deploy doc (RIC-739) #13639

Merged

Ant Rich and others added 5 commits September 16, 2026 02:41

xiaoyaner0201 mentioned this pull request Sep 16, 2026

feat(backend): policy-driven bulk API-key allowance resets when upstream quota renews #13853

Open

Merge remote-tracking branch 'origin/release/v3.8.51' into HEAD

a30fef4

diegosouzapw merged commit 27b4cab into diegosouzapw:release/v3.8.51 Sep 22, 2026
13 of 16 checks passed

This was referenced Sep 22, 2026

ci(merge-train): run the ci.yml:lint family on the combined tree #14489

Merged

fix(sse): rotation attribution diagnostics (skipped accounts, serving id, request correlation) #14223

Merged

This was referenced Sep 23, 2026

fix(backend): one unpriced model blocks a usage-limited API key on every model, with a misleading quota message #14604

Open

feat(api): configurable unpriced-usage budget policy + admin notice (#14604) #14608

Open

Navigation Menu

Uh oh!

feat(db): per-request cost ledger + per-key tpm/rpm/monthly quota (RIC-741) - #13610

feat(db): per-request cost ledger + per-key tpm/rpm/monthly quota (RIC-741)#13610
diegosouzapw merged 7 commits into
diegosouzapw:release/v3.8.51from
luyuehm:RIC-697-cost-quota

luyuehm commented Sep 13, 2026

Uh oh!

diegosouzapw commented Sep 15, 2026

Uh oh!

diegosouzapw commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

luyuehm commented Sep 13, 2026

Summary

Changes

Verification

Uh oh!

diegosouzapw commented Sep 15, 2026

Uh oh!

diegosouzapw commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants