fix: bill storage cost in all modes and show it by steebchen · Pull Request #3247 · theopenco/llmgateway

steebchen · 2026-07-26T18:24:18Z

Problem

Data retention storage cost (log.dataStorageCost, $0.01/M total tokens when retention is on) was recorded on every log and aggregated into hourly stats, but was almost never actually billed or shown:

Billing: the worker's credit deduction only deducted storage for api-keys mode logs — despite a comment claiming credits mode includes it, log.cost (= totalCost in costs.ts) never contains storage. Credits-mode orgs (including DevPass/chat plan pools), wallet-backed end-user traffic, and logs with zero inference cost (e.g. unbilled refusals, whose storage cost is intentionally preserved by zeroInferenceCosts) never paid storage at all. Example from admin: a DevPass org with retention on ran ~519M tokens/day ≈ $5.19/day of storage that was never deducted.
Admin dashboard: the org/project usage metrics endpoints didn't select dataStorageCost from the hourly stats, so the tiles summed exactly to inference cost with storage invisible.
User dashboard: "Total Spend" excluded storage (it only appeared in a subtitle fragment and in the usage cost-breakdown chart).

Changes

apps/worker/src/worker.ts: deduct storage cost from org credits for every log that has one, independent of usedMode and of whether inference cost is nonzero. Inference cost handling is unchanged (credits mode deducts full cost; api-keys/BYOK pays the provider directly). Storage is never counted as premium usage.
apps/api/src/routes/admin.ts: org and project metrics endpoints now aggregate and return dataStorageCost.
ee/admin: new "Storage Cost" tile on org and project usage metrics; "Total Cost" subtitle clarifies it excludes storage.
apps/ui: dashboard "Total Spend" (value, delta, sparkline, avg/request) now includes storage; the storage breakdown subtitle is unchanged.
Tests: 3 new worker specs covering credits-mode storage on top of inference, api-keys storage-only, and zero-inference-cost storage billing.

Notes

Wallet-backed (LLM SDK end-user) traffic now bills storage to the developer's org (which owns the retention setting); the wallet debit itself is unchanged (inference only).
pnpm vitest run apps/worker/src/log-processing.spec.ts: 15/15 pass. Failures in sync-models.spec.ts/stats-calculator.spec.ts are pre-existing shared-test-DB flakes unrelated to this diff.
pnpm build and pnpm format pass.

Screenshots

UI dashboard — Total Spend now includes storage, with the storage amount broken out in the subtitle:

Admin dashboard — new Storage Cost tile on the org usage metrics:

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- Added storage cost metrics to organization and project usage dashboards.
- Added dedicated “Storage Cost” cards showing data retention charges in USD.
- Updated total spend trends, averages, and KPIs to include storage costs.
Bug Fixes
- Corrected billing so data storage costs are charged consistently across usage modes, including usage with no inference cost.
- Improved storage cost reporting for more accurate usage totals.

chatgpt-codex-connector · 2026-07-26T18:24:22Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

coderabbitai · 2026-07-26T18:24:43Z

Walkthrough

Data storage costs are deducted independently during log processing, included in organization and project metrics responses, incorporated into dashboard spend calculations, and displayed as separate storage cost metrics in the admin UI.

Changes

Data storage cost accounting and metrics

Layer / File(s)	Summary
Independent storage cost deductions `apps/worker/src/worker.ts`, `apps/worker/src/log-processing.spec.ts`	Storage costs are deducted independently of inference mode. Credits-mode inference costs remain separately handled. Tests cover credits, API keys, and zero-inference-cost cases.
Organization and project metrics aggregation `apps/api/src/routes/admin.ts`	Organization and project metrics schemas, SQL totals, accumulators, and JSON responses now include `dataStorageCost`.
Dashboard headline spend calculations `apps/ui/src/components/dashboard/dashboard-client.tsx`	Headline spend, average request cost, previous-period comparisons, trend values, and the “Total Spend” card include storage costs.
Admin storage cost metric cards `ee/admin/src/app/organizations/[orgId]/org-metrics.tsx`, `ee/admin/src/app/organizations/[orgId]/projects/[projectId]/project-metrics.tsx`	Organization and project metrics pages show storage cost separately from metered usage cost.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

theopenco/llmgateway#2658: Both PRs modify batchProcessLogs and its tests, but address different billing logic.
theopenco/llmgateway#3364: Both PRs extend admin metrics with aggregated cost data, but add different metrics and logic.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main changes: billing storage costs in all modes and displaying those costs.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch verify-gemini-flash-lite-costs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

## Summary Marketing and docs copy claimed BYOK (bring-your-own-keys) usage is entirely free. That's still true for routing — 0% platform fee — but when an organization enables **full data retention**, storage is billed at **$0.01 per 1M tokens** from LLM Gateway credits in *all* modes, including BYOK (see theopenco#3247, which made that billing consistent). This PR adds the caveat everywhere the free-BYOK claim appears and links the [storage pricing docs](https://docs.llmgateway.io/features/data-retention#storage-pricing). ## Changes - **`packages/shared`**: add `MARKETING_STATS.dataStoragePrice` ("$0.01 per 1M tokens") next to `platformFee` as the single source of truth for surfaces that already consume `MARKETING_STATS`. - **Pricing page** (`apps/ui`): - `pricing-faq.tsx` — the "Is there a fee when I bring my own API keys?" answer now explains the optional storage charge and links "See storage pricing" to the docs section (answer is reused verbatim for FAQPage JSON-LD). - `pricing-table.tsx` — the Data Retention row gets a description ("Metadata is free; full payloads are $0.01/1M tokens") and a "See storage pricing →" link. - **Landing FAQ** (`landing/faq.tsx`) — the "How much does it cost?" BYOK bullet (both the JSON-LD string and the rendered accordion, kept in sync) now mentions the charge, with the rendered version linking the docs section. - **Comparison pages** — pricing FAQ answers on `/compare/open-router`, `/compare/portkey`, `/compare/litellm`, `/compare/vercel-ai-gateway`, `/compare/aws-bedrock`, `/compare/azure-ai-foundry`, and the `/open-source` FAQ. - **DevPass** (`apps/code/Faq.tsx`) — the pay-as-you-go answer gets the same parenthetical. - **Docs** (`learn/provider-keys.mdx`) — the "without additional gateway fees" intro now notes the optional storage charge and links `/features/data-retention#storage-pricing`. ## Deliberately unchanged - Headline one-liners, meta descriptions, hero badges, and referral copy saying BYOK has "0% / zero platform fees" — that claim remains accurate (storage is a separate, opt-in charge); the nuance lives in FAQ/detail copy where users check fees. - Comparison-table platform-fee rows ("5% or 0% (BYOK)") for the same reason. - Dated blog posts. ## Testing - `pnpm build` — 17/17 tasks pass. - `pnpm format` — clean. https://claude.ai/code/session_0196L4fULdokeKNu7stvbLWm  ## Summary by CodeRabbit * **Documentation** * Updated pricing FAQs and provider-key guidance to clarify that optional full data retention costs **$0.01 per 1M tokens**. * Added storage-pricing links and clearer explanations of retention charges. * **Pricing** * Displayed the data-storage price consistently across pricing, comparison, and retention sections. * Clarified that charges may apply to BYOK and managed usage when full data retention is enabled.

Data retention storage cost was recorded on every log but only ever deducted for api-keys mode logs with a nonzero inference cost. Credits mode (including dev/chat plan pools), wallet-backed traffic and zero-inference-cost logs (unbilled refusals) never paid it. Deduct storage for every log with a storage cost, independent of mode and inference cost. Also surface it: the admin org/project usage metrics now return and display a Storage Cost tile, and the dashboard's Total Spend headline includes storage (with the existing breakdown subtitle). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

coderabbitai · 2026-08-09T11:15:55Z

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

Copilot AI review requested due to automatic review settings July 26, 2026 18:24

Copilot AI reviewed Jul 26, 2026

steebchen force-pushed the verify-gemini-flash-lite-costs branch from cd42cd1 to f68523e Compare July 28, 2026 11:01

smakosh mentioned this pull request Aug 1, 2026

fix: disclose data storage cost in BYOK copy #3352

Merged

This was referenced Aug 1, 2026

fix: keep plan usage off real credits (PAYG=default only) #2658

Closed

feat: split BYOK vs credits in usage reporting #3400

Merged

steebchen force-pushed the verify-gemini-flash-lite-costs branch from f68523e to 2e01ff5 Compare August 7, 2026 20:31

steebchen force-pushed the verify-gemini-flash-lite-costs branch from 2e01ff5 to 94b9c8a Compare August 9, 2026 11:15

steebchen added this pull request to the merge queue Aug 9, 2026

Merged via the queue into main with commit 79b2cd3 Aug 9, 2026
13 checks passed

steebchen deleted the verify-gemini-flash-lite-costs branch August 9, 2026 11:53

coderabbitai Bot mentioned this pull request Aug 11, 2026

fix(admin): sum global stats costs in numeric #3547

Merged

Navigation Menu

fix: bill storage cost in all modes and show it - #3247

fix: bill storage cost in all modes and show it#3247
steebchen merged 1 commit into
mainfrom
verify-gemini-flash-lite-costs

steebchen commented Jul 26, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot commented Jul 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

coderabbitai Bot commented Jul 26, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot commented Aug 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

steebchen commented Jul 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

Notes

Screenshots

Summary by CodeRabbit

Uh oh!

chatgpt-codex-connector Bot commented Jul 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot commented Jul 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot commented Aug 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

steebchen commented Jul 26, 2026 •

edited

Loading

coderabbitai Bot commented Jul 26, 2026 •

edited

Loading