feat: split BYOK vs credits in usage reporting - #3400
Merged
Merged
Conversation
Adds creditsCost/apiKeysCost/creditsRequestCount/apiKeysRequestCount to activity breakdowns, sources, analytics, admin metrics, cost-by-model, timeseries and global stats. Fixes credits-runway and admin unusedCredits to use debited spend (creditsCost + BYOK storage), makes DevPass/ChatPlan margin count credits-mode cost only, and adds a usedMode filter to /logs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
WalkthroughThe PR adds credits-versus-BYOK request and cost metrics across API responses, analytics, admin metrics, billing calculations, and dashboards. It also adds URL-based usage-mode selectors and a ChangesCredits and BYOK API metrics
Usage-mode presentation
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant UsageModeSelector
participant Dashboard
participant API
User->>UsageModeSelector: select total, credits, or BYOK
UsageModeSelector->>Dashboard: update mode URL parameter
Dashboard->>API: request activity and metric data
API-->>Dashboard: blended and mode-split metrics
Dashboard->>Dashboard: normalize and render selected metrics
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR updates LLM Gateway’s reporting stack to clearly separate credits-billed usage from BYOK (“api-keys”) usage across API aggregation endpoints and both the user and admin dashboards, while keeping existing “blended” totals backward compatible.
Changes:
- API: Add per-mode split fields (
creditsCost,apiKeysCost,creditsRequestCount,apiKeysRequestCount) across activity/analytics/admin aggregation endpoints, plus a/logsusedModeserver-side filter. - Fix reporting correctness bugs where “debited spend” must exclude BYOK provider cost (e.g., credits runway and admin unused credits/overage).
- UI + Admin UI: Add a shared All / Credits / BYOK mode selector (persisted via
modeURL param) and normalize rows at the fetch boundary so charts/tables can toggle modes client-side without refetching.
Reviewed changes
Copilot reviewed 43 out of 43 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| ee/admin/src/lib/usage-mode.ts | Adds admin-side usage-mode parsing and helpers for selecting per-mode cost/request values. |
| ee/admin/src/components/usage-mode-selector.tsx | Introduces admin usage-mode selector persisted in mode URL param. |
| ee/admin/src/components/cost-by-model-timeseries-chart.tsx | Switches cost/request series to respect selected usage mode while keeping tokens blended. |
| ee/admin/src/components/cost-by-model-chart.tsx | Adds per-mode totals/rows rendering and selector for cost-by-model view. |
| ee/admin/src/app/page.tsx | Shows total spent plus split spent (credits vs BYOK) in admin overview. |
| ee/admin/src/app/organizations/page.tsx | Displays per-org split totals (credits vs BYOK) alongside blended total. |
| ee/admin/src/app/organizations/[orgId]/projects/[projectId]/project-metrics.tsx | Adds usage-mode selector and per-mode metric cards for project metrics. |
| ee/admin/src/app/organizations/[orgId]/org-metrics.tsx | Adds usage-mode selector and per-mode metric cards for org metrics. |
| ee/admin/src/app/global-stats/client.tsx | Updates global stats subtitle to show credits vs BYOK split when present. |
| ee/admin/src/app/devpass/[orgId]/page.tsx | Clarifies “real provider cost” is credits-mode only. |
| ee/admin/src/app/chat-plans/[orgId]/page.tsx | Clarifies “real provider cost” is credits-mode only. |
| apps/ui/src/types/activity.ts | Extends activity types to include per-mode split fields and storage split. |
| apps/ui/src/lib/usage-mode.ts | Adds shared UI usage-mode model + normalization helpers (applyUsageMode*). |
| apps/ui/src/components/usage/usage-client.tsx | Adds usage-mode selector and all-traffic note for blended-only metrics. |
| apps/ui/src/components/usage/usage-chart.tsx | Uses per-mode request counts for the requests-over-time chart. |
| apps/ui/src/components/usage/model-usage-table.tsx | Applies mode normalization so model rows reflect selected billing view. |
| apps/ui/src/components/usage/model-usage-client.tsx | Adds selector to model usage page header. |
| apps/ui/src/components/usage/cost-breakdown-chart.tsx | Applies mode normalization for breakdown and shows split totals in total mode. |
| apps/ui/src/components/shared/usage-mode-selector.tsx | Introduces reusable UI usage-mode segmented control stored in URL. |
| apps/ui/src/components/enterprise/feature-showcase.tsx | Updates mock analytics rows to include split fields for compatibility. |
| apps/ui/src/components/dashboard/developer-dashboard-client.tsx | Adds selector; narrows cost/requests to selected mode for developer dashboard. |
| apps/ui/src/components/dashboard/dashboard-client.tsx | Adds selector and mode-normalizes daily rows; shows split totals in total mode. |
| apps/ui/src/components/dashboard/activity-chart.tsx | Mode-normalizes /activity response for charts without extra fetches. |
| apps/ui/src/components/api-keys/api-key-stats-client.tsx | Adds selector; adjusts error-rate computation to remain blended while costs/requests can be per-mode. |
| apps/ui/src/components/analytics/chart-helpers.ts | Extends analytics row shapes to include per-mode split fields. |
| apps/ui/src/components/analytics/analytics-client.tsx | Adds selector and mode-normalizes analytics activity rows. |
| apps/ui/src/components/activity/recent-logs.tsx | Adds UI filter for usedMode and passes it to /logs. |
| apps/ui/src/components/activity/agents-view.tsx | Adds selector and mode-normalizes agent/source rows. |
| apps/ui/src/app/dashboard/[orgId]/org/team/team-client.tsx | Adds selector and mode-normalizes per-member usage rows. |
| apps/ui/src/app/dashboard/[orgId]/org/team/[userId]/member-detail-client.tsx | Adds selector and mode-normalizes member detail breakdowns/top lists. |
| apps/ui/src/app/dashboard/[orgId]/org/analytics/org-analytics-client.tsx | Adds selector; mode-normalizes org activity rows and breakdowns. |
| apps/api/src/routes/organization.ts | Fixes credits-runway burn-rate to use debited spend (credits + BYOK storage), not blended cost. |
| apps/api/src/routes/organization.spec.ts | Adds regression test for credits-runway debited-spend calculation. |
| apps/api/src/routes/logs.ts | Adds usedMode query param and SQL filter support for logs listing. |
| apps/api/src/routes/logs.spec.ts | Adds tests validating usedMode filtering and “all” behavior. |
| apps/api/src/routes/analytics.ts | Adds mode-split fields to analytics endpoints and response schemas. |
| apps/api/src/routes/analytics.spec.ts | Adds coverage for mode split across analytics endpoints. |
| apps/api/src/routes/admin.ts | Adds mode-split metrics; fixes unusedCredits/overage to use debited spend; excludes BYOK from “real provider cost” for plans. |
| apps/api/src/routes/admin-metrics-mode-split.spec.ts | Adds admin regression tests for splits and debited-spend computations. |
| apps/api/src/routes/activity.ts | Adds mode-split fields to activity endpoints (daily + breakdowns + sources). |
| apps/api/src/routes/activity.spec.ts | Adds tests covering activity mode split (day totals, breakdowns, sources). |
| apps/api/src/lib/user-usage-breakdown.ts | Extends user usage breakdown helper to include mode split fields. |
| apps/api/src/lib/mode-split.ts | Introduces shared helper for selecting + mapping split fields in rollup queries. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+49
to
+59
| <button | ||
| key={option.value} | ||
| type="button" | ||
| onClick={() => setMode(option.value)} | ||
| title={ | ||
| option.value === "api-keys" | ||
| ? "Usage served by your own provider keys (not billed to credits)" | ||
| : option.value === "credits" | ||
| ? "Usage billed against your credit balance" | ||
| : "All traffic" | ||
| } |
Comment on lines
+47
to
+52
| <Button | ||
| key={option.value} | ||
| variant={mode === option.value ? "default" : "outline"} | ||
| size="sm" | ||
| onClick={() => setMode(option.value)} | ||
| > |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
apps/api/src/routes/analytics.ts (1)
536-582: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftMake top-N model/provider cards mode-aware.
modelRowsare ordered by blendedcost, thentopModelsandtopProvidersare sliced from that blended set. The client only re-sorts the returned rows bycostafter applying the selected mode, so Credits or BYOK top spenders can be hidden if they are not in the blended top-N. Include a mode-aware ordering, or fetch enough candidates before the mode-specificmapModeSplit(...).slice(...)/sort(...).slice(...).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/routes/analytics.ts` around lines 536 - 582, Update the topModels and topProviders selection in the analytics route to rank candidates using the selected mode’s cost rather than only blended r.cost. Apply mapModeSplit-derived mode costs before slicing, or retain enough modelRows/provider entries to perform mode-specific sorting and top-N selection so Credits and BYOK leaders are not excluded.apps/ui/src/components/dashboard/dashboard-client.tsx (1)
305-369: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winUse the blended request count for reliability rates.
applyUsageModeToDailykeepscacheCountanderrorCountblended across all traffic, but it rewritesrequestCountforcreditsandapi-keysviews.dashboard-client.tsxreportscacheHitRate = totalCached / totalRequests, andErrorsReliabilityCardcomputes bothcacheRateanderrorRatefrom the mode-adjustedrequestCount. With mode filtering, these can exceed 100% or otherwise misreport blended metrics. Compute the denominator from the untransformed activity data for these rates.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/components/dashboard/dashboard-client.tsx` around lines 305 - 369, Use the untransformed rawActivityData requestCount total as the denominator for blended reliability metrics, while retaining activityData for mode-specific cost and request totals. Update cacheHitRate and the request-count value passed to ErrorsReliabilityCard so cache and error rates use the raw blended request count, and keep the existing zero-denominator safeguards.apps/ui/src/components/usage/model-usage-table.tsx (1)
134-151: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winSum mode-split fields when merging model entries across days.
The merge loop sums
requestCount,inputTokens,outputTokens,totalTokens, andcostfor repeatedprovider|idkeys, but does not sumcreditsRequestCount,apiKeysRequestCount,creditsCost, orapiKeysCost. For a model that appears across multiple days, the aggregated entry keeps only the first day's values for these four fields instead of the total across all days.This does not affect the current table, because it never reads these fields. If any future consumer of this aggregated map reads them, it gets an incorrect partial value.
🐛 Proposed fix to sum the mode-split fields
data.activity.forEach((day) => { day.modelBreakdown.forEach((rawModel) => { const model = applyUsageMode(rawModel, usageMode); const key = `${model.provider}|${model.id}`; if (modelMap.has(key)) { const existing = modelMap.get(key)!; existing.requestCount += model.requestCount; existing.inputTokens += model.inputTokens; existing.outputTokens += model.outputTokens; existing.totalTokens += model.totalTokens; existing.cost += model.cost; + existing.creditsRequestCount += model.creditsRequestCount; + existing.apiKeysRequestCount += model.apiKeysRequestCount; + existing.creditsCost += model.creditsCost; + existing.apiKeysCost += model.apiKeysCost; } else { modelMap.set(key, { ...model }); } }); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/components/usage/model-usage-table.tsx` around lines 134 - 151, Update the existing-entry merge branch in the modelMap aggregation loop to also add creditsRequestCount, apiKeysRequestCount, creditsCost, and apiKeysCost from each repeated model entry, alongside the existing totals. Keep the current provider|id grouping and other field aggregation unchanged.
🧹 Nitpick comments (1)
apps/api/src/lib/mode-split.ts (1)
51-58: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
mapModeSplitisn't null-safe, so four call sites inanalytics.tsre-implement its logic manually.mapModeSplitrequires a non-nullableModeSplitRow. Wherever the aggregate row can be undefined (a single-row.select()result, or aMap.get()lookup), the code falls back to four repeatedNumber(row?.field ?? 0)conversions instead of reusing the helper.
apps/api/src/lib/mode-split.ts#L51-L58: widenmapModeSplitto acceptModeSplitRow | null | undefinedand default each field to 0 internally, so callers can pass a possibly-undefined row directly.apps/api/src/routes/analytics.ts#L502-L505: replace the fourNumber(summaryRow?.field ?? 0)lines with...mapModeSplit(summaryRow).apps/api/src/routes/analytics.ts#L825-L828: replace the fourNumber(s?.field ?? 0)lines with...mapModeSplit(s).apps/api/src/routes/analytics.ts#L867-L870: replace the fourNumber(r?.field ?? 0)lines with...mapModeSplit(r).apps/api/src/routes/analytics.ts#L1286-L1289: replace the fourNumber(totals?.field ?? 0)lines with...mapModeSplit(totals).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/lib/mode-split.ts` around lines 51 - 58, Make mapModeSplit null-safe by accepting ModeSplitRow, null, or undefined and defaulting each mapped field to 0. In apps/api/src/lib/mode-split.ts#L51-L58 update mapModeSplit accordingly; in apps/api/src/routes/analytics.ts#L502-L505, `#L825-L828`, `#L867-L870`, and `#L1286-L1289` replace the repeated Number(field ?? 0) conversions with mapModeSplit applied to the corresponding row.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/ui/src/components/dashboard/activity-chart.tsx`:
- Around line 329-341: Mark token metrics as all-traffic values whenever the
selected usage mode is not total, without changing costs or request counts.
Update the Tokens metric in apps/ui/src/components/dashboard/activity-chart.tsx
lines 329-341, the Tokens summary in
apps/ui/src/app/dashboard/[orgId]/org/analytics/org-analytics-client.tsx lines
221-225, Total Tokens and Error Rate in
apps/ui/src/app/dashboard/[orgId]/org/team/[userId]/member-detail-client.tsx
lines 108-145, agent token totals in
apps/ui/src/components/activity/agents-view.tsx lines 703-708, and the Tokens
summary in apps/ui/src/components/dashboard/developer-dashboard-client.tsx lines
107-129; use the existing usage-mode state and either add an all-traffic label
or disable the metric as appropriate.
In `@apps/ui/src/components/dashboard/dashboard-client.tsx`:
- Around line 546-571: Update the subtitle construction in the spend summary
block to avoid presenting blended totalRequestCost as mode-specific when
usageMode is "credits" or "api-keys". Either omit the requests segment outside
"total" mode or explicitly label it as covering all traffic, while preserving
the existing total-mode wording and other mode-adjusted values.
In `@apps/ui/src/components/shared/usage-mode-selector.tsx`:
- Around line 48-68: Update the buttons rendered by USAGE_MODE_OPTIONS.map in
the usage mode selector to expose selection semantics: add the appropriate group
semantics to the surrounding control and set aria-pressed on each button based
on whether option.value matches mode, while preserving the existing visual
styling and click behavior.
---
Outside diff comments:
In `@apps/api/src/routes/analytics.ts`:
- Around line 536-582: Update the topModels and topProviders selection in the
analytics route to rank candidates using the selected mode’s cost rather than
only blended r.cost. Apply mapModeSplit-derived mode costs before slicing, or
retain enough modelRows/provider entries to perform mode-specific sorting and
top-N selection so Credits and BYOK leaders are not excluded.
In `@apps/ui/src/components/dashboard/dashboard-client.tsx`:
- Around line 305-369: Use the untransformed rawActivityData requestCount total
as the denominator for blended reliability metrics, while retaining activityData
for mode-specific cost and request totals. Update cacheHitRate and the
request-count value passed to ErrorsReliabilityCard so cache and error rates use
the raw blended request count, and keep the existing zero-denominator
safeguards.
In `@apps/ui/src/components/usage/model-usage-table.tsx`:
- Around line 134-151: Update the existing-entry merge branch in the modelMap
aggregation loop to also add creditsRequestCount, apiKeysRequestCount,
creditsCost, and apiKeysCost from each repeated model entry, alongside the
existing totals. Keep the current provider|id grouping and other field
aggregation unchanged.
---
Nitpick comments:
In `@apps/api/src/lib/mode-split.ts`:
- Around line 51-58: Make mapModeSplit null-safe by accepting ModeSplitRow,
null, or undefined and defaulting each mapped field to 0. In
apps/api/src/lib/mode-split.ts#L51-L58 update mapModeSplit accordingly; in
apps/api/src/routes/analytics.ts#L502-L505, `#L825-L828`, `#L867-L870`, and
`#L1286-L1289` replace the repeated Number(field ?? 0) conversions with
mapModeSplit applied to the corresponding row.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: d3c6fe9b-1145-4589-a4f0-43923402602a
📒 Files selected for processing (43)
apps/api/src/lib/mode-split.tsapps/api/src/lib/user-usage-breakdown.tsapps/api/src/routes/activity.spec.tsapps/api/src/routes/activity.tsapps/api/src/routes/admin-metrics-mode-split.spec.tsapps/api/src/routes/admin.tsapps/api/src/routes/analytics.spec.tsapps/api/src/routes/analytics.tsapps/api/src/routes/logs.spec.tsapps/api/src/routes/logs.tsapps/api/src/routes/organization.spec.tsapps/api/src/routes/organization.tsapps/ui/src/app/dashboard/[orgId]/org/analytics/org-analytics-client.tsxapps/ui/src/app/dashboard/[orgId]/org/team/[userId]/member-detail-client.tsxapps/ui/src/app/dashboard/[orgId]/org/team/team-client.tsxapps/ui/src/components/activity/agents-view.tsxapps/ui/src/components/activity/recent-logs.tsxapps/ui/src/components/analytics/analytics-client.tsxapps/ui/src/components/analytics/chart-helpers.tsapps/ui/src/components/api-keys/api-key-stats-client.tsxapps/ui/src/components/dashboard/activity-chart.tsxapps/ui/src/components/dashboard/dashboard-client.tsxapps/ui/src/components/dashboard/developer-dashboard-client.tsxapps/ui/src/components/enterprise/feature-showcase.tsxapps/ui/src/components/shared/usage-mode-selector.tsxapps/ui/src/components/usage/cost-breakdown-chart.tsxapps/ui/src/components/usage/model-usage-client.tsxapps/ui/src/components/usage/model-usage-table.tsxapps/ui/src/components/usage/usage-chart.tsxapps/ui/src/components/usage/usage-client.tsxapps/ui/src/lib/usage-mode.tsapps/ui/src/types/activity.tsee/admin/src/app/chat-plans/[orgId]/page.tsxee/admin/src/app/devpass/[orgId]/page.tsxee/admin/src/app/global-stats/client.tsxee/admin/src/app/organizations/[orgId]/org-metrics.tsxee/admin/src/app/organizations/[orgId]/projects/[projectId]/project-metrics.tsxee/admin/src/app/organizations/page.tsxee/admin/src/app/page.tsxee/admin/src/components/cost-by-model-chart.tsxee/admin/src/components/cost-by-model-timeseries-chart.tsxee/admin/src/components/usage-mode-selector.tsxee/admin/src/lib/usage-mode.ts
Comment on lines
+329
to
+341
| const usageMode = useUsageMode(); | ||
| const data = useMemo( | ||
| () => | ||
| rawData | ||
| ? { | ||
| ...rawData, | ||
| activity: rawData.activity.map((day) => | ||
| applyUsageModeToDaily(day, usageMode), | ||
| ), | ||
| } | ||
| : rawData, | ||
| [rawData, usageMode], | ||
| ); |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Mark metrics that remain blended across usage modes.
The selected mode changes costs and request counts. Tokens and error rates remain all-traffic values. Do not present these values as mode-specific metrics.
apps/ui/src/components/dashboard/activity-chart.tsx#L329-L341: Whenmodeis nottotal, label the Tokens metric as all traffic or disable it.apps/ui/src/app/dashboard/[orgId]/org/analytics/org-analytics-client.tsx#L221-L225: Label the Tokens summary as all traffic when a non-total mode is active.apps/ui/src/app/dashboard/[orgId]/org/team/[userId]/member-detail-client.tsx#L108-L145: Label Total Tokens and Error Rate as all traffic when a non-total mode is active.apps/ui/src/components/activity/agents-view.tsx#L703-L708: Label agent token totals as all traffic when a non-total mode is active.apps/ui/src/components/dashboard/developer-dashboard-client.tsx#L107-L129: Label the Tokens summary as all traffic when a non-total mode is active.
📍 Affects 5 files
apps/ui/src/components/dashboard/activity-chart.tsx#L329-L341(this comment)apps/ui/src/app/dashboard/[orgId]/org/analytics/org-analytics-client.tsx#L221-L225apps/ui/src/app/dashboard/[orgId]/org/team/[userId]/member-detail-client.tsx#L108-L145apps/ui/src/components/activity/agents-view.tsx#L703-L708apps/ui/src/components/dashboard/developer-dashboard-client.tsx#L107-L129
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/ui/src/components/dashboard/activity-chart.tsx` around lines 329 - 341,
Mark token metrics as all-traffic values whenever the selected usage mode is not
total, without changing costs or request counts. Update the Tokens metric in
apps/ui/src/components/dashboard/activity-chart.tsx lines 329-341, the Tokens
summary in
apps/ui/src/app/dashboard/[orgId]/org/analytics/org-analytics-client.tsx lines
221-225, Total Tokens and Error Rate in
apps/ui/src/app/dashboard/[orgId]/org/team/[userId]/member-detail-client.tsx
lines 108-145, agent token totals in
apps/ui/src/components/activity/agents-view.tsx lines 703-708, and the Tokens
summary in apps/ui/src/components/dashboard/developer-dashboard-client.tsx lines
107-129; use the existing usage-mode state and either add an all-traffic label
or disable the metric as appropriate.
Comment on lines
+546
to
+571
| label={ | ||
| usageMode === "credits" | ||
| ? "Credits Spend" | ||
| : usageMode === "api-keys" | ||
| ? "BYOK Usage" | ||
| : "Total Spend" | ||
| } | ||
| value={`$${totalCost.toFixed(2)}`} | ||
| subtitle={ | ||
| totalRequests > 0 | ||
| ? `avg $${avgCostPerRequest.toFixed(4)} per request${ | ||
| totalRequestCost > 0 | ||
| ? ` • $${totalRequestCost.toFixed(2)} requests` | ||
| : "" | ||
| }${ | ||
| totalDataStorageCost > 0 | ||
| ? ` • $${totalDataStorageCost.toFixed(4)} storage` | ||
| : "" | ||
| }` | ||
| : `${format(from, "MMM d")} – ${format(to, "MMM d")}` | ||
| usageMode === "total" && | ||
| totalCreditsCost > 0 && | ||
| totalApiKeysCost > 0 | ||
| ? `$${totalCreditsCost.toFixed(2)} credits • $${totalApiKeysCost.toFixed(2)} BYOK (not billed)` | ||
| : usageMode === "api-keys" && totalCost > 0 | ||
| ? "Served by your provider keys — not billed to credits" | ||
| : totalRequests > 0 | ||
| ? `avg $${avgCostPerRequest.toFixed(4)} per request${ | ||
| totalRequestCost > 0 | ||
| ? ` • $${totalRequestCost.toFixed(2)} requests` | ||
| : "" | ||
| }${ | ||
| totalDataStorageCost > 0 | ||
| ? ` • $${totalDataStorageCost.toFixed(4)} storage` | ||
| : "" | ||
| }` | ||
| : `${format(from, "MMM d")} – ${format(to, "MMM d")}` |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Spend subtitle mixes a mode-adjusted figure with a blended figure.
In Credits or BYOK mode, the subtitle falls through to avg $X per request • $Y requests • $Z storage (Line 562-570). totalDataStorageCost ($Z storage) is mode-adjusted, because dataStorageCost has per-mode columns and gets rewritten by applyUsageModeToDaily. totalRequestCost ($Y requests, Line 340-341) has no per-mode column and always stays blended.
The result: within the same subtitle, $Z storage reflects only the selected mode, but $Y requests reflects all traffic. Combined with the mode-specific label ("Credits Spend" / "BYOK Usage") and the mode-specific $X per request average, this makes the request-fee figure look like it belongs to the selected mode when it does not.
Either omit the blended $Y requests segment when usageMode !== "total", or label it explicitly as blended (for example, "$Y requests (all traffic)").
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/ui/src/components/dashboard/dashboard-client.tsx` around lines 546 -
571, Update the subtitle construction in the spend summary block to avoid
presenting blended totalRequestCost as mode-specific when usageMode is "credits"
or "api-keys". Either omit the requests segment outside "total" mode or
explicitly label it as covering all traffic, while preserving the existing
total-mode wording and other mode-adjusted values.
Comment on lines
+48
to
+68
| {USAGE_MODE_OPTIONS.map((option) => ( | ||
| <button | ||
| key={option.value} | ||
| type="button" | ||
| onClick={() => setMode(option.value)} | ||
| title={ | ||
| option.value === "api-keys" | ||
| ? "Usage served by your own provider keys (not billed to credits)" | ||
| : option.value === "credits" | ||
| ? "Usage billed against your credit balance" | ||
| : "All traffic" | ||
| } | ||
| className={cn( | ||
| "rounded-md px-3 py-1 text-xs font-medium transition-colors", | ||
| mode === option.value | ||
| ? "bg-background text-foreground shadow-sm" | ||
| : "text-muted-foreground hover:text-foreground", | ||
| )} | ||
| > | ||
| {option.label} | ||
| </button> |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Expose the selected mode to assistive technology.
Line 62 applies selected state only through CSS. Add group semantics and aria-pressed to each button. This lets screen readers identify the active usage mode.
Proposed fix
<div
+ role="group"
+ aria-label="Usage mode"
className={cn(
"inline-flex items-center rounded-lg border border-border/60 bg-muted/40 p-0.5",
className,
@@
key={option.value}
type="button"
+ aria-pressed={mode === option.value}
onClick={() => setMode(option.value)}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/ui/src/components/shared/usage-mode-selector.tsx` around lines 48 - 68,
Update the buttons rendered by USAGE_MODE_OPTIONS.map in the usage mode selector
to expose selection semantics: add the appropriate group semantics to the
surrounding control and set aria-pressed on each button based on whether
option.value matches mode, while preserving the existing visual styling and
click behavior.
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
BYOK ("api-keys" mode) and credits usage were blended in every reported spend number, in both the user dashboard and the admin dashboard. Billing itself was correct — the worker only debits credits for
usedMode="credits"rows (plus storage cost for BYOK rows) — but reporting showed the blended total everywhere, so neither users nor admins could tell billed credits spend apart from unbilled BYOK usage.The data layer was already fully instrumented:
log.usedModerecords the billing mode per request, and every hourly rollup table has carriedcreditsCost/apiKeysCost/creditsRequestCount/apiKeysRequestCountsince the tables were created (PR #1612), so the split is complete for all history and no migration or backfill is needed. The gaps were purely in the reporting endpoints and the UIs.Approach
API (additive, backward compatible): every aggregation endpoint now returns the four split fields next to each blended
cost/requestCount—/activitynested breakdowns (model/apiKey/user) and/activity/sources, all four/analytics/*endpoints, and the admin metrics, org list, org/project metrics, cost-by-model, timeseries, model-provider-stats and global-stats endpoints. Existing blended fields keep their semantics./logsgains a server-sideusedModefilter.UI: a shared All / Credits / BYOK segmented selector (persisted in the
modeURL param) on every relevant page. Since the server returns all variants at once, switching is a pure client-side view toggle — no extra fetches. Rows are normalized once at the fetch boundary (applyUsageMode/applyUsageModeToDaily), so all downstream charts and reducers work unchanged. Wired in: dashboard, Usage & Metrics, model usage, project analytics, org analytics, team + member detail, developer "my usage", API key stats, agents/sources, and a "Billing" filter on the logs list. Same pattern inee/admin: global metrics, org list, org/project metrics, cost-by-model tables + timeseries, global stats.Real bugs fixed along the way
GET /orgs/{id}/credits-runway) divided the balance by blended 7-day spend, understating runway for BYOK-heavy orgs. It now uses what the worker actually debits:creditsCost + apiKeysDataStorageCost.unusedCredits/overagesubtracted blended spend from topped-up credits; now derived from debited spend only (the blendedtotalSpentfield is unchanged, withtotalCreditsSpent/totalApiKeysSpentadded alongside).Known limitations (deliberate)
team.ts/ gatewayapi-key-usage-limits.ts) still counts BYOK cost toward limits — changing that is a behavior decision left out of scope.creditsDataStorageCost— billing question, untouched here, flagged as follow-up.Verification
pnpm build— all 17 tasks pass.pnpm test:unit— 222 files, 3698 tests pass, including new coverage: mixed-mode seeds asserting the splits on/activity(day totals, model/apiKey/user/source breakdowns), all four/analytics/*endpoints, the credits-runway formula ((7 + 0.7)/7 = 1.1, explicitly not707.7/7), the/logs?usedMode=filter, and a newadmin-metrics-mode-split.spec.ts(metrics split + debited-spendunusedCredits, org list/org/project splits, cost-by-model splits, DevPass margin excluding BYOK).🤖 Generated with Claude Code
Summary by CodeRabbit
Screenshots
User dashboard — All view (split subtitle on Total Spend)
User dashboard — BYOK view
Usage & Metrics — Costs tab with Credits (billed) / BYOK (not billed) footer
Org analytics — selector in header
Admin — global dashboard (Credit flow spent split, Cost by Model selector)
Admin — org detail metrics, All view
Admin — org detail metrics, Credits view
Admin — organizations list (Total Spent split line)