feat: add org analytics, member & API-key stats - #2769
Merged
Merged
Conversation
Surface admin-style usage analytics to organizations in apps/ui: - New project "Analytics" page with Cost by Model (bar, Cost/ Requests/Tokens) and Cost by Model Over Time (stacked area with Mappings/Canonical toggle). Both derive from the existing /activity endpoint client-side, so they reuse the dashboard date picker and timezone-correct aggregation. - New per-API-key statistics page (/api-keys/[keyId]) with summary cards and the two cost-by-model charts scoped to one key. Repoints the "View Statistics" action to it. - New enterprise-gated org Members pages: a per-member usage table (cost/tokens/requests, attributed by apiKey.createdBy) and a member detail view (most-used model/provider/app, cost by model, top providers/apps). Backed by a new /analytics route restricted to enterprise owners/admins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
WalkthroughAdds enterprise-only analytics endpoints ( ChangesEnterprise Analytics: Member Usage API + UI
Sequence Diagram(s)sequenceDiagram
participant Browser
participant MembersClient
participant AnalyticsAPI
participant DB
Browser->>MembersClient: Load /org/members?from=...&to=...
MembersClient->>AnalyticsAPI: GET /analytics/members?organizationId&from&to
AnalyticsAPI->>DB: Check caller role in organization
AnalyticsAPI->>DB: Verify org exists + plan=enterprise
AnalyticsAPI->>DB: Fetch org members + project API keys
AnalyticsAPI->>DB: Aggregate hourly stats by API key
AnalyticsAPI-->>MembersClient: MemberUsage[] sorted by cost
MembersClient-->>Browser: Render members table with error rates
Browser->>MemberDetailClient: Load /org/members/{userId}?from&to
MemberDetailClient->>AnalyticsAPI: GET /analytics/members/{userId}?organizationId&from&to
AnalyticsAPI->>DB: Validate membership in org
AnalyticsAPI->>DB: Fetch member API keys for org projects
AnalyticsAPI->>DB: Compute summary + top models/providers
AnalyticsAPI-->>MemberDetailClient: summary + topByModel + topByProvider
MemberDetailClient-->>Browser: Render stat cards + CostByModelCard + provider table
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
apps/api/src/routes/analytics.ts (1)
40-46: 💤 Low valueConsider validating date inputs.
If
fromortoare malformed date strings (e.g., "not-a-date"),new Date()returnsInvalid Date, which propagates to the query and may cause unexpected behavior. A validation check before parsing could make error handling more predictable.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/api/src/routes/analytics.ts` around lines 40 - 46, In the date range parsing logic where from and to parameters are converted to Date objects, add validation to ensure the date strings are valid before creating Date objects. After constructing startDate and endDate from the from and to parameters using new Date(), check if either date is Invalid (using isNaN() or checking the valueOf() result), and if so, throw an error or return an appropriate error response to the caller. This prevents malformed date strings from propagating invalid Date objects downstream.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@apps/ui/src/app/dashboard/`[orgId]/org/members/[userId]/member-detail-client.tsx:
- Around line 38-68: The member-detail-client component currently allows any
user to query the protected /analytics/members/{userId} endpoint and silently
degrades when access is denied. Add enterprise/admin permission gating before
the useEffect hook by checking if the current user has the required permissions
(using a similar pattern to the members list page), and display an explicit
access denied UI component (such as an upgrade notice or access required
message) when permissions are insufficient instead of proceeding with the query.
Conditionally enable the api.useQuery call only after confirming the user has
appropriate access, so the endpoint is not queried without authorization.
In `@apps/ui/src/app/dashboard/`[orgId]/org/members/members-client.tsx:
- Around line 92-97: The api.useQuery hook in the members-client component is
not explicitly handling API failures, causing network errors and HTTP error
responses to silently fall back to the empty state. Destructure the error
property from the api.useQuery call alongside data and isLoading, then add
conditional rendering logic to explicitly check for and display an error message
when error exists, reserving the "No members found" empty state only for cases
where the query succeeded but returned no data. This same pattern should be
applied to the other query location mentioned at lines 152-170 to ensure
consistent error handling throughout the component.
In `@apps/ui/src/components/analytics/chart-helpers.ts`:
- Around line 178-179: The sanitizeKey function can produce identical outputs
from different input strings, causing multiple distinct models to share the same
chart key and overwrite each other's data. Replace the character-replacement
approach in sanitizeKey with a collision-safe method such as hashing the input
(e.g., using a hash function) or prefixing/encoding the sanitized string with a
unique identifier derived from the original model string to ensure each model
gets a unique key even after normalization.
In `@apps/ui/src/components/analytics/cost-by-model-over-time-card.tsx`:
- Around line 98-103: The formatTimestamp function (and similarly on line 208)
uses new Date() to parse date-only strings like "2026-06-01", which the
ECMAScript spec interprets as UTC midnight. This causes the date to display as
the previous day in US timezones. Fix this by adjusting the date parsing logic
to account for the timezone offset when dealing with date-only strings in the
daily bucket case, ensuring the formatted date represents the correct local date
rather than being shifted backward by one day.
In `@apps/ui/src/components/dashboard/dashboard-sidebar.tsx`:
- Around line 178-181: The new menu entry with href "org/members" added to the
sidebar is not being recognized by the parent Settings active-state check, so
the Settings section won't highlight as active when navigating to the members
page. Locate the isActive predicate for the Settings menu item (around lines
472-479) and add "org/members" to the route matching conditions so that the
parent Settings section correctly appears active when the user is on the members
page.
---
Nitpick comments:
In `@apps/api/src/routes/analytics.ts`:
- Around line 40-46: In the date range parsing logic where from and to
parameters are converted to Date objects, add validation to ensure the date
strings are valid before creating Date objects. After constructing startDate and
endDate from the from and to parameters using new Date(), check if either date
is Invalid (using isNaN() or checking the valueOf() result), and if so, throw an
error or return an appropriate error response to the caller. This prevents
malformed date strings from propagating invalid Date objects downstream.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 38ee7d37-497f-43ac-a07d-b05408ce91f7
📒 Files selected for processing (16)
apps/api/src/routes/analytics.tsapps/api/src/routes/index.tsapps/ui/src/app/dashboard/[orgId]/[projectId]/analytics/page.tsxapps/ui/src/app/dashboard/[orgId]/[projectId]/api-keys/[keyId]/page.tsxapps/ui/src/app/dashboard/[orgId]/org/members/[userId]/member-detail-client.tsxapps/ui/src/app/dashboard/[orgId]/org/members/[userId]/page.tsxapps/ui/src/app/dashboard/[orgId]/org/members/members-client.tsxapps/ui/src/app/dashboard/[orgId]/org/members/page.tsxapps/ui/src/components/analytics/analytics-client.tsxapps/ui/src/components/analytics/chart-helpers.tsapps/ui/src/components/analytics/cost-by-model-card.tsxapps/ui/src/components/analytics/cost-by-model-over-time-card.tsxapps/ui/src/components/api-keys/api-key-stats-client.tsxapps/ui/src/components/api-keys/api-keys-list.tsxapps/ui/src/components/dashboard/animated-nav-icons.tsxapps/ui/src/components/dashboard/dashboard-sidebar.tsx
Address PR review findings: - Gate the member detail page on enterprise plan + owner/admin and only enable the /analytics query once authorized; show an explicit access notice otherwise (matches the members list page). - Surface query errors on the members page instead of silently showing "No members found". - Make sanitizeKey collision-safe (encode non-alphanumerics by code point) so distinct model ids can't overwrite each other in the chart. - Parse date-only bucket strings with parseISO to avoid the UTC-midnight off-by-one in negative-offset timezones. - Mark the sidebar Settings section active on the org/members route. - Validate from/to dates in the analytics route and return 400 on malformed input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Limit the project Analytics and per-API-key stats pages to the last 7 days for non-enterprise plans: a locked "Last 7 days" control with an upsell popover replaces the full date picker, and the effective range is clamped server-query-side so it can't be widened via URL params. Enterprise keeps the full DateRangePicker (any week/month/quarter). Also add a callout on the org Members page pointing enterprise admins to the equivalent per-API-key analytics — useful when usage runs through many keys rather than people. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/ui/src/components/analytics/analytics-date-range.tsx (1)
75-76: ⚡ Quick winUse
FREE_PLAN_RANGE_DAYSfor displayed copy to prevent drift.The logic is centralized in
FREE_PLAN_RANGE_DAYS, but the UI still hardcodes"Last 7 days"and"last 7 days". If the constant changes, behavior and copy will diverge.Suggested update
- Last 7 days + Last {FREE_PLAN_RANGE_DAYS} days ... - Your plan shows the last 7 days. Upgrade to Enterprise to break + Your plan shows the last {FREE_PLAN_RANGE_DAYS} days. Upgrade to Enterprise to breakAlso applies to: 84-85
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/ui/src/components/analytics/analytics-date-range.tsx` around lines 75 - 76, Replace the hardcoded strings "Last 7 days" and "last 7 days" in the analytics-date-range component with dynamic text that uses the FREE_PLAN_RANGE_DAYS constant to ensure the UI copy stays in sync with the constant. Both occurrences (at lines 75-76 and 84-85) should reference this constant when building the display text, so if the constant value changes, the UI automatically reflects the update without maintaining separate hardcoded strings.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@apps/ui/src/components/analytics/analytics-date-range.tsx`:
- Around line 75-76: Replace the hardcoded strings "Last 7 days" and "last 7
days" in the analytics-date-range component with dynamic text that uses the
FREE_PLAN_RANGE_DAYS constant to ensure the UI copy stays in sync with the
constant. Both occurrences (at lines 75-76 and 84-85) should reference this
constant when building the display text, so if the constant value changes, the
UI automatically reflects the update without maintaining separate hardcoded
strings.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 64bc2bdc-56d0-4a63-9aa6-2039e7dd10f0
📒 Files selected for processing (4)
apps/ui/src/app/dashboard/[orgId]/org/members/members-client.tsxapps/ui/src/components/analytics/analytics-client.tsxapps/ui/src/components/analytics/analytics-date-range.tsxapps/ui/src/components/api-keys/api-key-stats-client.tsx
🚧 Files skipped from review as they are similar to previous changes (3)
- apps/ui/src/components/analytics/analytics-client.tsx
- apps/ui/src/components/api-keys/api-key-stats-client.tsx
- apps/ui/src/app/dashboard/[orgId]/org/members/members-client.tsx
The per-member "top apps" / x-source breakdown was the only query reading the raw log table. No aggregated table exists at the apiKey+source grain (projectHourlySourceStats is project-level only), so the feature can't be served from aggregates. Drop it from the analytics endpoint, member detail UI, and the demo seed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@apps/ui/src/app/dashboard/`[orgId]/org/members/[userId]/member-detail-client.tsx:
- Around line 267-276: The empty state message "No data" is being shown
prematurely while the query is still loading because the condition only checks
if data is empty but does not account for the loading state. Modify the
condition that currently evaluates (data?.topProviders.length ?? 0) === 0 to
also require that isLoading is false, so the "No data" TableRow only renders
after the data has finished loading and is confirmed to be empty.
In `@packages/db/src/seed-demo-analytics.ts`:
- Line 204: The ID template for analytics rows at line 204 is missing the model
provider field, causing ID collisions when different providers share the same
model name. Add model.provider to the id template string alongside the existing
model.model property to ensure each provider-model combination generates a
unique identifier and prevent silent row loss from ON CONFLICT DO NOTHING.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 4177d0b7-0f44-4770-995a-04383f82122f
📒 Files selected for processing (3)
apps/api/src/routes/analytics.tsapps/ui/src/app/dashboard/[orgId]/org/members/[userId]/member-detail-client.tsxpackages/db/src/seed-demo-analytics.ts
💤 Files with no reviewable changes (1)
- apps/api/src/routes/analytics.ts
Comment on lines
+267
to
+276
| {(data?.topProviders.length ?? 0) === 0 ? ( | ||
| <TableRow> | ||
| <TableCell | ||
| colSpan={3} | ||
| className="py-6 text-center text-muted-foreground" | ||
| > | ||
| No data | ||
| </TableCell> | ||
| </TableRow> | ||
| ) : ( |
Contributor
There was a problem hiding this comment.
Avoid showing “No data” while the query is still loading.
When isLoading is true, data is still undefined, so the table renders an empty-state row prematurely. Gate the empty state behind !isLoading to prevent misleading UI.
Suggested patch
<TableBody>
- {(data?.topProviders.length ?? 0) === 0 ? (
+ {isLoading ? (
+ <TableRow>
+ <TableCell
+ colSpan={3}
+ className="py-6 text-center text-muted-foreground"
+ >
+ —
+ </TableCell>
+ </TableRow>
+ ) : (data?.topProviders.length ?? 0) === 0 ? (
<TableRow>
<TableCell
colSpan={3}
className="py-6 text-center text-muted-foreground"
>
No data
</TableCell>
</TableRow>
) : (📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| {(data?.topProviders.length ?? 0) === 0 ? ( | |
| <TableRow> | |
| <TableCell | |
| colSpan={3} | |
| className="py-6 text-center text-muted-foreground" | |
| > | |
| No data | |
| </TableCell> | |
| </TableRow> | |
| ) : ( | |
| <TableBody> | |
| {isLoading ? ( | |
| <TableRow> | |
| <TableCell | |
| colSpan={3} | |
| className="py-6 text-center text-muted-foreground" | |
| > | |
| — | |
| </TableCell> | |
| </TableRow> | |
| ) : (data?.topProviders.length ?? 0) === 0 ? ( | |
| <TableRow> | |
| <TableCell | |
| colSpan={3} | |
| className="py-6 text-center text-muted-foreground" | |
| > | |
| No data | |
| </TableCell> | |
| </TableRow> | |
| ) : ( |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@apps/ui/src/app/dashboard/`[orgId]/org/members/[userId]/member-detail-client.tsx
around lines 267 - 276, The empty state message "No data" is being shown
prematurely while the query is still loading because the condition only checks
if data is empty but does not account for the loading state. Modify the
condition that currently evaluates (data?.topProviders.length ?? 0) === 0 to
also require that isLoading is false, so the "No data" TableRow only renders
after the data has finished loading and is confirmed to be empty.
| hourOutputCost += outputCost; | ||
|
|
||
| keyModelStatsRows.push({ | ||
| id: `akms-${key.id}-${dayOffset}-${hour}-${model.model}`, |
Contributor
There was a problem hiding this comment.
Include provider in model-stat row IDs to prevent silent row loss
At Line 204, the generated id excludes model.provider. If two providers share the same model name for the same key/hour/day, IDs collide and one insert is skipped by ON CONFLICT DO NOTHING, which skews seeded analytics.
Suggested fix
- id: `akms-${key.id}-${dayOffset}-${hour}-${model.model}`,
+ id: `akms-${key.id}-${dayOffset}-${hour}-${model.provider}-${model.model}`,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| id: `akms-${key.id}-${dayOffset}-${hour}-${model.model}`, | |
| id: `akms-${key.id}-${dayOffset}-${hour}-${model.provider}-${model.model}`, |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/db/src/seed-demo-analytics.ts` at line 204, The ID template for
analytics rows at line 204 is missing the model provider field, causing ID
collisions when different providers share the same model name. Add
model.provider to the id template string alongside the existing model.model
property to ensure each provider-model combination generates a unique identifier
and prevent silent row loss from ON CONFLICT DO NOTHING.
smakosh
added this pull request to the merge queue
Jun 21, 2026
huangyingting
pushed a commit
to repomesh/llmgateway
that referenced
this pull request
Jun 21, 2026
## Summary Documents the org analytics, per-API-key statistics, and Enterprise member analytics shipped in theopenco#2769 — across the public changelog and the docs knowledge base. ### Changelog - New entry `2026-06-21-org-analytics` ("Usage Analytics by Model, Key and Member") for the public changelog, covering the project Analytics page, per-key statistics, and member analytics with plan gating. ### Knowledge base (`apps/docs`) - **New page — Analytics** (`learn/analytics`): project Cost by Model and Cost by Model Over Time charts, the Mappings / Canonical toggle, and how the data is computed. - **New page — Member Analytics** (`learn/member-analytics`): the Enterprise per-member usage table and member detail view (summary cards, most-used model/provider, cost by model, top providers), with Enterprise + owner/admin gating. - **API Keys page**: added a **Per-Key Statistics** section documenting the per-key analytics page. - Wired both new pages into `learn/meta.json` and the knowledge-base index. ### Screenshots Eight dashboard screenshots (light + dark) captured from a locally seeded **DataFlow AI** enterprise org via Playwright, dropped under `apps/docs/public/learn/` and referenced with `ThemedImage`: - `analytics-{light,dark}` - `api-key-statistics-{light,dark}` - `member-analytics-{light,dark}` - `member-analytics-detail-{light,dark}` ## Testing - `pnpm format` clean - `turbo run build --filter=docs --filter=ui` passes (validates changelog frontmatter + MDX) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added project Analytics page displaying cost, requests, and tokens by model with date range selection and visual cost trends * Added per-API-key statistics page accessible from the API Keys list * Added Enterprise Member Analytics for organization usage breakdown by team member, including cost, tokens, requests, and error rates * **Documentation** * Added Analytics, API Keys, and Member Analytics learn pages to documentation <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Luca Steeb <contact@luca-steeb.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Surfaces admin-style usage analytics to organizations in
apps/ui. Four additions, all using the existing date picker, typed API client, and TanStack Query.Project "Analytics" page (
/dashboard/[orgId]/[projectId]/analytics)/activityendpoint's per-bucketmodelBreakdown, so they reuse timezone-correct aggregation and the shared date-range picker (no new chart endpoints).Analyticssidebar item +AnimatedChartAreaicon.Per-API-key statistics (
/dashboard/[orgId]/[projectId]/api-keys/[keyId])/activity?apiKeyId=)./usage?apiKeyId=).Enterprise Members analytics (
/dashboard/[orgId]/org/members)/org/members/[userId]): summary cards, most-used model / provider / app, cost-by-model, and top providers / apps tables./analyticsAPI route (GET /analytics/members,GET /analytics/members/{userId}), restricted to enterprise plan + owner/admin. Usage is attributed byapiKey.createdBy(the only usage→user link); "top apps" comes from a bounded raw-logsource query.Implementation notes
apps/api/src/routes/analytics.tsaggregates from the pre-rolledapiKeyHourlyStats/apiKeyHourlyModelStatstables (joined toapiKey.createdBy), matching how the rest of analytics reads hourly rollups.extractCanonicalModelId(drops provider prefix + tag).Testing
turbo buildpasses forapi+ui(and@llmgateway/db); eslint + prettier clean.pnpm seeddoesn't backfill those, so locally they're empty until there's per-key usage.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Improvements