-
Notifications
You must be signed in to change notification settings - Fork 190
docs: add analytics changelog and KB pages #2780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
47f3236
docs: add analytics changelog and KB pages
smakosh 75498f2
Optimised images with calibre/image-actions
steebchen 511eb3d
Optimised images with calibre/image-actions
steebchen 32a8b21
Optimised images with calibre/image-actions
steebchen db13a82
docs(changelog): add org-analytics OG image
smakosh 6084815
Merge branch 'main' into docs/analytics-changelog-and-kb
smakosh f5b9bd6
Optimised images with calibre/image-actions
steebchen cc84ce4
Optimised images with calibre/image-actions
steebchen 3bf53a1
Optimised images with calibre/image-actions
steebchen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| title: Analytics | ||
| description: Break down cost, requests, and tokens by model for any project | ||
| icon: ChartArea | ||
| --- | ||
|
|
||
| import { ThemedImage } from "@/components/themed-image"; | ||
|
|
||
| The Analytics page shows where a project's spend actually goes. It breaks usage down by model — as a ranking and over time — so you can see which models drive cost, requests, and tokens across any date range. | ||
|
|
||
| <ThemedImage alt="Project Analytics" basePath="/learn/analytics" /> | ||
|
|
||
| Open it from the **Analytics** item in the project sidebar. Like the rest of the dashboard, the page respects the shared date-range picker at the top, so every chart reflects the same window. | ||
|
|
||
| ## Cost by Model | ||
|
|
||
| A horizontal bar chart that ranks the models used in the selected range. Switch the metric with the tabs above the chart: | ||
|
|
||
| | Tab | What it ranks | | ||
| | ------------ | -------------------------------------- | | ||
| | **Cost** | Total spend per model, in USD | | ||
| | **Requests** | Number of requests routed to the model | | ||
| | **Tokens** | Total tokens (input + output) | | ||
|
|
||
| Use it to spot the one or two models responsible for most of your bill, or to confirm that traffic is spread the way you expect. | ||
|
|
||
| ## Cost by Model Over Time | ||
|
|
||
| A stacked area chart that plots the same three metrics across the date range, with one band per model. It has the same **Cost / Requests / Tokens** tabs, plus a **Mappings / Canonical** toggle: | ||
|
|
||
| - **Mappings** — each model variant is shown separately, exactly as it was requested (for example a custom provider mapping is kept distinct from the built-in model). | ||
| - **Canonical** — variants of the same underlying model are collapsed into one canonical model (the provider prefix and tag are dropped), so `openai/gpt-5.5` and a custom mapping of it count as a single line. | ||
|
|
||
| Switch to **Canonical** when you care about the underlying model's total footprint; stay on **Mappings** when you need to compare specific routes. | ||
|
|
||
| ## How the data is computed | ||
|
|
||
| Both charts derive from the same activity data the rest of the dashboard already reads — there is no separate analytics pipeline to wait on. Aggregation happens per time bucket and is timezone-correct, so totals line up with the Activity and Usage pages for the same range. | ||
|
|
||
| For a per-key view of the same breakdowns, see [API Keys](/learn/api-keys#per-key-statistics). For an org-wide, per-person view on the Enterprise plan, see [Member Analytics](/learn/member-analytics). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| --- | ||
| title: Member Analytics | ||
| description: See organization usage and cost broken down per team member | ||
| icon: ChartPie | ||
| --- | ||
|
|
||
| import { Callout } from "fumadocs-ui/components/callout"; | ||
| import { ThemedImage } from "@/components/themed-image"; | ||
|
|
||
| Member Analytics breaks your organization's usage down by person, so you can see who is spending what across every project. It lives on the **Members** page and is available on the **Enterprise** plan. | ||
|
|
||
| <ThemedImage alt="Member Analytics" basePath="/learn/member-analytics" /> | ||
|
|
||
| <Callout type="info"> | ||
| Member analytics require the **Enterprise plan** and an organization **owner** | ||
| or **admin** role. Non-enterprise organizations see an upgrade card, and | ||
| members without admin access see an access notice instead of the data. | ||
| </Callout> | ||
|
|
||
| ## Members Table | ||
|
|
||
| The Members page adds a usage table sorted by spend, so the heaviest users surface first. Each row shows that member's totals for the selected date range: | ||
|
|
||
| | Column | Description | | ||
| | -------------- | -------------------------------------------- | | ||
| | **Member** | Name and email of the team member | | ||
| | **Cost** | Total spend attributed to the member, in USD | | ||
| | **Tokens** | Total tokens (input + output) | | ||
| | **Requests** | Number of requests | | ||
| | **Error rate** | Share of the member's requests that failed | | ||
| | **API keys** | How many API keys the member created | | ||
|
|
||
| Usage is attributed by **who created each API key** — spend lands on the member who owns the key that made the request, which is the only link between usage and a user. | ||
|
|
||
| ## Member Detail | ||
|
|
||
| Click a member to open their detail page, scoped to the same date range: | ||
|
|
||
| <ThemedImage | ||
| alt="Member Analytics detail" | ||
| basePath="/learn/member-analytics-detail" | ||
| /> | ||
|
|
||
| The detail view includes: | ||
|
|
||
| - **Summary cards** — the member's cost, tokens, requests, and error rate for the range. | ||
| - **Most used** — their top model, provider, and app. | ||
| - **Cost by model** — the same breakdown as the project [Analytics](/learn/analytics) page, scoped to this member. | ||
| - **Top providers and top apps** — tables ranking where the member's traffic goes. | ||
|
|
||
| This makes it easy to attribute cost to a team, investigate a spike, or confirm a member is using the models and providers you expect. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| --- | ||
| id: "60" | ||
| slug: "org-analytics" | ||
| date: "2026-06-21" | ||
| title: "Usage Analytics by Model, Key and Member" | ||
| summary: "See exactly where your spend goes. Every project gets a Cost by Model analytics page, every API key gets its own statistics page, and Enterprise orgs get per-member usage breakdowns — all on the date-range picker you already use. Member analytics are available on Enterprise." | ||
| image: | ||
| src: "/changelog/org-analytics.png" | ||
| alt: "Usage analytics on LLM Gateway: cost by model, per-API-key statistics, and per-member spend breakdowns" | ||
| width: 1536 | ||
| height: 1024 | ||
| --- | ||
|
|
||
| The dashboard could already tell you how much you spent and how many requests you sent — but not where it actually went. Which model is eating the budget? Which API key drives the traffic? Which person on the team? **Analytics** answers all three, with cost, request, and token breakdowns by model, by key, and by member. | ||
|
|
||
| ## Cost by model, on every project | ||
|
|
||
| Each project now has an **Analytics** page in the sidebar with two charts: | ||
|
|
||
| - **Cost by Model** — a horizontal bar chart ranking your models, with **Cost / Requests / Tokens** tabs. | ||
| - **Cost by Model Over Time** — a stacked area chart over your selected range, with the same tabs plus a **Mappings / Canonical** toggle that collapses provider-tagged variants (e.g. `openai/gpt-5.5` and a custom mapping of the same model) into one canonical model. | ||
|
|
||
| Both derive from the same activity data the dashboard already reads, so they're timezone-correct, respect the shared date-range picker, and need no new data to populate. | ||
|
|
||
| ## Per-API-key statistics | ||
|
|
||
| Every API key gets a dedicated statistics page: summary cards for **cost, tokens, requests, and error rate**, plus the two cost-by-model charts scoped to that single key. The **View Statistics** action in the API-keys list now opens it directly, so you can see at a glance whether a key is healthy and what it's spending on. | ||
|
|
||
| ## Member analytics | ||
|
|
||
| Enterprise organizations also get usage broken down by person. The **Members** page adds a per-member table sorted by spend, and each member has a detail view: | ||
|
|
||
| | Surface | What it shows | | ||
| | ----------------- | ------------------------------------------------------------------- | | ||
| | **Members table** | Per member: cost, tokens, requests, error rate, and API-key count. | | ||
| | **Summary cards** | That member's cost, tokens, requests, and error rate for the range. | | ||
| | **Most used** | Their top model, provider, and app. | | ||
| | **Breakdowns** | Cost by model, plus top providers and top apps tables. | | ||
|
|
||
| Usage is attributed by who created each API key, so spend lands on the member who owns the key. Member analytics are exposed through new `GET /analytics/members` and `GET /analytics/members/{userId}` endpoints, restricted to organization owners and admins on the **Enterprise plan** — non-enterprise orgs see an upgrade card, and non-admins see an access notice. | ||
|
|
||
| --- | ||
|
|
||
| **[Cost breakdown docs →](https://docs.llmgateway.io/features/cost-breakdown)** | **[Open your dashboard →](https://llmgateway.io/dashboard)** | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.