Skip to content

fix(code): scope DevPass charts to dev-plan API key - #2269

Merged
smakosh merged 5 commits into
mainfrom
claude/fix-api-key-chart-scope-BDmQ1
May 12, 2026
Merged

smakosh merged 5 commits into
mainfrom
claude/fix-api-key-chart-scope-BDmQ1

Conversation

@smakosh

@smakosh smakosh commented May 12, 2026 •

Copy link
Copy Markdown
Member

Limit the model breakdown chart and usage overview in the DevPass
dashboard to the API key(s) created for the dev plan, including any
previously rotated keys, so usage from API keys created via the
llmgateway dashboard for the same project is excluded.

  • Return apiKeyIds (active + rotated) from /dev-plans/status
  • Accept comma-separated apiKeyId in /activity and /logs
  • Source AgentModelUsageChart from the aggregated /activity endpoint

Summary by CodeRabbit

  • Refactor
    • Agent model usage analytics restructured to provide project-level performance insights
    • Performance metrics now aggregated and calculated per project instead of organization-wide
    • Agent usage chart now conditionally displays only when a project is selected
    • Chart header subtitle updated to clarify project-scoped reporting
    • Data fetching mechanism updated to retrieve project-filtered activity data

Review Change Stack

Limit the model breakdown chart and usage overview in the DevPass
dashboard to the API key(s) created for the dev plan, including any
previously rotated keys, so usage from API keys created via the
llmgateway dashboard for the same project is excluded.

- Return apiKeyIds (active + rotated) from /dev-plans/status
- Accept comma-separated apiKeyId in /activity and /logs
- Source AgentModelUsageChart from the aggregated /activity endpoint
@coderabbitai

coderabbitai Bot commented May 12, 2026 •

Copy link
Copy Markdown
Contributor

Walkthrough

Dashboard components refactored to use project-scoped agent model usage data. AgentModelUsageChart now fetches from the activity API filtered by projectId instead of logs filtered by sources, and projectId is threaded through CodingAgents from DashboardClient and UsageOverview.

Changes

Project-scoped agent model usage refactor

Layer / File(s) Summary
AgentModelUsageChart refactor to activity API
apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx
Component is refactored to accept projectId instead of sources array, switch query endpoint from /logs to /activity, add isHourlyRange() helper for time-range classification, and completely rewrite data aggregation to compute per-model metrics (cost, requests, tokens) from activity response data with updated chart header text.
CodingAgents projectId integration
apps/code/src/app/dashboard/components/CodingAgents.tsx
Component signature extended to accept projectId prop; logs query updated to conditionally include projectId parameter and require both orgId and projectId to be truthy; AgentDetail type updated to require projectId; projectId is passed to both AgentDetail and AgentModelUsageChart in place of stats.agent.sources.
Project ID threading from DashboardClient and UsageOverview
apps/code/src/app/dashboard/DashboardClient.tsx, apps/code/src/app/dashboard/components/UsageOverview.tsx
DashboardClient extracts projectId from devPlanStatus and passes it to CodingAgents; UsageOverview removes ALL_CODING_AGENT_SOURCES import and conditionally renders AgentModelUsageChart only when projectId is available, passing projectId as prop instead of aggregated sources.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • theopenco/llmgateway#2233: Adds client-side per-model breakdown and sortable table to agent model usage, complementing this PR's shift to project-scoped activity API data.
  • theopenco/llmgateway#2252: Modifies dashboard agent/model usage chart rendering and component interfaces, overlapping with changes to AgentModelUsageChart usage and data sources.
  • theopenco/llmgateway#2243: Adds AgentModelUsageChart and CodingAgents components that are now refactored in this PR to use projectId-based activity fetching.
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Title check ⚠️ Warning The title 'fix(code): scope DevPass charts to dev-plan API key' refers to scoping charts by API key, but the actual implementation scopes charts by projectId to survive API key rotation, not by the API key itself. Update the title to reflect the actual implementation: 'fix(code): scope DevPass charts to projectId' or 'fix(code): scope DevPass charts by dev-plan project' to accurately describe the change.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/fix-api-key-chart-scope-BDmQ1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/api/src/routes/dev-plans.ts (1)

793-831: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use the same Dev Plan key predicate for apiKey creation/lookup and apiKeyIds collection.

Line 813 calls getOrCreatePersonalOrgApiKey, but that helper currently accepts any non-deleted project key. In projects with existing dashboard-created keys, /status can return a non-DevPass apiKey while apiKeyIds stays empty or mismatched, which breaks this PR’s key-scoped dashboard behavior.

Suggested fix
 async function getOrCreatePersonalOrgApiKey(
 	orgId: string,
 	projectId: string,
 	userId: string,
 ): Promise<string> {
 	// Check for existing API key
 	const existingKey = await db.query.apiKey.findFirst({
 		where: {
 			projectId: {
 				eq: projectId,
 			},
+			description: {
+				eq: "Dev Plan API Key",
+			},
 			status: {
 				ne: "deleted",
 			},
 		},
 	});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/api/src/routes/dev-plans.ts` around lines 793 - 831, The code uses
getOrCreatePersonalOrgApiKey to set apiKey but collects apiKeyIds only for keys
with description "Dev Plan API Key", causing mismatches; update the
lookup/creation to use the same "Dev Plan API Key" predicate so both single
apiKey and apiKeyIds refer to the same Dev Plan key set: either change
getOrCreatePersonalOrgApiKey to accept/force a description parameter (e.g., "Dev
Plan API Key") and use that when querying/inserting, or replace the call with an
explicit db.query.apiKey.findFirst/insert that filters on projectId and
description === "Dev Plan API Key"; ensure the variable apiKey returned matches
the keys collected into apiKeyIds.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx`:
- Around line 170-190: AgentModelUsageChart lost agent-level filtering—restore
source-scoped filtering by adding an agent/source param to the query and to the
component props; update AgentModelUsageChart to accept the agent identifier
(e.g., agentId or source) and include it in the api.useQuery params.query
(alongside timeRange and apiKeyId: apiKeyIdParam) so the backend receives
source-scoped requests, or alternatively call the source-scoped API path used on
the agent detail page; ensure the same fix is applied to the other chart usages
noted (lines ~198-231 and ~296-297) so all charts filter by the agent/source
when rendered on the agent detail page.

---

Outside diff comments:
In `@apps/api/src/routes/dev-plans.ts`:
- Around line 793-831: The code uses getOrCreatePersonalOrgApiKey to set apiKey
but collects apiKeyIds only for keys with description "Dev Plan API Key",
causing mismatches; update the lookup/creation to use the same "Dev Plan API
Key" predicate so both single apiKey and apiKeyIds refer to the same Dev Plan
key set: either change getOrCreatePersonalOrgApiKey to accept/force a
description parameter (e.g., "Dev Plan API Key") and use that when
querying/inserting, or replace the call with an explicit
db.query.apiKey.findFirst/insert that filters on projectId and description ===
"Dev Plan API Key"; ensure the variable apiKey returned matches the keys
collected into apiKeyIds.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 451502df-4d64-4691-85fd-4ac695181c0f

📥 Commits

Reviewing files that changed from the base of the PR and between d0f6638 and e6e844d.

⛔ Files ignored due to path filters (2)
  • apps/code/src/lib/api/v1.d.ts is excluded by !**/v1.d.ts
  • apps/ui/src/lib/api/v1.d.ts is excluded by !**/v1.d.ts
📒 Files selected for processing (7)
  • apps/api/src/routes/activity.ts
  • apps/api/src/routes/dev-plans.ts
  • apps/api/src/routes/logs.ts
  • apps/code/src/app/dashboard/DashboardClient.tsx
  • apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx
  • apps/code/src/app/dashboard/components/CodingAgents.tsx
  • apps/code/src/app/dashboard/components/UsageOverview.tsx

Comment thread apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx Outdated
steebchen and others added 3 commits May 12, 2026 08:15
The personal org+project is dedicated to DevPass (personal orgs are
hidden from the regular llmgateway UI per organization.ts), so scoping
charts to that project is sufficient and survives API key rotation
without needing to track historical key IDs.

- Drop the apiKeyIds plumbing from /dev-plans/status and the
  comma-separated apiKeyId support in /activity and /logs
- Scope CodingAgents and AgentModelUsageChart by projectId from
  /dev-plans/status

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx (1)

79-83: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Agent detail chart lost per-agent scoping and now renders project-wide data.

Line 79 and Line 181 only scope by projectId, so in the agent-detail context this chart no longer matches the selected agent’s stats/table. Please restore an agent/source filter path for detail mode (or a dedicated detail query) so the chart remains consistent with the selected agent.

Also applies to: 170-183

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx` around lines
79 - 83, The chart lost per-agent scoping because AgentModelUsageChart only
accepts projectId; restore an agent/source filter by adding an agentId (or
sourceId) prop to the AgentModelUsageChart component and ensure the data query /
filter includes that identifier (e.g., add agentId to the query's where/filter
conditions used to build ChartRow results). Update any call sites that render
AgentModelUsageChart in detail mode to pass the selected agent's id, and ensure
the data-fetching function or hook that produces ChartRow honors agentId when
present (fall back to project-wide when agentId is undefined).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/code/src/app/dashboard/components/CodingAgents.tsx`:
- Around line 622-623: The query in the CodingAgents component is disabled when
projectId is missing (enabled: !!orgId && !!projectId) but the component still
renders the empty “No agent activity yet” state; update CodingAgents to
explicitly handle the missing projectId case instead of showing the empty-state
copy: add a guard early (if (!projectId) return null or a specific placeholder)
or add an explicit branch before the empty-state block that checks !projectId
and renders nothing or a different message; ensure you reference the existing
query config (enabled: !!orgId && !!projectId) and the empty-state rendering
logic so the component doesn’t present misleading “no activity” text when the
query was never run.

---

Duplicate comments:
In `@apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx`:
- Around line 79-83: The chart lost per-agent scoping because
AgentModelUsageChart only accepts projectId; restore an agent/source filter by
adding an agentId (or sourceId) prop to the AgentModelUsageChart component and
ensure the data query / filter includes that identifier (e.g., add agentId to
the query's where/filter conditions used to build ChartRow results). Update any
call sites that render AgentModelUsageChart in detail mode to pass the selected
agent's id, and ensure the data-fetching function or hook that produces ChartRow
honors agentId when present (fall back to project-wide when agentId is
undefined).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 20c3b891-a044-4444-9641-d6e548d34ae5

📥 Commits

Reviewing files that changed from the base of the PR and between e6e844d and edfa84e.

📒 Files selected for processing (4)
  • apps/code/src/app/dashboard/DashboardClient.tsx
  • apps/code/src/app/dashboard/components/AgentModelUsageChart.tsx
  • apps/code/src/app/dashboard/components/CodingAgents.tsx
  • apps/code/src/app/dashboard/components/UsageOverview.tsx

Comment thread apps/code/src/app/dashboard/components/CodingAgents.tsx
@smakosh
smakosh enabled auto-merge May 12, 2026 10:21
@smakosh
smakosh added this pull request to the merge queue May 12, 2026
Merged via the queue into main with commit 0bb5eb8 May 12, 2026
11 checks passed
@smakosh
smakosh deleted the claude/fix-api-key-chart-scope-BDmQ1 branch May 12, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants