Show what each bot spends - #185
Conversation
Cost and token data was on the wire and the UI threw it away — but the
harness also never had a consistent number to keep: thread.token-usage
.updated means a per-call delta on Claude, a running thread total on
Codex, and a per-step figure plus a turn total on Antigravity, so summing
it double-counted. And only Claude reports a price at all.
- turn.completed gains usage {input, output}: THIS turn's total, from
what each driver already has (Claude result.usage incl. cache reads,
Codex tokenUsage.last ?? total, Antigravity result.usage). The live
indicator is unchanged and never summed.
- ProviderSnapshot.billing: Claude and Codex both strip API keys and run
on the CLI's own login, so both report "subscription" — a Claude cost
is an equivalent, and the UI captions it that way.
- store.addTaskUsage banks each settled turn onto TaskRecord.usage
{input, output, costUsd, turns}; flows to clients via wireTask.
- UI: header chip for the open task ("12.4k tok · $0.06", cost only
when known); a Usage card in bot settings; a Usage tab in app
settings ranking bots by cost then tokens with a fleet total.
Item 1.5 of docs/plans/agent-harness-upgrades-v2.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 44 minutes Limit details: You’ve used all 3 included reviews currently available under your plan. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (18)
Comment |
main's #192 landed its own per-task usage tally (last token-usage event per turn, no cost). Reconciled: TaskRecord.usage keeps the superset shape {input, output, costUsd, turns}; one addTaskUsage (with main's NaN/negative sanitizing); at turn.completed the driver's own per-turn figure (turn.completed.usage) is authoritative and main's last-reported value is the fallback for drivers that only stream the running indicator. Records written before cost existed read costUsd as null. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
main gained #205 (token spend in the task picker, using the same TaskRecord.usage), the store change stream, activity states, and typed approvals. Reconciled: one addTaskUsage with the superset shape (costUsd kept; emits a bot change like main's did), settle path banks the turn's authoritative usage then moves the bot to idle via setActivity, the Usage settings tab sits beside main's new Companion tab. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
In plain language
Before: the engines reported tokens (and Claude a dollar cost) on every turn, and the app threw it away. It also turned out the per-driver token reports meant different things (a per-call delta on Claude, a running thread total on Codex), so summing them naively double-counted.
What changes in the app: a small chip in the chat header shows what the current task has spent — e.g.
12.4k tok · $0.06(dollars only for Claude, the only engine that reports a price). Bot settings get a Usage card; App settings get a Usage tab ranking every bot by cost then tokens with a total — the "which of my bots is costing me money" answer. Cost is captioned honestly: on a subscription it's an equivalent, not a charge. Figures start from zero on upgrade.In plain terms
An app that runs agents unattended should be able to say what that costs. Cost/token data was already on the wire from the drivers and the UI discarded it — but two things the plan assumed turned out to be wrong, and both shaped this:
cost: null. So this is token-first, dollars where known.thread.token-usage.updatedmeans different things per driver — a per-API-call delta on Claude, the thread's running total on Codex, a per-step figure and a turn total on Antigravity. Summing it naively double-counts. The harness never had a consistent figure to keep.Changes
turn.completed.usage?: { input, output }— this turn's total, the one figure the harness banks. Each driver computes it from what it already has: Clauderesult.usage(input + cache read + cache creation), CodextokenUsage.last ?? total, Antigravityresult.usage. The live indicator is unchanged and never summed.ProviderSnapshot.billing?: "metered" | "subscription"— Claude and Codex both deliberately strip API keys and run on the CLI's own login, so both reportsubscription; a Claude cost is an equivalent and the UI says so.store.addTaskUsage()banks each settled turn ontoTaskRecord.usage { input, output, costUsd, turns }(cost staysnulluntil any turn reports one); flows to clients through the existingwireTask. Bot and fleet totals are client-side sums.12.4k tok · $0.06(cost only when known); hover for turns / in-out split / billing caption; click → bot settings.Figures start from zero on upgrade (older turns weren't recorded per task). Out of scope: reset/period windows, per-model breakdown, charts.
Item 1.5 of
docs/plans/agent-harness-upgrades-v2.md.Test plan
turn.completed.usageon the existing fakes: claude ({12, 5}incl. cache read), codex ({7, 3}from a thread total), antigravity ({105, 20}fromresult.usage)server/store.test.ts(+4): banks and counts turns; cost null until reported then sums only reported; token-less turn still counted; unknown task ignoredsrc/lib/usage.test.ts(5): compact tokens, small dollars stay visible, chip text, cross-task sums, billing captionspnpm typecheckclean;pnpm vitest rungreen (65 files, 536 passed)$; bot Usage card and app-settings Usage tab populate;/api/instancesreportsbilling: subscriptionfor claude and codex🤖 Generated with Claude Code