Copilot: surface AI credit usage for token-based-billing seats - #2647
Copilot: surface AI credit usage for token-based-billing seats#2647KSEGIT wants to merge 13 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aaeb632f41
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Codex review: needs maintainer review before merge. Reviewed August 15, 2026, 10:58 AM ET / 14:58 UTC. ClawSweeper reviewWhat this changesAdds a user-entered, per-Copilot-account AI-credit entitlement that turns token-billed seat consumption into a menu-card progress row and immediately rebuilds cached rows when the value changes. Merge readinessKeep open for maintainer product sign-off: the branch contains the still-missing menu-card UI and per-account setting beyond the merged decoding work, has no blocking code finding, and includes current-head live proof. Priority: P2 Review scores
Verification
How this fits togetherCodexBar fetches Copilot quota data, converts it to shared provider detail rows, persists selected-account settings, and renders those rows in the menu card. This change supplies an optional user-entered credit ceiling between the fetched consumption value and that row renderer. flowchart LR
A[Copilot usage API] --> B[Usage snapshot]
C[Selected account setting] --> D[Credit entitlement parser]
B --> E[Credit detail row]
D --> E
E --> F[Cached usage snapshot]
C --> G[Cached row update]
G --> F
F --> H[Menu card]
Decision needed
Why: VISION.md requires sign-off for this new persisted feature, and source review cannot determine whether the card-only scope is the intended product contract. Before merge
Agent review detailsSecurityNone. Review metrics
Root-cause clusterRelationship: Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Approve the narrow seat-only path if the stored, user-entered ceiling is acceptable, keeping credits in shared detail rows and deferring icon/widget/CLI semantics to a separately scoped decision. Do we have a high-confidence way to reproduce the issue? Not applicable as a bug reproduction: the requested new behavior is directly shown on a live token-billed Business seat in four current-head screenshots. Is this the best way to solve the issue? Yes for the seat-only scope: it uses the existing Copilot response, avoids inventing an unknown entitlement, and reuses shared detail rows; accepting the new persisted setting remains a maintainer decision. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against f15f142a7787. LabelsLabel justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (32 earlier review cycles; latest 8 shown)
|
|
Thanks for this, and for the excellent original report in #2593 — you found both the |
clawsweeper
Bot
added
rating: 🧂 unranked krab
|
Re The upstream data is real, not inferredLive from a Copilot Business seat (redacted), via the endpoint CodexBar already calls: "quota_snapshots": {
"premium_interactions": { "unlimited": true, "entitlement": 0, "remaining": 0,
"percent_remaining": 100.0, "credits_used": 31,
"token_based_billing": true, "overage_permitted": true },
"chat": { "unlimited": true, "entitlement": 0, "credits_used": 0 },
"completions": { "unlimited": true, "entitlement": 0, "credits_used": 0 }
},
"organization_login_list": ["<ORG>"],
"copilot_plan": "business", "token_based_billing": true,
"quota_reset_date": "2026-09-01"Note And the org endpoint: { "usageItems": [
{ "product": "Copilot", "sku": "Copilot AI Credits", "model": "Code Review model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 31.13 },
{ "product": "Copilot", "sku": "Copilot Cloud Agent", "model": "Coding Agent model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 49.97 } ] }31.13 + 49.97 = 81.10, which matches the "81 / 6,000 AI credits" numerator on the org billing page exactly. That agreement is the strongest evidence I have that the org lane reads the right thing. What the card actually rendersAsserted on exact strings through the real
The org tests drive What I can't evidence yetNo screenshot. Producing one means building this branch, packaging it, and running it against a real Copilot Business account — which per AGENTS.md is exactly the kind of validation that can raise Keychain prompts, so I didn't do it unasked. Happy to if you want it. Two of three surfaces are untouched. I've added a table to the PR description rather than leave it implied: the menu card shows credits; the menu bar icon, the widget, and So today a Business token-billing user still gets an empty icon and has to open the menu. I can wire the icon up in this PR — publishing credits as a Since the PR was opened
|
clawsweeper
Bot
added
rating: 🦪 silver shellfish
bfe422a to
b7406fb
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
merge-risk: 🚨 auth-provider
clawsweeper
Bot
added
the
rating: 🦐 gold shrimp
ddd77ff to
2d25ce6
Compare
|
Fixed the P2 release-note finding. Rebased onto current The entry had drifted into a released section through no intent of mine: an earlier rebase auto-merged it under what was then It now sits under One deliberate divergence from the recommendation: the suggestion was to remove the entry and let the release process place it when shipped, but main's own Re-verified on the new base: Still outstanding and unchanged: the current-head offline text↔bar proof, which is mine to capture, and the scope sign-off called out under "Decision needed". |
|
Current-head live capture, taken from a build of this branch running against my Copilot Business seat:
What this shows: a real token-billed Business seat rendering Being straight about its limits: this is the end state, not the transition. It does not by itself distinguish the cached text→bar repair from an ordinary successful refresh — the card reads "Updated just now", so a fetch had just landed. The offline sequence that isolates the cached rewrite (clear the entitlement with the network down, watch the row collapse to plain text and come back as a bar while the last-updated timestamp never advances) is still to come, and I'll post it as a follow-up. Posting this now because it does close one gap the last review named: the live bar existing at the current head rather than only at |
|
Current-head behavior evidence, all four frames captured from a build of this branch against my live Copilot Business seat ( 1 — Bar renders at current head, online.
2 — The cached row survives a failed refresh. Network off; the card shows "The Internet connection appears to be offline" and the credit row is still rendered as a bar at
3 — A changed entitlement re-renders the row. Allowance changed
4 — And that survives offline too. Network off again; offline banner present, row still
What these do and don't establishThey establish, on the current head: the bar renders on a real token-billed Business seat; the cached row survives a failed refresh with its denominator intact at two different entitlements; and changing the entitlement re-renders the ratio. They do not isolate the in-place cached rewrite as tightly as I'd like. Frames 2 and 4 are both offline but sit either side of a brief reconnect, so there is no single unbroken offline stretch spanning an entitlement change. I'd rather say that plainly than let the sequence imply more than it shows. Worth noting why the literal "clear it and watch it return to text" capture is awkward on my setup specifically: the settings field resolves as Unit coverage for the exact text↔bar transition is in @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
clawsweeper
Bot
added
proof: sufficient
Remove the opt-in organization billing lane pending an approved authorization model for org-wide billing reads with device-flow tokens. Rewrite the cached seat credit row immediately when the entitlement changes so a stale denominator or bar never survives a failed refresh.
Text-only credit rows carried no structured usage, so entering an entitlement during an offline or failed refresh left the cached row without its bar. Stamp usageValue on the row at fetch time and use it as the repair numerator.
Rebases landed the entry under 0.49.6, which shipped on 2026-08-14, so merging would advertise this feature as part of that release. Move it to 0.49.7 Unreleased; 0.49.6 is now byte-identical to main. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016puVootfTssyPQe2TzmK1M
The credit-entitlement additions to MenuCardView, SettingsStore, and UsageFetcher shift 13 allowlisted provider constructs past the gatekeeper's two-line anchor tolerance. Point each entry at its current line; the anchors, provider sets, and fingerprints are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016puVootfTssyPQe2TzmK1M
2d25ce6 to
31281db
Compare
2 participants
Closes #2593.
The problem
On a Copilot Business seat with
token_based_billing: true,GET /copilot_internal/userreports every quota snapshot asunlimited: true, entitlement: 0, remaining: 0, percent_remaining: 100. The#1258guards correctly drop those so no misleading "0% used" bar appears — but the result is a Copilot card showing only the plan label and no usage at all.Real consumption is available in
credits_used, already present on each quota snapshot in the response CodexBar already fetches (landed in #2613).GitHub publishes no credit entitlement anywhere
This is the finding that shaped the design. I probed all 8 documented billing endpoints plus
budgets,cost-centers,included_credits,ai_credit/entitlement,copilot/metricsandusage/summary. None expose the included-credit ceiling — the "6,000" that the org billing page displays.discountQuantityreveals only what included credits absorbed, so the ceiling is observable only once exceeded.It is derivable as seats × per-seat allowance, but the per-seat figure is currently a promotional 3,000 against a standard 1,900 — a hardcoded table would silently produce a wrong bar when the promo ends. So the denominator is user-entered and never inferred. A row with no entitlement renders as text, never a bar with a fabricated ceiling.
If you know of an endpoint I missed, that would simplify this considerably.
What this adds
Seat credit bar (builds on #2613). Extends the shared provider-detail row contract with an optional row id, progress ratio, and retained numeric usage (
ProviderDetailSection.Row), so the existing "Credits used" row becomes a "31 / 3000" bar when a seat entitlement is set, and stays plain text otherwise. Created only when it carries real signal (token-billed / unlimited quota / credits > 0 / entitlement set), so metered accounts reportingcredits_used: 0don't gain a permanent empty row. No guard line inCopilotUsageFetcher.swiftis modified — the#1258guard block is byte-identical to main.Per-account entitlements. The seat allowance is stored on the selected
ProviderTokenAccount(following the z.ai per-account fields precedent), resolved incopilotSettingsSnapshotwith the existing global UserDefaults value as fallback — so existing single-account setups keep working unchanged, and two Copilot accounts with different allowances each get the right denominator. The settings field writes to the selected account when one exists.Cached rows repair immediately. Changing or clearing the entitlement rewrites the cached "Credits used" row synchronously, in both the live snapshot and the last-known-reset baseline. The numerator comes from the row's structured progress, falling back to the retained
usageValueon text-only rows — never re-parsed from display text. This is what lets a cached text-only row grow a bar the moment an entitlement is entered even when the follow-up refresh never lands (offline, token lost, 401), and it is why the row carriesusageValueat all.Which surfaces this covers
Worth being explicit, since it's a menu bar app:
codexbar usageCLICredits are detail rows, not a
RateWindow, because a row with no user-entered entitlement has no percentage to show.MenuBarLayout,IconRemainingResolverand the widget all build fromprimary/secondary/tertiary/extraRateWindows, so they see nothing.The consequence: a Business token-billing user still gets an empty menu bar icon and must open the menu to see credits. I'd rather state that than have you find it.
Making the icon work needs a decision I didn't want to make unilaterally: publishing credits as a
NamedRateWindowwhen an entitlement exists would light up the icon, widget and the existing "menu bar secondary metric" picker for free, but Copilot's extra windows are gated behindcopilotBudgetExtrasEnabledin the card renderer, so it risks either double-rendering the row or coupling credits to an unrelated setting. Happy to do it in this PR if you'll say which shape you want.Live behavior on the current head
Captured from a build of this branch against my live Copilot Business seat (token-based billing). Full sequence and caveats in this comment.
2455 / 30002455 / 30002455 / 5000, shorter fillStated plainly: the two offline frames sit either side of a brief reconnect, so they do not form one unbroken offline stretch spanning an entitlement change. The exact text↔bar transition is covered by unit tests rather than by these captures — see
UsageStoreCoverageTests.Testing
Verified on the current head against
e915ca13f:make check— 0 violations, 0 serious across 1,878 files.ProviderArchitectureGatekeeperTests— 38/38. The credit-entitlement additions shift 13 allowlisted provider constructs past the gatekeeper's two-line anchor tolerance, so this branch re-anchors those entries to their current lines; anchors, provider sets and fingerprints are unchanged.One pre-existing failure, unrelated to this PR:
MiniMaxMenuCardBillingTestsexpects"1,234"but gets"1.234"(locale-dependent number grouping; this machine isen_PL). It reproduces identically on a pristineorigin/mainworktree with none of these changes applied. Note that a plainmake testaborts on it rather than reporting and continuing — that is why the evidence above is sharded.All new tests use
ProviderHTTPTransportStubor the existingURLProtocolstub. No live network, no Keychain access, per AGENTS.md.Regression coverage worth calling out: a test asserts the
#1258behaviour still holds — a Business token-billing payload yields credits andprimary == nil, secondary == nil. Per-account tests cover: account override wins over the global fallback, fallback when the account has no value, two accounts producing different snapshot entitlements, and a Codable round-trip through the config store (including legacy JSON without the new keys). Cached-repair tests cover text→bar transition, clear-restores-text-only, no-numeric-usage no-op, and the stale-reset-baseline path.Known limitations
L("AI credits")andL("credits used")have noLocalizable.stringsentries.Scripts/check-app-locales.mjsenforces all 23 catalogs together, so a partial addition failsmake check— left out deliberately rather than done badly. Happy to add all 23 here if you'd prefer.resetsAtis available and a monthly credit budget is a good fit for it (cf. Show ahead/behind pace for all providers and in the menu bar #807, Add Projections to Copilot Tracker #2166).Commits
12 focused commits, each with its own tests, left unsquashed so they can be reviewed in sequence. Squash on merge if you'd rather.
🤖 Generated with Claude Code