Add custom-pricing overlay above models.dev and builtin rates - #3016
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
|
🦞👀 Pull request received. I will update this pull request when review starts. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6f92c6255e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let overlay = customPricing ?? self.customPricingOverlay() | ||
| if let rates = overlay.rates(providerID: self.codexModelsDevProviderID, model: model) | ||
| ?? overlay.rates(model: model) | ||
| { |
There was a problem hiding this comment.
When cached day totals are valid but their pricing rows are absent or untrusted, makeCodexBilledDayEntry falls back to codexAggregateCostUSD, which still resolves only models.dev or bundled rates and never reaches this overlay block. Such reports therefore ignore a matching custom rate—including explicit free rates—and show the wrong spend; the aggregate path needs to consult the same overlay before its existing fallbacks.
Useful? React with 👍 / 👎.
| modelsDevCacheRoot: URL? = nil, | ||
| customPricing: CostUsageCustomPricing? = nil) -> Double? | ||
| { | ||
| let overlay = customPricing ?? self.customPricingOverlay() |
There was a problem hiding this comment.
For normal Codex reports, codexResolvedCostUSD calls this method once per usage row, so omitting customPricing causes every row to perform a filesystem check and, when the overlay exists, reread, parse, and SHA-256 hash the JSON file. Large histories can consequently incur thousands of redundant file parses during each report; load the overlay into the scan/pricing context once and pass it through.
Useful? React with 👍 / 👎.
| let dayKey = unixMs.map { | ||
| CostUsageDayRange.dayKey( | ||
| from: Date(timeIntervalSince1970: TimeInterval($0) / 1000), | ||
| calendar: range.calendar) | ||
| } ?? range.untilKey |
There was a problem hiding this comment.
If an unresolved fork has neither session timestamp—possible for legacy/incomplete cached metadata—this fallback assigns it to the query's untilKey. The same old fork is therefore reported as an unmetered request in every 7-day or 30-day window, always on that window's final day, regardless of when its file existed; use the available mtimeUnixMs as the fallback or exclude entries whose date cannot be established.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
P2
|
Codex review: needs real behavior proof before merge. Reviewed August 17, 2026, 4:12 PM ET / 20:12 UTC. ClawSweeper reviewWhat this changesThe stacked branch adds a custom-pricing JSON overlay for Codex cost estimates and carries prerequisite spend-provenance, time-zone, fork-accounting, and cache changes. Merge readiness⛔ Blocked until real behavior proof from a real setup is added - 7 items remain Keep open: the updated head still has two source-confirmed pricing errors, lacks real after-fix behavior proof, and introduces a persistent user configuration contract without maintainer product confirmation. Priority: P2 Review scores
Verification
How this fits togetherCodexBar scans local Codex sessions, resolves token prices from models.dev or bundled rates, and presents cached cost estimates in usage and spend views. This PR inserts a local JSON rate source ahead of those existing pricing sources. flowchart LR
A[Local Codex sessions] --> B[Usage scanner]
C[Custom pricing JSON] --> D[Pricing resolver]
E[Models.dev catalog] --> D
F[Bundled rates] --> D
B --> D
D --> G[Cached cost report]
G --> H[Usage and spend views]
Decision needed
Why: The branch adds a persisted configuration surface and pricing precedence not described by the current models.dev and bundled-rate documentation; maintainers must decide whether to support that contract long-term. Before merge
Findings
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Keep the documented managed pricing sources unless the owner sponsors a stable advanced override; if sponsored, narrow and document the JSON contract, preserve unknown-rate behavior, clamp malformed token classes, and prove a real local scan. Do we have a high-confidence way to reproduce the issue? Yes for the patch defects: call the pricing resolver with a matching overlay that omits an active token class, or with cached tokens above input; current source shows the wrong fallback or overbilling path. Is this the best way to solve the issue? No: the implementation needs the two narrow correctness repairs, real-scan evidence, and maintainer confirmation that the JSON file is a supported configuration interface. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against f7723d33a169. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (1 earlier review cycle)
|
Pinned calendars now reach cached Codex reads, vendor-reported costs keep their source, and shorter summaries no longer inherit snapshot-wide metered spend. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Cached day totals now honor overlay rates, including explicit free rates, without rereading custom-pricing.json for every usage row. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…imit. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
6f92c62 to
725419a
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 725419a597
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| modelsDevCacheRoot: URL? = nil, | ||
| customPricing: CostUsageCustomPricing? = nil) -> Double? | ||
| { | ||
| if let cost = (customPricing ?? self.customPricingOverlay()).estimatedCodexCostUSD( |
There was a problem hiding this comment.
When a matching custom entry omits a rate needed by the current token mix, estimatedCodexCostUSD returns nil, but this if let treats that exactly like “no overlay matched” and continues into models.dev or bundled pricing. For example, { "gpt-5.4": { "input": 2.5 } } with output tokens silently produces the standard full-model estimate even though the overlay contract says missing fields remain unknown; distinguish a missing match from a matched-but-incomplete entry and return nil for the latter.
Useful? React with 👍 / 👎.
| let cached = max(0, cachedInputTokens) | ||
| let written = max(0, cacheWriteInputTokens) | ||
| let uncachedInput = max(0, inputTokens - cached - written) |
There was a problem hiding this comment.
When a malformed or inconsistent usage row reports cached input greater than total input, this path bills every reported cached token, unlike the existing Codex pricing path that clamps cached reads to inputTokens; cache writes can likewise exceed the remaining input. Thus enabling a custom overlay can invent input and overstate the same row that bundled pricing handles defensively, so clamp cached reads to total input and writes to the remainder before computing uncachedInput.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
the
merge-risk: 🚨 compatibility
2 participants
Summary
custom-pricing.jsonas an exact-match overlay: overlay > models.dev > builtin.0as free.Stacked PR 2/5 (Wave C). Depends on #3015; review the custom-pricing commit once that lands.
Review follow-up
codexAggregateCostUSDconsults the overlay (including free0) before bundled/models.dev rates.CodexReportDayPricingContext.customPricing.CostUsagePricing+Overlay.swiftsoCostUsagePricingstays undertype_body_length.Test plan
Fixtures only. No Keychain, no live accounts, no developer Application Support overlay file.
CODEXBAR_SUPPRESS_TEST_KEYCHAIN_ACCESS=1 swift test --filter CostUsageCustomPricingTests