Skip to content

Fix Claude vendor pricing after #3002 - #3012

Merged
steipete merged 7 commits into
mainfrom
fix/3002-claude-vendor-pricing
Aug 17, 2026
Merged

Fix Claude vendor pricing after #3002#3012
steipete merged 7 commits into
mainfrom
fix/3002-claude-vendor-pricing

Conversation

@steipete

Copy link
Copy Markdown
Owner

Summary

This supersedes #3002 while preserving all contributor commits and credit to @Yuxin-Qiao.

  • merge the first-party Claude bare-model pricing work onto current main
  • keep recognizable model families in their owning vendor catalog, with Claude-family IDs Anthropic-first
  • preserve approved provider-qualified routes and leave unknown prefixes or ambiguous cross-vendor bare IDs unpriced
  • make unknown-price refresh and Pi pricing fingerprints use the same vendor set
  • scope custom Codex session roots to their matching trace database instead of borrowing ambient account state

Root causes

The original CI failures came from generated and line-sensitive metadata drifting as the pricing paths changed: the parser fingerprint was stale, gatekeeper anchors shifted, and the Pi unrelated-provider fixture used a catalog that became relevant to Claude pricing. The contributor fixed those on the original head, but merging with #2946 changed the same pricing file again and re-staled the parser fingerprint.

The clean textual merge also hid a semantic conflict: #2946 requires explicit routes and unknown prefixes to stay isolated, while a fixed bare-ID vendor fallback could choose the wrong first-party catalog when names collide. This branch reconciles those rules with deterministic family ownership and fail-closed ambiguity handling.

The full-suite run additionally exposed a pre-existing test isolation bug: custom Codex session roots still scanned the ambient state_5.sqlite, causing the macOS shard to time out on a large local database. The scoped trace-database fix reduced that suite from 93 seconds to 3.5 seconds.

Proof

  • swiftlint --strict --quiet
  • make check
  • swift test --filter 'CostUsagePricing|CostUsageFetcher|ProviderArchitectureGatekeeper'
  • swift test --filter PiSessionCostScannerTests
  • swift test --filter CodexLocalProjectUsageTests
  • make test: 883 selections in 74 groups; all passed on the first attempt
  • isolated CLI fixture: bare deepseek-v4-flash priced 110 tokens at USD 0.0000168; unknown-route/deepseek-v4-flash remained present and unpriced
  • final autoreview: no actionable findings

This PR does not merge or close #3002.

Yuxin-Qiao and others added 7 commits August 17, 2026 14:40
Co-authored-by: Cursor <cursoragent@cursor.com>
Bare DeepSeek IDs now invalidate Pi pricing caches and trigger models.dev refresh on the same vendor list used for lookup.

Co-authored-by: Cursor <cursoragent@cursor.com>
The pricing-file change left CodexParserHash.generated.swift stale and failed lint.

Co-authored-by: Cursor <cursoragent@cursor.com>
…back.

Google is now a Claude first-party pricing vendor, so the unrelated-catalog case uses Groq instead.

Co-authored-by: Cursor <cursoragent@cursor.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cde7312168

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +892 to +894
private static func claudeModelsDevModelIDs(for rawModel: String) -> [String] {
let normalized = self.normalizeClaudeModel(rawModel)
return normalized == rawModel ? [rawModel] : [rawModel, normalized]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply OpenAI alias normalization to bare Claude routes

When a Claude transcript contains the recognizable bare OpenAI alias gpt-5.6, this helper applies only normalizeClaudeModel, so it searches the OpenAI catalog for gpt-5.6 but not the canonical gpt-5.6-sol. A catalog containing only the canonical entry is already a supported case in CostUsagePricingTests, and this new Claude routing path consequently leaves that usage unpriced even though the equivalent Codex lookup resolves the alias. Use the owning vendor's normalization when constructing targets for recognized non-Claude families.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Aug 17, 2026
@clawsweeper

clawsweeper Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codex review: needs changes before merge. Reviewed August 17, 2026, 11:50 AM ET / 15:50 UTC.

ClawSweeper review

What this changes

The PR routes bare Claude-session model IDs to first-party models.dev catalogs, preserves explicit provider routes, refreshes matching pricing caches, and scopes custom Codex session roots to their own trace database.

Merge readiness

⚠️ Needs maintainer review before merge - 3 items remain

Keep this owner-authored PR open. The Claude vendor-routing work is still needed, but the new OpenAI route omits existing alias normalization and can leave valid gpt-5.6 Claude-session usage unpriced.

Priority: P2
Reviewed head: cde73121685b3669d1655ccbeb750d75d4a62211

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The PR has credible runtime evidence and focused coverage, but the OpenAI alias gap prevents the intended Claude vendor pricing from being complete.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (live_output): The PR body includes after-fix isolated CLI output showing a bare vendor model priced and an unknown route left unpriced; the remaining alias case is a functional review defect, not missing proof.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The PR body includes after-fix isolated CLI output showing a bare vendor model priced and an unknown route left unpriced; the remaining alias case is a functional review defect, not missing proof.
Evidence reviewed 5 items Current main behavior: Current main limits Claude pricing lookup to the Anthropic catalog, so the first-party vendor routing remains a meaningful unmerged change.
Alias regression: The new target builder normalizes every Claude-route model only with normalizeClaudeModel, although the selected OpenAI vendor has its own gpt-5.6-to-gpt-5.6-sol normalization.
Established supported contract: Existing tests establish that an OpenAI catalog containing only gpt-5.6-sol must price the gpt-5.6 alias; the new Claude tests do not cover that route.
Findings 1 actionable finding [P2] Normalize bare OpenAI aliases before Claude catalog lookup
Security None None.

How this fits together

CodexBar scans local Claude and Codex transcripts, maps model IDs to models.dev vendor catalogs, then displays estimated spend. This PR changes the Claude mapping and cache-refresh inputs while keeping routed models from borrowing another vendor’s price.

flowchart LR
A[Local Claude transcript] --> B[Model ID routing]
B --> C[First-party vendor catalog]
C --> D[models.dev price lookup]
D --> E[Estimated spend]
A --> F[Unknown-price refresh]
F --> C
G[Scoped Codex session root] --> H[Matching trace database]
Loading

Before merge

  • Normalize bare OpenAI aliases before Claude catalog lookup (P2) - The new Claude route classifies gpt-5.6 as OpenAI but builds targets only with normalizeClaudeModel, which leaves that alias unchanged. The existing canonical-only OpenAI catalog case resolves it as gpt-5.6-sol; a Claude transcript with the same alias will therefore remain unpriced. Use the selected vendor's normalization and add that regression case.
  • Resolve merge risk (P1) - Merging would retain a compatibility gap: recognizable bare OpenAI aliases in Claude transcripts can remain unpriced when models.dev publishes only their canonical model ID.
  • Complete next step (P2) - The remaining P2 defect is a narrow mechanical normalization and regression-test repair on this PR branch.

Findings

  • [P2] Normalize bare OpenAI aliases before Claude catalog lookup — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:892-894
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Pricing and test surface 12 files changed; production +108, tests +314 The functional change spans price routing, cache fingerprints, local data scope, and focused regression coverage.

Root-cause cluster

Relationship: canonical
Canonical: #3012
Summary: This PR is the intended reconciliation branch for the open Claude bare-model pricing proposal and the already-merged provider-qualified route work.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Preserve OpenAI alias routing (recommended)
    Normalize recognized OpenAI Claude-route IDs with the existing OpenAI normalizer and cover the canonical-only catalog case before merge.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Normalize recognized bare OpenAI Claude-route IDs before models.dev lookup and add a canonical-only gpt-5.6-sol regression test; preserve explicit-route isolation.

Technical review

Best possible solution:

Reuse the OpenAI-specific normalization when creating lookup targets for recognized OpenAI Claude routes, and add a regression test using a canonical-only catalog while preserving explicit-route isolation.

Do we have a high-confidence way to reproduce the issue?

Yes, by using the established canonical-only OpenAI catalog fixture with a Claude-session gpt-5.6 model ID; the new route currently probes only the unnormalized ID.

Is this the best way to solve the issue?

No; the branch should apply the owning OpenAI vendor’s existing normalization before lookup, which is narrower and maintains the established alias contract.

Full review comments:

  • [P2] Normalize bare OpenAI aliases before Claude catalog lookup — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:892-894
    The new Claude route classifies gpt-5.6 as OpenAI but builds targets only with normalizeClaudeModel, which leaves that alias unchanged. The existing canonical-only OpenAI catalog case resolves it as gpt-5.6-sol; a Claude transcript with the same alias will therefore remain unpriced. Use the selected vendor's normalization and add that regression case.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 2fe9de487f8a.

Labels

Label changes:

  • add P2: The PR fixes bounded local spend-estimation behavior but does not affect core availability or data integrity.
  • add merge-risk: 🚨 compatibility: The new vendor-routing behavior can fail to price an already-supported OpenAI alias in Claude transcripts.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix isolated CLI output showing a bare vendor model priced and an unknown route left unpriced; the remaining alias case is a functional review defect, not missing proof.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes after-fix isolated CLI output showing a bare vendor model priced and an unknown route left unpriced; the remaining alias case is a functional review defect, not missing proof.

Label justifications:

  • P2: The PR fixes bounded local spend-estimation behavior but does not affect core availability or data integrity.
  • merge-risk: 🚨 compatibility: The new vendor-routing behavior can fail to price an already-supported OpenAI alias in Claude transcripts.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body includes after-fix isolated CLI output showing a bare vendor model priced and an unknown route left unpriced; the remaining alias case is a functional review defect, not missing proof.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix isolated CLI output showing a bare vendor model priced and an unknown route left unpriced; the remaining alias case is a functional review defect, not missing proof.

Evidence

Acceptance criteria:

  • [P1] swift test --filter CostUsagePricingTests.
  • [P1] make check.
  • [P1] make test.

What I checked:

Likely related people:

  • steipete: Authored the current routing reconciliation and has the strongest recent history on the pricing implementation. (role: recent area contributor; confidence: high; commits: cde73121685b, ad400726bba6; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift, Sources/CodexBarCore/CodexLocalDataScope.swift)
  • Yuxin-Qiao: Authored the preceding Claude bare-model pricing commits and the merged provider-qualified pricing work that supplies the alias and route-isolation contract. (role: original feature contributor; confidence: high; commits: e0b22be9a4fa, fbe70c3e9c65; files: Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift, Tests/CodexBarTests/CostUsagePricingTests.swift)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Apply OpenAI alias normalization for recognized Claude-route OpenAI models.
  • Add a canonical-only gpt-5.6-sol catalog test for a bare Claude-session gpt-5.6 ID.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit 92c7174 into main Aug 17, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants