Skip to content

Add China API models.dev pricing lookup - #2376

Open
joeVenner wants to merge 9 commits into
steipete:mainfrom
joeVenner:codex/modelsdev-china-api-models
Open

Add China API models.dev pricing lookup#2376
joeVenner wants to merge 9 commits into
steipete:mainfrom
joeVenner:codex/modelsdev-china-api-models

Conversation

@joeVenner

@joeVenner joeVenner commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add provider-aware models.dev pricing lookup for China API provider families.
  • Route Alibaba coding plans to alibaba-coding-plan/alibaba-coding-plan-cn, Alibaba token plans to alibaba-token-plan/alibaba-token-plan-cn, z.ai to zai/zai-coding-plan, and DeepSeek to deepseek.
  • Wire those provider-scoped lookups into the Pi session runtime cost path and expose the supported providers through codexbar cost.

Proof

  • Previous-context check: PR Improve China Kimi/GLM routes: Coding Plan 5h, Kimi Monthly, clearer labels #2351 covers Kimi/GLM quota routing, not these models.dev provider pricing mappings. PR feat(providers): add Qwen Cloud individual token-plan provider #2361 adds Qwen Cloud token-plan usage, not Qwen model pricing lookup.
  • Absence check against current base returned no matches for the IDs used here:
    git grep -n -E "glm-5\.1|glm-5v-turbo|qwen3\.7-max|deepseek-v4-pro" origin/main -- Sources Tests docs
  • Source catalog check from https://models.dev/api.json showed these live provider keys: alibaba, alibaba-cn, alibaba-coding-plan, alibaba-coding-plan-cn, alibaba-token-plan, alibaba-token-plan-cn, deepseek, zai, zai-coding-plan, zhipuai, zhipuai-coding-plan.
  • Live catalog spot checks showed:
    • alibaba-coding-plan: qwen3.7-max / input 2.5 / output 7.5 / cache_read 0.5 / cache_write 3.125
    • alibaba-token-plan: qwen3.7-plus / input 0 / output 0 / cache_read 0 / cache_write 0
    • zai: glm-5v-turbo / input 1.2 / output 4 / cache_read 0.24 / cache_write 0
    • zhipuai: glm-5v-turbo / input 5 / output 22 / cache_read 1.2 / cache_write 0, intentionally not used for z.ai routing
    • deepseek: deepseek-v4-pro / input 0.435 / output 0.87 / cache_read 0.003625 / context 1000000
  • Runtime regression test uses a cached models.dev fixture with deliberately divergent fallback entries, then verifies PiSessionCostScanner.loadDailyReport computes provider-scoped costUSD for Alibaba coding plan, Alibaba token plan, z.ai, and DeepSeek Pi JSONL rows.
  • Built CLI proof used an isolated temp home with a synthetic .pi/agent/sessions/2026-07-28T10-00-00-000Z_china-proof.jsonl containing only assistant rows. No real account files, Keychain, or browser cookies were read. Command shape:
    HOME=[temp] CFFIXED_USER_HOME=[temp] XDG_CONFIG_HOME=[temp]/.config XDG_CACHE_HOME=[temp]/.cache .build/debug/CodexBarCLI cost --provider <provider> --format json --pretty --refresh
  • Built CLI output excerpts:
    { "provider" : "alibaba", "source" : "local", "sessionTokens" : 150000, "sessionCostUSD" : 0.625,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "qwen3.7-max", "totalTokens" : 150000, "cost" : 0.625 }] }] }
    { "provider" : "alibabatokenplan", "source" : "local", "sessionTokens" : 150000, "sessionCostUSD" : 0,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "qwen3.7-plus", "totalTokens" : 150000, "cost" : 0 }] }] }
    { "provider" : "zai", "source" : "local", "sessionTokens" : 180000, "sessionCostUSD" : 0.3224,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "glm-5v-turbo", "totalTokens" : 180000, "cost" : 0.3224 }] }] }
    { "provider" : "deepseek", "source" : "local", "sessionTokens" : 160000, "sessionCostUSD" : 0.08703625,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "deepseek-v4-pro", "totalTokens" : 160000, "cost" : 0.08703625 }] }] }
    The Alibaba token-plan zero is expected from the current public models.dev catalog for alibaba-token-plan/qwen3.7-plus.

Verification

  • CLANG_MODULE_CACHE_PATH=/private/tmp/codexbar-clang-cache swift test --disable-sandbox --scratch-path /Users/mosaab/Documents/Projects/CodexBar/.build --filter PiSessionCostScannerTests: 19 tests passed.
  • CLANG_MODULE_CACHE_PATH=/private/tmp/codexbar-clang-cache swift test --disable-sandbox --scratch-path /Users/mosaab/Documents/Projects/CodexBar/.build --filter 'provider lookup resolves current China API models': 1 test passed.
  • git diff --check: clean.
  • make check completed generated parser hash, package/docs/locales checks, SwiftFormat lint mode, and SwiftLint strict with 0 violations, then exited on the sandbox-only macOS plist cache write permission error after lint shutdown.

No live provider probes, browser-cookie imports, or Keychain reads were run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac458da2f6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift Outdated
Comment thread Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift Outdated
@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Jul 27, 2026
@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codex review: needs changes before merge. Reviewed August 15, 2026, 12:20 PM ET / 16:20 UTC.

ClawSweeper review

What this changes

The PR adds models.dev-backed Pi-session cost estimates and codexbar cost support for Alibaba plans, z.ai, and DeepSeek.

Merge readiness

⚠️ Needs maintainer review before merge - 3 items remain

Keep open: the PR remains useful, but its catalog selection ignores the configured regional provider profile, so China-region Alibaba and BigModel CN z.ai users can receive incorrect cost estimates.

Priority: P2
Reviewed head: c79a442cf44e9edf512c8167216ce9e04808d819

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The PR has credible built-CLI proof and focused tests, but a P1 regional-pricing correctness flaw remains.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (live_output): The PR body provides after-fix output from the built CLI against isolated synthetic Pi sessions; it demonstrates the added local cost paths, though it does not cover the missing regional selection.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The PR body provides after-fix output from the built CLI against isolated synthetic Pi sessions; it demonstrates the added local cost paths, though it does not cover the missing regional selection.
Evidence reviewed 5 items PR catalog selection: The new pricing helper always tries the international Alibaba and global z.ai catalog IDs before regional alternatives; it never accepts provider region as input.
Existing regional contract: Current main persists selected regions for Alibaba Coding Plan and z.ai, including cn and bigmodel-cn, but the PR's cost path receives only UsageProvider.
Missing current-main implementation: Current main has no China-plan catalog IDs or provider-aware models.dev pricing helper; this capability has not already landed.
Findings 1 actionable finding [P1] Select models.dev catalog from the configured region
Security None None.

How this fits together

CodexBar reads local Pi session JSONL files and combines token counts with cached models.dev pricing to produce menu and CLI cost reports. Provider configuration determines which regional service endpoint a profile represents, so pricing must use the matching catalog.

flowchart LR
A[Pi session files] --> B[Cost scanner]
C[Provider profile region] --> D[Catalog selection]
E[models.dev cache] --> D
D --> B
B --> F[Daily cost report]
F --> G[Menu and CLI cost output]
Loading

Before merge

  • Select models.dev catalog from the configured region (P1) - These fixed arrays always choose the global catalog first, while the existing Alibaba and z.ai profiles persist cn/bigmodel-cn regions. A China-profile user therefore receives global pricing whenever model IDs overlap; pass the configured region through this lookup and include it in Pi cache invalidation, with divergent-region fixtures. This is the still-unfixed P1 from the previous review.
  • Resolve merge risk (P1) - Merging would expose cost totals that can use global rates for configured China-region Alibaba or BigModel CN z.ai profiles; cached Pi costs also need to be invalidated or recomputed when that selection changes.
  • Complete next step (P2) - A narrow, mechanical repair can thread existing region configuration into catalog selection and cache identity without choosing new product policy.

Findings

  • [P1] Select models.dev catalog from the configured region — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:794-799
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Diff footprint production +203/-33; tests +626/-373 across 11 files The provider feature is small in runtime surface but includes substantial gatekeeper-fixture churn that merits focused review.

Merge-risk options

Maintainer options:

  1. Honor configured regions before merge (recommended)
    Thread Alibaba and z.ai region selection into pricing lookup and Pi-cost cache identity, then add divergent-catalog coverage for global and China profiles.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Thread the configured Alibaba and z.ai region into models.dev catalog selection, invalidate or recompute cached Pi pricing when it changes, and add divergent-region regression coverage.

Technical review

Best possible solution:

Make cost estimates follow the already-configured provider region and preserve correct estimates across region changes.

Do we have a high-confidence way to reproduce the issue?

Yes—source reproducible: configure an Alibaba China or z.ai BigModel CN profile with catalog entries that differ from global rates; the new lookup accepts only the provider and always uses its fixed global-first order.

Is this the best way to solve the issue?

No: the provider-scoped lookup is the right seam, but it must receive the existing regional profile selection and make cached Pi prices sensitive to it.

Full review comments:

  • [P1] Select models.dev catalog from the configured region — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:794-799
    These fixed arrays always choose the global catalog first, while the existing Alibaba and z.ai profiles persist cn/bigmodel-cn regions. A China-profile user therefore receives global pricing whenever model IDs overlap; pass the configured region through this lookup and include it in Pi cache invalidation, with divergent-region fixtures. This is the still-unfixed P1 from the previous review.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f15f142a7787.

Labels

Label justifications:

  • P2: The PR adds bounded provider cost-reporting capability, but its current regional pricing defect prevents a reliable merge.
  • merge-risk: 🚨 compatibility: Existing Alibaba and z.ai regional profile settings would not control the new estimates, producing wrong costs for configured China variants.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (live_output): The PR body provides after-fix output from the built CLI against isolated synthetic Pi sessions; it demonstrates the added local cost paths, though it does not cover the missing regional selection.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides after-fix output from the built CLI against isolated synthetic Pi sessions; it demonstrates the added local cost paths, though it does not cover the missing regional selection.

Evidence

Acceptance criteria:

  • [P1] swift test --filter 'provider lookup resolves current China API models'.
  • [P1] swift test --filter 'pi scanner prices China provider sessions with provider scoped models dev catalogs'.
  • [P1] make check.
  • [P1] make test.

What I checked:

Likely related people:

  • steipete: Current-main blame attributes the regional provider configuration snapshot to Peter Steinberger, and the owner gave the concrete review direction for this overlapping pricing series. (role: recent provider-area reviewer; confidence: medium; commits: 0e453c4a5b2a; files: Sources/CodexBarCore/Providers/Alibaba/AlibabaCodingPlanProviderDescriptor.swift, Sources/CodexBarCore/Providers/Zai/ZaiProviderDescriptor.swift)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Add divergent global-versus-China catalog fixtures through the actual configured-profile cost path.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (31 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-08T13:48:39.921Z sha 740a86b :: found issues before merge. :: [P1] Honor the configured pricing region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T16:55:47.392Z sha 740a86b :: needs changes before merge. :: [P1] Route pricing by the configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T22:59:22.842Z sha 740a86b :: needs changes before merge. :: [P1] Select pricing catalog from configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T05:44:46.604Z sha 740a86b :: found issues before merge. :: [P1] Route price catalogs by configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T14:32:38.447Z sha 5ce6ad6 :: needs changes before merge. :: [P1] Route price catalogs by configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T22:38:22.265Z sha a3b98f8 :: needs changes before merge. :: [P1] Select the pricing catalog from the configured region
  • reviewed 2026-08-12T09:14:43.939Z sha a3b98f8 :: found issues before merge. :: [P1] Route price selection through the configured region
  • reviewed 2026-08-15T15:26:48.488Z sha 63704d9 :: needs changes before merge. :: [P1] Select the catalog from the configured region

@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 28, 2026
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 68c556c to bccd82b Compare July 31, 2026 17:24
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 31, 2026
@steipete

steipete commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Thanks for the models.dev pricing series! The China API provider mappings (Alibaba, z.ai, DeepSeek) here haven't landed on main (the scanner still only maps anthropic/openai), so this is still valuable — but the branch has been merge-dirty since July 21, and #2374/#2376/#2377 all touch the same files (PiSessionCostScanner.swift, CostUsagePricing.swift, CLICostCommand.swift), so they'll conflict with each other as well as with main. Could you rebase within the next two weeks, ideally picking one PR to land first and stacking the other two on top afterwards?

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Aug 7, 2026
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 740a86b to 5ce6ad6 Compare August 9, 2026 14:28
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Aug 9, 2026
@joeVenner

Copy link
Copy Markdown
Contributor Author

@steipete — this PR has been rebased onto latest main and is ready for review. Thanks!

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Aug 12, 2026
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from a3b98f8 to 63704d9 Compare August 15, 2026 15:20
…DevCostUSD out of enum to satisfy lint limits
ClawSweeper flagged that .alibaba and .alibabatokenplan fell back to the direct 'alibaba' catalog when their dedicated plan catalogs lacked a model. Plan billing can differ from direct API billing, so return no estimate instead. Add regression coverage for missing plan-catalog rows.
After rebasing the China API models.dev pricing work, ProviderArchitectureGatekeeperTests failed because new provider-specific branches (Alibaba, Zai, DeepSeek) appeared in shared cost code and existing allowlist anchors shifted. Add '// Provider-specific by design:' markers at each provider-owned dispatch point, update the gatekeeper's hardcoded supportsTokenSnapshot set and shifted suppressed references, and remove obsolete CostUsageFetcher/PiSessionCostScanner allowlist entries whose anchors no longer match.

Closes provider-architecture gatekeeper failures on the China API rebase.
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 63704d9 to c79a442 Compare August 15, 2026 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants