Avoid fabricated costs for unknown models - #429
Merged
Merged
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThe change adds known-only pricing APIs, preserves token usage when pricing is unavailable, and records standardized data-quality notes in scanner and viewer results. ChangesPricing quality
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant SessionScanner
participant PricingHelpers
participant ScanResult
participant DataQualityIndicator
SessionScanner->>PricingHelpers: Estimate cost from token usage
PricingHelpers-->>SessionScanner: Known cost or undefined
SessionScanner->>ScanResult: Preserve usage and add quality note
ScanResult->>DataQualityIndicator: Provide model and cost metadata
DataQualityIndicator-->>ScanResult: Display unavailable-cost note
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/replay-core/src/pricing.ts`:
- Around line 107-121: Reject versioned future major model IDs before the
generic family fallbacks in the pricing resolver, while preserving supported and
legacy Claude 3/4 IDs; update the version parsing or allowlist around the Opus,
Sonnet, and Haiku handling in packages/replay-core/src/pricing.ts:107-121. Add
assertions covering claude-opus-5, claude-sonnet-5, and claude-haiku-5 in
packages/replay-core/test/pricing.test.ts:107-114, expecting undefined pricing.
In `@packages/viewer/src/components/DataQualityIndicator.tsx`:
- Around line 36-37: Update DataQualityIndicator so the unavailable-cost note is
added whenever token usage exists and meta.stats.costEstimate is undefined,
regardless of whether meta.model is present. Add a corresponding test case in
packages/viewer/src/components/__tests__/data-quality-indicator.test.ts lines
56-64 covering token usage with no model and no cost estimate.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 18fe41d0-afe1-4fff-9363-d280966a20d0
📒 Files selected for processing (8)
packages/cli/src/scanner.tspackages/cli/test/scanner.test.tspackages/replay-core/src/pricing.tspackages/replay-core/src/transform.tspackages/replay-core/test/pricing.test.tspackages/replay-core/test/transform-comprehensive.test.tspackages/viewer/src/components/DataQualityIndicator.tsxpackages/viewer/src/components/__tests__/data-quality-indicator.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/replay-core/src/pricing.ts`:
- Around line 106-124: Update the Claude pricing selection logic around
hasUnsupportedClaudeMajor and the Opus, Sonnet, and Haiku branches to parse
complete version tokens in both family-first and major-first naming forms.
Reject any minor version greater than 6 before matching supported 4-5/4-6
pricing, using token-bounded matches so values such as 4-60 are not treated as
4-6. Add regression tests covering unsupported minors and both identifier
formats for each Claude family.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 22aa4267-fee3-451b-b73e-063a2082aa8d
📒 Files selected for processing (5)
packages/cli/src/scanner.tspackages/replay-core/src/pricing.tspackages/replay-core/test/pricing.test.tspackages/viewer/src/components/DataQualityIndicator.tsxpackages/viewer/src/components/__tests__/data-quality-indicator.test.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- packages/viewer/src/components/DataQualityIndicator.tsx
- packages/replay-core/test/pricing.test.ts
- packages/viewer/src/components/tests/data-quality-indicator.test.ts
- packages/cli/src/scanner.ts
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/replay-core/src/pricing.ts`:
- Around line 141-145: Update the version parsing logic around the minorToken
handling to distinguish dated release suffixes from numeric minor versions,
rejecting numeric minor values above 6 rather than silently omitting them.
Ensure inputs such as 4-100 and another three-digit minor value are rejected and
cannot fall through to legacy Opus pricing, while preserving valid minor-version
and dated-suffix behavior; add regression coverage for both cases.
- Around line 112-117: Update the Sonnet pricing branch in the
version-resolution logic to use sonnetVersion?.major === 4 after the existing
4.5/4.6 special case, so major-first IDs such as claude-4-1-sonnet select
MODEL_PRICING["sonnet-4"] instead of the generic sonnet pricing.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 3df04a32-ff7b-40df-a5ac-030890473418
📒 Files selected for processing (2)
packages/replay-core/src/pricing.tspackages/replay-core/test/pricing.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/replay-core/test/pricing.test.ts
Comment on lines
+141
to
+145
| const minorToken = match[2]; | ||
| return { | ||
| major: Number(match[1]), | ||
| ...(minorToken && minorToken.length <= 2 ? { minor: Number(minorToken) } : {}), | ||
| }; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reject numeric minor versions with three or more digits.
Line 144 discards a numeric minor token when it has more than two digits. claude-opus-4-100 then has no minor version, bypasses Line 107, and receives legacy Opus pricing at Line 110.
Distinguish dated release suffixes from numeric minor versions. Reject every numeric minor version above 6. Add regression coverage for 4-100 and another three-digit minor value.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/replay-core/src/pricing.ts` around lines 141 - 145, Update the
version parsing logic around the minorToken handling to distinguish dated
release suffixes from numeric minor versions, rejecting numeric minor values
above 6 rather than silently omitting them. Ensure inputs such as 4-100 and
another three-digit minor value are rejected and cannot fall through to legacy
Opus pricing, while preserving valid minor-version and dated-suffix behavior;
add regression coverage for both cases.
Owner
Author
|
Final validation complete:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
Unknown OpenCode, Hermes, local, and future model IDs were silently priced at Sonnet rates. This produced a precise-looking dollar value with no evidence that the rate was applicable.
Compatibility
getModelPricing,estimateCost, andestimateCostSimplebehavior is unchangedValidation
pnpm lint:checkpnpm typecheckpnpm testpnpm --filter @vibe-replay/provider-hermes testpnpm test:cloudflarepnpm buildSummary by CodeRabbit