-
Couldn't load subscription status.
- Fork 2.4k
fix: apply tiered pricing for Gemini models via Vertex AI #8018
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Modified calculateCost method to handle models where cacheReadsPrice is only defined in tiers - Added comprehensive tests for Vertex AI tiered pricing calculation - Fixes issue where local cost calculation always showed highest tier rates Fixes #8017
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reviewed my own code and found it surprisingly coherent. Must be a bug in the review process.
Review Summary
The fix correctly implements tiered pricing for Gemini models via Vertex AI. The logic properly handles models with and without tiers, and gracefully handles missing cache prices.
Suggestions for improvement:
-
src/api/providers/gemini.ts:313 - When
cacheReadsPriceis undefined, we're setting it to 0. Consider adding a comment explaining this is intentional for models without cache support. -
src/api/providers/gemini.ts:297 - The tier selection uses
find()which returns the first matching tier. Consider adding a comment clarifying that tiers must be ordered from lowest to highest context window. -
src/api/providers/tests/vertex-tiered-pricing.spec.ts - Consider adding a test case for when
cacheReadTokensis provided butcacheReadsPriceis undefined to explicitly test the fallback behavior.
What works well:
- Comprehensive test coverage that validates both tiered and flat pricing scenarios
- The test at line 35-37 effectively validates that tier 1 pricing is used for tokens under 200K, directly addressing the reported bug
- Proper handling of edge cases where prices might only be defined in tiers
Overall, this is a solid fix that addresses the issue reported in #8017.
Summary
This PR fixes the local cost calculation for Gemini models when used through Vertex AI, ensuring that tiered pricing and cache discounts are properly applied.
Problem
When using Gemini models through Vertex AI, the local cost calculation was always showing the highest tier rates instead of applying the correct tiered pricing based on token usage. This was causing discrepancies between the displayed costs in Roo-Code and the actual billing from Google Cloud.
Solution
Modified the
calculateCostmethod inGeminiHandlerto:cacheReadsPriceis only defined within tiers (not at the top level)Changes
src/api/providers/gemini.ts: UpdatedcalculateCostmethod to handle tiered pricing correctlysrc/api/providers/__tests__/vertex-tiered-pricing.spec.ts: Comprehensive test suite for Vertex AI tiered pricingTesting
Impact
This fix ensures that users see accurate cost estimates in their local logs and UI when using Gemini models through Vertex AI. The actual billing by Google Cloud was already correct - this only affects the local display.
Fixes #8017
Important
Fixes local cost calculation for Gemini models via Vertex AI by implementing tiered pricing in
gemini.tsand adding comprehensive tests.cacheReadsPricedefined only within tiers.calculateCostinGeminiHandleringemini.tsto handle tiered pricing.vertex-tiered-pricing.spec.tsfor testing tiered pricing scenarios.This description was created by
for d53ece6. You can customize this summary. It will automatically update as commits are pushed.