Skip to content

Conversation

@roomote
Copy link

@roomote roomote bot commented Sep 16, 2025

Summary

This PR fixes the local cost calculation for Gemini models when used through Vertex AI, ensuring that tiered pricing and cache discounts are properly applied.

Problem

When using Gemini models through Vertex AI, the local cost calculation was always showing the highest tier rates instead of applying the correct tiered pricing based on token usage. This was causing discrepancies between the displayed costs in Roo-Code and the actual billing from Google Cloud.

Solution

Modified the calculateCost method in GeminiHandler to:

  • Properly handle models where cacheReadsPrice is only defined within tiers (not at the top level)
  • Apply tiered pricing based on input token count
  • Gracefully handle missing prices with appropriate fallbacks

Changes

  • Modified src/api/providers/gemini.ts: Updated calculateCost method to handle tiered pricing correctly
  • Added src/api/providers/__tests__/vertex-tiered-pricing.spec.ts: Comprehensive test suite for Vertex AI tiered pricing

Testing

  • ✅ All new tests pass for tiered pricing scenarios
  • ✅ Existing Gemini provider tests continue to pass
  • ✅ Verified correct pricing for tokens under 200K (tier 1) and over 200K (tier 2)
  • ✅ Cache read pricing correctly applied based on tier

Impact

This fix ensures that users see accurate cost estimates in their local logs and UI when using Gemini models through Vertex AI. The actual billing by Google Cloud was already correct - this only affects the local display.

Fixes #8017


Important

Fixes local cost calculation for Gemini models via Vertex AI by implementing tiered pricing in gemini.ts and adding comprehensive tests.

  • Behavior:
    • Fixes local cost calculation for Gemini models via Vertex AI to apply tiered pricing and cache discounts.
    • Handles models with cacheReadsPrice defined only within tiers.
    • Applies tiered pricing based on input token count and handles missing prices with fallbacks.
  • Code Changes:
    • Updates calculateCost in GeminiHandler in gemini.ts to handle tiered pricing.
    • Adds vertex-tiered-pricing.spec.ts for testing tiered pricing scenarios.
  • Testing:
    • New tests verify tiered pricing for tokens under and over 200K, and cache read pricing.
    • Ensures models without tiered pricing use flat rates.
    • All tests pass, ensuring accurate cost estimates in local logs and UI.

This description was created by Ellipsis for d53ece6. You can customize this summary. It will automatically update as commits are pushed.

- Modified calculateCost method to handle models where cacheReadsPrice is only defined in tiers
- Added comprehensive tests for Vertex AI tiered pricing calculation
- Fixes issue where local cost calculation always showed highest tier rates

Fixes #8017
@roomote roomote bot requested review from cte, jr and mrubens as code owners September 16, 2025 06:37
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Sep 16, 2025
Copy link
Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed my own code and found it surprisingly coherent. Must be a bug in the review process.

Review Summary

The fix correctly implements tiered pricing for Gemini models via Vertex AI. The logic properly handles models with and without tiers, and gracefully handles missing cache prices.

Suggestions for improvement:

  1. src/api/providers/gemini.ts:313 - When cacheReadsPrice is undefined, we're setting it to 0. Consider adding a comment explaining this is intentional for models without cache support.

  2. src/api/providers/gemini.ts:297 - The tier selection uses find() which returns the first matching tier. Consider adding a comment clarifying that tiers must be ordered from lowest to highest context window.

  3. src/api/providers/tests/vertex-tiered-pricing.spec.ts - Consider adding a test case for when cacheReadTokens is provided but cacheReadsPrice is undefined to explicitly test the fallback behavior.

What works well:

  • Comprehensive test coverage that validates both tiered and flat pricing scenarios
  • The test at line 35-37 effectively validates that tier 1 pricing is used for tokens under 200K, directly addressing the reported bug
  • Proper handling of edge cases where prices might only be defined in tiers

Overall, this is a solid fix that addresses the issue reported in #8017.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Sep 16, 2025
@daniel-lxs daniel-lxs moved this from Triage to PR [Needs Prelim Review] in Roo Code Roadmap Sep 16, 2025
@hannesrudolph hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Sep 16, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 17, 2025
@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Sep 17, 2025
@daniel-lxs daniel-lxs moved this from PR [Needs Prelim Review] to PR [Needs Review] in Roo Code Roadmap Sep 19, 2025
@mrubens mrubens merged commit c0e2c5e into main Sep 20, 2025
15 checks passed
@mrubens mrubens deleted the fix/gemini-vertex-tiered-pricing branch September 20, 2025 02:49
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Sep 20, 2025
@github-project-automation github-project-automation bot moved this from PR [Needs Review] to Done in Roo Code Roadmap Sep 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working lgtm This PR has been approved by a maintainer PR - Needs Review size:S This PR changes 10-29 lines, ignoring generated files.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

[BUG] Gemini: Local cost calculation ignores tiered pricing and cache discounts

5 participants