fix(gemini): read web search cost from model_info instead of hardcode

[Chesars]

Relevant issues

Fixes #24369

Pre-Submission checklist

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

The Gemini web search cost calculator hardcoded $0.035 per request, which is only correct for Gemini 2.x. Gemini 3.x charges $0.014 per request (pricing docs).

Read from search_context_cost_per_query in model_info (same field already used by Anthropic, OpenAI, and Perplexity) with fallback to the legacy $0.035 for models not yet in the JSON.

Also add search_context_cost_per_query to all 25 Gemini models that support web search in model_prices_and_context_window.json.

Tests added

• test_web_search_cost_from_model_info — reads $0.014 from model_info
• test_web_search_cost_legacy_fallback — falls back to $0.035
• test_web_search_cost_zero_requests — zero requests = zero cost
• test_web_search_cost_no_usage_details — missing usage = zero cost

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 22, 2026 9:48pm

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing Chesars:fix/gemini-web-search-cost (21b9c68) with main (c89496f)¹}

Footnotes

1. No successful run was found on litellm_staging_03_22_2026 (c89496f) during the generation of this report, so main (c89496f) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[greptile-apps]

Greptile Summary

This PR fixes Gemini web search cost calculation by reading search_context_cost_per_query and web_search_billing_unit from model_prices_and_context_window.json instead of hardcoding $0.035 per request. It also fixes a pre-existing bug where the non-streaming Vertex AI path was not populating web_search_requests on usage.prompt_tokens_details, and eliminates the duplicate cost logic in vertex_ai/gemini/cost_calculator.py by delegating to the shared gemini/cost_calculator.py.

• Billing mode distinction: Gemini 3.x models now charge $0.014 per individual search query (web_search_billing_unit: "per_query"), while Gemini 2.x models retain the flat $0.035 per grounded prompt (per_prompt, the default).
• JSON pricing updated: All 25 affected Gemini models in model_prices_and_context_window.json and the backup file now carry search_context_cost_per_query and, where applicable, web_search_billing_unit.
• Non-streaming path fix: _calculate_web_search_requests is now called in the non-streaming response handler (it was already called in streaming), so actual search query counts flow through to the cost calculator correctly.
• web_search_billing_unit not typed in ModelInfo: The new JSON field web_search_billing_unit is used via model_info.get(...) in the cost calculator but is not declared in the ModelInfo (or ModelInfoBase) TypedDict, unlike search_context_cost_per_query which is. Adding it to the TypedDict would give static analysis coverage consistent with how other model-info fields are handled.

Confidence Score: 4/5

• Safe to merge with one minor type-completeness issue to address
• The core logic is correct — pricing is now read from model_info with a safe fallback, billing modes are driven by JSON config rather than hardcoded Python, and the non-streaming path bug is fixed. Tests are thorough and mock-only. The only gap is that web_search_billing_unit is not declared in the ModelInfo TypedDict, creating a minor inconsistency with search_context_cost_per_query which is declared.
• model_prices_and_context_window.json — one vertex_ai-language-models alias for gemini-3-flash-preview still carries the legacy $0.035 price (noted in a prior review thread). litellm/llms/gemini/cost_calculator.py — web_search_billing_unit needs to be added to the ModelInfo TypedDict.

Important Files Changed

Filename	Overview
litellm/llms/gemini/cost_calculator.py	Refactored to read pricing and billing mode from model_info instead of hardcoding $0.035; logic is correct, but web_search_billing_unit is missing from the ModelInfo TypedDict.
litellm/llms/vertex_ai/gemini/cost_calculator.py	Simplified to delegate to the shared gemini/cost_calculator.py, eliminating duplicated hardcoded cost logic; clean refactor.
litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py	Adds the missing _calculate_web_search_requests call in the non-streaming path to populate web_search_requests on usage.prompt_tokens_details; safe since prompt_tokens_details is always initialized by _calculate_usage.
tests/test_litellm/llms/gemini/test_cost_calculator.py	Five focused unit tests covering per-query billing, per-prompt billing, default billing mode, zero requests, and missing usage details — all mocked, no network calls, correct location.
model_prices_and_context_window.json	Adds search_context_cost_per_query and web_search_billing_unit to all 25 affected Gemini models; Gemini 3.x correctly uses $0.014 with per_query, Gemini 2.x uses $0.035 with default per_prompt. One vertex_ai-language-models alias for gemini-3-flash-preview has $0.035 instead of $0.014 (noted in previous review thread).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[API Response received] --> B{Streaming?}
    B -- Yes --> C[_calculate_usage]
    B -- No --> D[_calculate_usage]
    C --> E[_calculate_web_search_requests\ngrounding_metadata]
    D --> F[_calculate_web_search_requests\ngrounding_metadata\n NEW in this PR]
    E --> G[set usage.prompt_tokens_details\n.web_search_requests]
    F --> G
    G --> H[cost_per_web_search_request\nusage, model_info]
    H --> I{search_context_cost_per_query\nin model_info?}
    I -- Yes --> J[_cost = search_context_cost_per_query\n.search_context_size_medium]
    I -- No --> K[_cost = 0.035 fallback]
    J --> L{web_search_billing_unit}
    K --> L
    L -- per_query --> M[total = _cost × n_requests]
    L -- per_prompt default --> N[total = _cost × 1\nflat fee per grounded call]

Loading

Comments Outside Diff (1)

1. litellm/llms/gemini/cost_calculator.py, line 61 (link)

   web_search_billing_unit missing from ModelInfo TypedDict

   web_search_billing_unit is read here via .get() but is not declared in the ModelInfo TypedDict, whereas the sibling field search_context_cost_per_query is declared there (in ModelInfoBase at litellm/types/utils.py:235). This inconsistency means:

   • Static type checkers (mypy/pyright) will not flag misspellings of the key.
   • IDEs won't autocomplete the field when working with ModelInfo objects.

   To be consistent with search_context_cost_per_query, web_search_billing_unit should be added to ModelInfoBase (or ModelInfo) in litellm/types/utils.py:

   web_search_billing_unit: Optional[str]  # "per_query" | "per_prompt"

_{Reviews (5): Last reviewed commit: "fix: remove web_search_billing_unit from..." | Re-trigger Greptile}