fix(pricing): drop the unpublished cached rate from the Gemini Live preview entries

[devin-ai-integration]

TLDR

Problem this solves:

• Two Gemini Live preview entries price a cached read Google does not sell
• Both claim supports_prompt_caching: true for a model whose card says not supported

How it solves it:

• Drop cache_read_input_token_cost and set supports_prompt_caching: false on both preview entries
• Unit tests pin the family invariant, the cached-token bill and the capability flag
• Integration test proves /spend/calculate bills 100k cached tokens like fresh ones on both ids

This supersedes #40018, which sits on a fork no maintainer credential can push to and was based on the stale litellm_internal_staging. The pricing and capability changes are the contributor's own commits, carried over with authorship intact and credited as co-author on the test commit. Relative to that PR, the GA gemini-live-2.5-flash-native-audio correction had already landed on main, and azure/gpt-realtime-2 was removed from the map on main as past its deprecation date, so both parts dropped out and only the two preview entries change. The vendor reads backing the values (Vertex pricing table printing N/A in both cached columns for every Live row, the context-caching page listing no Live model, the model card marking caching not supported) are reproduced in the original PR's description

User Flow

Before: a developer planning around the Gemini Live preview model reads the gateway catalog and is told a cache exists that it never gets

1. They read GET https://litellm-domain/v1/model/info and the gemini-live-2.5-flash-preview-native-audio-09-2025 entry reports supports_prompt_caching: true and cache_read_input_token_cost: 7.5e-08
2. They restructure their prompts around a shared prefix to earn the cached discount, and no session ever reports cached tokens back
3. They check what a cached read would cost: POST https://litellm-domain/spend/calculate with 100,000 cached and 1,000 fresh input tokens returns {"cost":0.008}, charging the cached tokens at $0.075 per 1M
4. Nothing reachable from the gateway tells them this model has no cache

After: the same catalog reports what Google documents, so the developer never chases a cache that does not exist

1. The same GET https://litellm-domain/v1/model/info now reports supports_prompt_caching: false and no cached rate
2. They skip the shared-prefix restructuring, because the catalog says the model has no prompt caching
3. The same POST https://litellm-domain/spend/calculate with 100,000 cached and 1,000 fresh input tokens returns {"cost":0.0505}, every input token at the fresh $0.50 per 1M
4. The catalog and the bill match Google's own card for that model id

Relevant issues

Supersedes #40018

Affected release

Linear ticket

Resolves LIT-7110

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

tests/test_litellm/test_cost_calculator.py carries the contributor's tests reworked onto main's layout: the gpt-realtime-2 family invariant that a cached audio read implies an equal cached audio write, a cached count on the preview entries leaving the bill at the fresh-input total, and supports_prompt_caching answering false for the Live entries while still answering true for gemini-2.5-flash so a swallowed lookup error cannot read as a pass. tests/integration/spend/test_spend_calculate.py::test_live_preview_entry_charges_cached_tokens_at_the_fresh_rate runs the same check through a real proxy for both the bare and gemini/ ids, asserting equal cost with 100k cached and 0 cached tokens. Re-adding cache_read_input_token_cost: 7.5e-08 to the bare preview entry turns that test red (0.008 vs 0.0505)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Proxy started on the local cost map with LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config proof_config.yaml --port 4111, where proof_config.yaml lists gemini/gemini-live-2.5-flash-preview-native-audio-09-2025 with a dummy key. /spend/calculate is a pure catalog lookup, so no provider call is involved

Before (merge base b0ac23d, origin/main)

$ curl -s 'http://localhost:4111/v1/model/info' -H 'Authorization: Bearer sk-proof-1234' | jq '.data[] | .model_info | {supports_prompt_caching, cache_read_input_token_cost}'
{
  "supports_prompt_caching": true,
  "cache_read_input_token_cost": 7.5e-08
}

$ curl -s http://localhost:4111/spend/calculate -H 'Authorization: Bearer sk-proof-1234' -H 'Content-Type: application/json' -d '{"completion_response":{"model":"gemini/gemini-live-2.5-flash-preview-native-audio-09-2025","choices":[{"index":0,"message":{"role":"assistant","content":"x"},"finish_reason":"stop"}],"usage":{"prompt_tokens":101000,"completion_tokens":0,"total_tokens":101000,"prompt_tokens_details":{"cached_tokens":100000}}}}'
{"cost":0.008}

After (tip fd57f3c, litellm_realtime_cost_map_residuals)

$ curl -s 'http://localhost:4111/v1/model/info' -H 'Authorization: Bearer sk-proof-1234' | jq '.data[] | .model_info | {supports_prompt_caching, cache_read_input_token_cost}'
{
  "supports_prompt_caching": false,
  "cache_read_input_token_cost": null
}

$ curl -s http://localhost:4111/spend/calculate -H 'Authorization: Bearer sk-proof-1234' -H 'Content-Type: application/json' -d '{"completion_response":{"model":"gemini/gemini-live-2.5-flash-preview-native-audio-09-2025","choices":[{"index":0,"message":{"role":"assistant","content":"x"},"finish_reason":"stop"}],"usage":{"prompt_tokens":101000,"completion_tokens":0,"total_tokens":101000,"prompt_tokens_details":{"cached_tokens":100000}}}}'
{"cost":0.050499999999999996}

After: the stale 7.5e-08 cache-read rate is gone and the entry no longer advertises prompt caching, so 100k cached + 1k fresh tokens bill at the base input rate (101000 * 5e-07 = 0.0505) instead of 0.008

Caveats

The two preview entries cite the Gemini API pricing page rather than the Vertex one, so their context windows are left alone rather than assumed to match the Vertex model card. Nothing bills off supports_prompt_caching; both readers gate on is True, so the flag change only corrects what get_model_info reports

Link to Devin session: https://app.devin.ai/sessions/4a9380d932644b428478a7eebbd41b57
Open in Devin Desktop: https://app.devin.ai/desktop/session/4a9380d932644b428478a7eebbd41b57?variant=devin

---

Note

Medium Risk
Changes spend and model-info behavior for two realtime models; corrected billing may increase reported costs where cached tokens were previously underpriced.

Overview
Corrects Gemini Live preview native-audio catalog entries (gemini-live-2.5-flash-preview-native-audio-09-2025 and the gemini/ alias) so they match Google’s docs: cache_read_input_token_cost is removed and supports_prompt_caching is set to false in both model_prices_and_context_window.json and its backup.

Without a cached read rate, 100k “cached” prompt tokens bill at the fresh input rate instead of the old phantom discount (e.g. spend/calculate moves from ~$0.008 to ~$0.0505 for the example usage). Unit tests lock in the Live native-audio family (no cached rate, equal cost with/without cached_tokens, caching unsupported vs gemini-2.5-flash control) plus a gpt-realtime-2 audio cache read/write parity check. An integration test on /spend/calculate covers both model ids.

^{Reviewed by Cursor Bugbot for commit 790500e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ marty-sullivan
❌ kerry-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 5/5

The pull request appears safe to merge; the current metadata and tests consistently enforce fresh-rate billing for these non-caching preview models.

Summary

Corrects the Gemini Live preview pricing metadata so unsupported prompt caching is no longer advertised or billed at an unpublished discounted rate.

• Removes cache_read_input_token_cost from both preview model entries in the primary and backup pricing maps.
• Sets supports_prompt_caching to false for both entries.
• Adds unit and proxy integration coverage confirming cached-token counts are charged at the fresh-input rate.
• The changes since the previous review replace unnecessary test docstrings with dated vendor-source comments and strengthen the billing assertion by comparing equivalent cached and fresh requests.

_{Reviews (2) · Last reviewed commit: "test(cost): cite dated sources for Live ..."}

[cursor]

Stale Bugbot comment from a previous run.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_realtime_cost_map_residuals (790500e) with main (b395bfe)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 790500e. Configure here.}