-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
feat(gemini): add gemini-3.1-flash-lite-image pricing and fix its capability fields #36113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
devin-ai-integration
wants to merge
3
commits into
litellm_internal_staging
from
litellm_gemini_3_1_flash_lite_image
Closed
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
586568f
feat(gemini): add gemini-3.1-flash-lite-image (Nano Banana 2 Lite) pr…
chiruno-9 c8941f6
fix(gemini): align gemini-3.1-flash-lite-image capabilities with mode…
mateo-berri 246dd79
test(gemini): drop inline comments from flash-lite-image pricing test
mateo-berri File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
150 changes: 150 additions & 0 deletions
150
tests/test_litellm/test_gemini_3_1_flash_lite_image_pricing.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,150 @@ | ||
| """Pricing entry for ``gemini-3.1-flash-lite-image`` (Google's Nano Banana 2 Lite). | ||
|
|
||
| Google publishes: $0.25/1M input, $1.50/1M text output, and $30/1M image-output | ||
| tokens for the Lite image model (https://cloud.google.com/vertex-ai/generative-ai/pricing). | ||
| A 1K image is ~1120 output image tokens => ~$0.0336 / image. | ||
|
|
||
| Without this entry, ``completion_cost`` raises "model isn't mapped yet" and Vertex | ||
| generateContent pass-through cost tracking silently logs $0. These tests pin the | ||
| values in both the primary price map and the ``litellm/`` backup, and verify | ||
| ``get_model_info`` / ``completion_cost`` surface them. | ||
| """ | ||
|
|
||
| import json | ||
| import os | ||
| import sys | ||
|
|
||
| sys.path.insert(0, os.path.abspath("../..")) | ||
|
|
||
| import litellm | ||
| from litellm import completion_cost | ||
| from litellm.types.utils import CompletionTokensDetailsWrapper, ModelResponse, Usage | ||
|
|
||
| VARIANTS = [ | ||
| "gemini-3.1-flash-lite-image", | ||
| "gemini/gemini-3.1-flash-lite-image", | ||
| "vertex_ai/gemini-3.1-flash-lite-image", | ||
| ] | ||
|
|
||
| EXPECTED = { | ||
| "input_cost_per_token": 2.5e-07, | ||
| "output_cost_per_token": 1.5e-06, | ||
| "output_cost_per_image_token": 3e-05, | ||
| "mode": "image_generation", | ||
| } | ||
|
|
||
| EXPECTED_CAPABILITIES = { | ||
| "max_output_tokens": 4096, | ||
| "max_tokens": 4096, | ||
| "supports_response_schema": False, | ||
| "supports_reasoning": True, | ||
| } | ||
|
|
||
| EXPECTED_PER_ROUTE = { | ||
| "gemini-3.1-flash-lite-image": { | ||
| "supports_prompt_caching": True, | ||
| "supports_function_calling": False, | ||
| }, | ||
| "vertex_ai/gemini-3.1-flash-lite-image": { | ||
| "supports_prompt_caching": True, | ||
| "supports_function_calling": False, | ||
| }, | ||
| "gemini/gemini-3.1-flash-lite-image": { | ||
| "supports_prompt_caching": False, | ||
| "supports_function_calling": True, | ||
| "input_cost_per_token_batches": 1.25e-07, | ||
| "output_cost_per_token_batches": 7.5e-07, | ||
| }, | ||
| } | ||
|
|
||
|
|
||
| def _load_json(path: str) -> dict: | ||
| with open(path, encoding="utf-8") as f: | ||
| return json.load(f) | ||
|
|
||
|
|
||
| def _backup_path() -> str: | ||
| return os.path.join( | ||
| os.path.dirname(litellm.__file__), | ||
| "model_prices_and_context_window_backup.json", | ||
| ) | ||
|
|
||
|
|
||
| def _main_path() -> str: | ||
| return os.path.join( | ||
| os.path.dirname(__file__), "..", "..", "model_prices_and_context_window.json" | ||
| ) | ||
|
|
||
|
|
||
| class TestGeminiFlashLiteImagePricingData: | ||
| """Both price maps must carry Google's published Nano Banana 2 Lite costs.""" | ||
|
|
||
| def test_present_in_both_maps(self): | ||
| main = _load_json(_main_path()) | ||
| backup = _load_json(_backup_path()) | ||
| for key in VARIANTS: | ||
| for label, data in (("main", main), ("backup", backup)): | ||
| assert key in data, f"{key} missing from {label} JSON" | ||
| entry = data[key] | ||
| for field, value in EXPECTED.items(): | ||
| assert entry[field] == value, f"{key} {field} in {label}: {entry.get(field)} != {value}" | ||
|
|
||
| def test_capabilities_match_model_cards(self): | ||
| main = _load_json(_main_path()) | ||
| backup = _load_json(_backup_path()) | ||
| for key in VARIANTS: | ||
| expected = {**EXPECTED_CAPABILITIES, **EXPECTED_PER_ROUTE[key]} | ||
| for label, data in (("main", main), ("backup", backup)): | ||
| entry = data[key] | ||
| for field, value in expected.items(): | ||
| assert entry[field] == value, f"{key} {field} in {label}: {entry.get(field)} != {value}" | ||
|
|
||
| def test_grounding_fields_absent(self): | ||
| """Grounding with Google Search is unsupported on Lite, so no search pricing.""" | ||
| for path in (_main_path(), _backup_path()): | ||
| data = _load_json(path) | ||
| for key in VARIANTS: | ||
| for field in ( | ||
| "supports_web_search", | ||
| "search_context_cost_per_query", | ||
| "web_search_billing_unit", | ||
| ): | ||
| assert field not in data[key], f"{key} should not define {field}" | ||
|
|
||
| def test_image_output_pricing_consistent(self): | ||
| """1120 image-output tokens * output_cost_per_image_token == output_cost_per_image.""" | ||
| backup = _load_json(_backup_path()) | ||
| entry = backup["gemini-3.1-flash-lite-image"] | ||
|
greptile-apps[bot] marked this conversation as resolved.
|
||
| assert round(1120 * entry["output_cost_per_image_token"], 6) == entry["output_cost_per_image"] | ||
|
|
||
|
|
||
| class TestGeminiFlashLiteImageModelInfo: | ||
| """``get_model_info`` and ``completion_cost`` must report the new costs.""" | ||
|
|
||
| def test_get_model_info_and_cost(self): | ||
| original = litellm.model_cost | ||
| try: | ||
| litellm.model_cost = _load_json(_backup_path()) | ||
| info = litellm.get_model_info("gemini-3.1-flash-lite-image") | ||
| assert info["input_cost_per_token"] == EXPECTED["input_cost_per_token"] | ||
| assert info["output_cost_per_token"] == EXPECTED["output_cost_per_token"] | ||
|
|
||
| resp = ModelResponse() | ||
| resp.model = "gemini-3.1-flash-lite-image" | ||
| resp.usage = Usage( | ||
| prompt_tokens=7, | ||
| completion_tokens=1120, | ||
| total_tokens=1127, | ||
| completion_tokens_details=CompletionTokensDetailsWrapper( | ||
| image_tokens=1120, text_tokens=0 | ||
| ), | ||
| ) | ||
| cost = completion_cost( | ||
| completion_response=resp, | ||
| model="gemini-3.1-flash-lite-image", | ||
| custom_llm_provider="vertex_ai", | ||
| ) | ||
| expected_cost = 1120 * 3e-05 + 7 * 2.5e-07 | ||
| assert abs(cost - expected_cost) < 1e-6, f"unexpected cost {cost}" | ||
| finally: | ||
| litellm.model_cost = original | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing Vertex cache read pricing
Medium Severity
The Vertex
gemini-3.1-flash-lite-imageentries setsupports_prompt_cachingtotruebut omitcache_read_input_token_cost. This leads to LiteLLM undercounting spend for cached input tokens, as Google bills them at$0.025/1M.Reviewed by Cursor Bugbot for commit 246dd79. Configure here.