fix(registry): add Gemini Omni 1.1 Flash, xAI grok-imagine image models, Mistral cache-read pricing, GLM 5.3 Flash + Kimi K2.7 Code entries - #38560
Conversation
…imagine image models and grok-4.20 aliases, mistral cache-read pricing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
|
PR #38560 (BerriAI/litellm, author devin-ai-integration[bot]) has no labels — the |
Greptile SummaryThis PR expands the model registry with Gemini, xAI, Mistral, Z.AI, Databricks, and Moonshot metadata and pricing, while completing the prior fix for generated-image accounting.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| litellm/litellm_core_utils/llm_cost_calc/utils.py | Adds validated request-parameter fallbacks for image-cost tier selection and consistently forwards the resolved size, quality, and count. |
| model_prices_and_context_window.json | Adds and corrects provider model metadata and uses the per-image field consumed by the default image calculator for the new xAI models. |
| litellm/model_prices_and_context_window_backup.json | Keeps the backup registry synchronized with the primary registry changes. |
| tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py | Adds focused coverage for request-quality and valid-size fallback behavior in image cost routing. |
| tests/test_litellm/test_utils.py | Extends registry endpoint validation to accept Gemini’s Interactions API path. |
Reviews (6): Last reviewed commit: "fix(cost): only let a WxH request size d..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…upstream gemini 3.5 transcribe entries Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…_per_image for default calculator Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
|
Fixed in afe61aa: generated-image price now lives in input_cost_per_image, the key default_image_cost_calculator reads, matching the gpt-image-1 convention |
…evin/1787857843-registry-audit-rolling
…hot/kimi-k2.7-code; xai grok-imagine-image-pro deprecation date Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
…code entries Gemini omni 1.1 flash and omni flash preview only answer on the Interactions API, so both now list /v1beta/interactions as their endpoint and 1.1 flash gets the 131072 / 65536 limits the models API reports. grok-4.20-multi-agent and -latest now match the dated entry (mode responses, /v1/responses only), and all three drop function calling and tool choice since the API rejects client-side tools outside a beta. kimi-k2.7-code gets the capability flags kimi-k2.6 carries (tools, reasoning, JSON mode, image and video input) plus max_output_tokens. grok-imagine-image-2.0 gets a low quality tier at $0.04 so quality=low is not billed at the $0.06 default.
… API The models API reports 131072 input / 65536 output for the preview model and the Interactions API accepts 100k tokens but rejects 130k, so the 1,048,576 input limit copied from the docs was wrong.
…e response omits it
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b8f9123. Configure here.
2 participants
TLDR
Problem this solves:
How it solves it:
cache_read_input_token_costfrom the official pricing pagedeprecation_date: 2026-05-15toxai/grok-imagine-image-prosupported_endpointsenum accept/v1beta/interactions, the Interactions API path the two Gemini Omni entries now declarequalityandWxHsizewhen the provider's image response omits them (xAI's do), soquality: lowonxai/grok-imagine-image-2.0is billed at its $0.04 tier instead of the $0.06 default; a non-WxHsize such asautois ignored as beforeUser Flow
Before: a developer whose gateway routes the new Gemini, xAI, Z.AI, and Moonshot models sees $0 spend for them, and Mistral prompt-cache hits are billed at the full input rate
gemini/gemini-omni-1.1-flash,xai/grok-imagine-image,zai/glm-5.3-flash,moonshot/kimi-k2.7-code, anddatabricks/databricks-glm-5-3-flashto the proxy config and restarts itmodel: gemini/gemini-omni-1.1-flash(the only API Google serves this model on) and gets a normal 200 with nox-litellm-response-costheader andx-litellm-response-cost-original: 0.0; the same happens on POST https://litellm-domain/v1/chat/completions for the Z.AI and Moonshot models. POST https://litellm-domain/v1/responses withmodel: xai/grok-4.20-multi-agentis priced only because xAI's reply names the dated-0309slug, and GET https://litellm-domain/health?model=xai/grok-4.20-multi-agent fails withMulti Agent requests are not allowed on chat completionsmodel: xai/grok-imagine-imageorxai/grok-imagine-image-2.0, get their image back with no cost header, and the spend log row says $0model: mistral/mistral-large-latesttwice with the same long prefix; when the second reply reports cached prompt tokens, the cost header still charges them at the full input rateAfter: the same requests are priced from the official provider rates
gemini/gemini-omni-1.1-flash,xai/grok-imagine-image,zai/glm-5.3-flash,moonshot/kimi-k2.7-code, anddatabricks/databricks-glm-5-3-flashto the proxy config and restarts it/v1beta/interactionsas their endpoint, the multi-agent aliases list/v1/responseswith function calling off, andmoonshot/kimi-k2.7-codereports tool calling, reasoning, and vision (the Databricks entry carries limits and capabilities only, since Databricks has not published a rate)model: gemini/gemini-omni-1.1-flashand thex-litellm-response-costheader carries the real cost at the official per-token rates; the same holds for the Z.AI and Moonshot chat requests, the multi-agent Responses request keeps its price, and GET https://litellm-domain/health?model=xai/grok-4.20-multi-agent passesmodel: xai/grok-imagine-imageand the cost header reads $0.02 per generated image;xai/grok-imagine-image-2.0reads $0.06, or $0.04 when the request carriesquality: lowChanges by provider (all values from official docs)
Gemini, https://ai.google.dev/gemini-api/docs/pricing
gemini/gemini-omni-1.1-flash(new, GA ofgemini-omni-flash-preview): input $1.50/M (text/image/video/audio), output $9.00/M text, $17.50/M videoGET /v1beta/models/gemini-omni-1.1-flashreportsinputTokenLimit: 131072andoutputTokenLimit: 65536, and live Interactions API probes agree (a 100k-token input is accepted, 130k and above are rejected with 400Invalid input received.), somax_input_tokens: 131072andmax_output_tokens: 65536are set from the API.gemini/gemini-omni-flash-previewreports the same limits from the models API and behaves the same on the probes, so its 1,048,576 input limit (copied from the docs when it was added) is corrected to 131072 / 65536 as wellgenerateContentanswers both omni models with 400This model only supports Interactions API., sosupported_endpointson both entries is/v1beta/interactions(the proxy route and the playground label for the Interactions API) instead of the/v1/chat/completionsthey carried before.modestayschatbecause the registry schema'smodeenum has no interactions valuetpm/rpmfollow the repo's existing convention: matchesgemini-omni-flash-preview(800000/2000)gemini-3.5-transcribe/-livewere added here originally but have since landed onlitellm_internal_stagingwith matching prices, so this PR now defers to the upstream entriesxAI, https://docs.x.ai/docs/models and https://docs.x.ai/developers/migration/may-15-retirement
xai/grok-imagine-image,xai/grok-imagine-image-2026-03-02(new): $0.02 per generated imagexai/grok-imagine-image-quality,-quality-20260403,-quality-latest,xai/grok-imagine-image-pro(new): $0.05 per generated imagexai/grok-imagine-image-2.0(new): $0.06 per generated image at the default 1K resolution / medium quality tier (the API'simage_pricefor the model), plus alow/1024-x-1024/grok-imagine-image-2.0entry at $0.04 that the image cost calculator picks up when the request carriesquality: low, following thelow/1024-x-1024/gpt-image-1convention. The 2K tiers ($0.06 low, $0.08 medium) key on xAI'sresolutionparameter, which the cost calculator does not see, so they are not encodedinput_cost_per_imagebecause xAI image generation is priced by the default per-image calculator, which reads that key (same convention asgpt-image-1). xAI's separate $0.002 image-input (edit/reference) rate has no representable key on this path, so it is not encodedxai/grok-4.20,-reasoning,-reasoning-latest,-non-reasoning,-non-reasoning-latest,-multi-agent,-multi-agent-latest(new): official aliases of the existing datedgrok-4.20-*-0309entries, same pricing ($1.25/M in, $2.50/M out, $0.20/M cached)-multi-agentand-multi-agent-latestmatch the dated entry'smode: responsesand/v1/responsesas the only endpoint (xAI serves the multi-agent model on the Responses API only), and all three multi-agent entries now carrysupports_function_calling: falseandsupports_tool_choice: false: a Responses request with a function tool is rejected with 400Client-side tools for multi-agent models require beta accessxai/grok-imagine-image-pro:deprecation_date: 2026-05-15added; the retirement page lists it in the May 15, 2026 batch (redirects togrok-imagine-image-qualityafterwards). The other seven retired slugs on that page already carry2026-05-15in the registrygrok-imagine-video/grok-imagine-video-1.5(pricing is per-second per resolution tier, no flat rate representable),grok-tts/grok-stt/grok-voice-*(no pricing published)Mistral, https://docs.mistral.ai/inference/pricing
Added
cache_read_input_token_cost(10% of input price, per pricing page "Cached input" column):mistral-large-2512/mistral-large-3/mistral-large-latest: $0.05/Mmistral-medium-2604/mistral-medium-3-5/mistral-medium-latest: $0.15/Mmistral-small-2603/mistral-small-latest: $0.015/Mministral-3-14b-2512: $0.02/M;ministral-3-8b-2512/ministral-8b-2512/ministral-8b-latest: $0.015/M;ministral-3-3b-2512: $0.01/Mcodestral-2508/codestral-latest: $0.03/M;codestral-embed/codestral-embed-2505: $0.015/MNot changed: OCR cached-page pricing (no registry key for cached OCR pages),
mistral-embedprice (not listed on the current pricing page),voxtral-mini-realtime(no published rate found). Thecodestral-embedcached price is published and encoded for completeness, but embedding responses never report cached tokens, so it does not affect spendZ.AI (absorbed from #38627), https://docs.z.ai/guides/overview/pricing
zai/glm-5.3-flash(new): input $0.15/M, cached input $0.03/M, output $0.50/M (list prices; the page's 50% promo ending 2026-09-09 is not encoded), 1M context / 128K max output per https://docs.z.ai/guides/llm/glm-5.3-flashsupports_vision: trueadded on top of the source PR: the model page lists Video / Image / Text / File input modalityDatabricks (absorbed from #38583), https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models
databricks/databricks-glm-5-3-flash(new): 1,048,576 input context, text+image input, function calling, reasoning, prompt caching; 131,072 max output matching the siblingdatabricks-glm-5-2entryMoonshot (absorbed from #38576), https://platform.kimi.ai/docs/pricing/chat-k27-code
moonshot/kimi-k2.7-code(new): input $0.95/M (cache miss), cache hit $0.19/M, output $4.00/M, 262,144 context, all read from the raw pricing table and cross-checked via r.jina.aisupports_function_calling,supports_tool_choice,supports_reasoning,supports_response_schema,supports_vision,supports_video_input, andmax_output_tokens: 262144, matching the existingmoonshot/kimi-k2.6entry. The model page lists text, image, and video input, thinking mode, tool calls, and JSON mode, and a live tool call returnedfinish_reason: tool_callswith reasoning tokensAbsorption of other open registry PRs
Absorbed (verified against official docs, field-level): #38627, #38583, #38576, now superseded and closed
Re-verified and dropped as unverifiable (left open, not absorbed):
deprecation_date: 2026-02-28for the grok-2-era slugs appears nowhere on xAI's docs; the official retirement page only documents the May 15, 2026 batch. Its twomode: responsesfixes are already on this branchgemini/gemini-live-2.5-flash-native-audio, but the official Gemini API pricing page lists this model only under the sluggemini-2.5-flash-native-audio-preview-12-2025, which the registry already covers with matching prices; the pricing page also shows no context-caching or grounding rates for it, so the extracache_read/ web-search fields in feat(pricing): add gemini-live-2.5-flash-native-audio to model cost map #38573 are unverifiable*_above_272k_tokenstiers could not be confirmed from the public pages this runsupports_vision: false): Novita's own model page currently lists "Input Capabilities: text, image", contradicting the changeCode/behavior PRs #36233 and #29898 touch the registry only in support of their code change; left alone, not absorbed. #30383 does not touch the registry
Audit notes (checked, no change needed)
gemini-embedding-2,gemini-3.1-flash-image, Mistral OCR 4.x, Voxtral, Ministral 3, Leanstral entries already present and correctRelevant issues
Fixes #38608
Deprecation dates use the
deprecation_datekey that #26900 surfaces throughGET /model/deprecations, matching the 586 existing entries that already carry itLinear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or moretests/test_litellm/test_model_prices_schema.py,tests/test_litellm/test_model_cost_aliases.py, andtests/test_litellm/test_xai_grok_4_3_model_metadata.pypass (36 passed),ci_cd/check_files_match.pypasses with the backup JSON in sync, andci_cd/generate_model_prices_schema.pyproduces no diffuv run pytest tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py -k falls_back_to_requested(7 passed); dropping the size guard makes theautocase fail withValueError: invalid literal for int() with base 10: 'auto'@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Both legs boot a real proxy (
--num_workers 2, Postgres spend logging,LITELLM_LOCAL_MODEL_COST_MAP=Trueso each tree prices from its own registry) from the named commit with the same config: the eleven models named in the User Flow plusmistral/mistral-large-latest, real provider keys for Gemini, xAI, Moonshot, and Mistral. Before is the merge baseca0b951a43(port 30782 on this boot), After is the PR tipb8f91235a6(port 42311).$KEYis the proxy master key. Long response ids are truncatedBefore (ca0b951)
After (b8f9123)
Live PR risk A/B, same two proxies (Before ca0b951 on 25661, After b8f9123 on 42311)
Verdict: PASS
Breaking: none observed. Backward incompatible: the image cost router now reads
qualityand aWxHsizefrom the request when the provider's image response omits them, so tiered image requests to providers whose responses omit those fields (xAI,dall-e-3) are billed at their registry tier instead of the standard 1024x1024 price; intended, listed under Caveats. Regression risk:dall-e-3quality: hdand1792x1024requests could not be exercised live (OpenAI retired DALL-E 2 and 3 on 2026-05-12 and answersThe model 'dall-e-3' does not existon both legs; Azure's catalog no longer lists it), so that path rests on the parametrized unit tests. Dependency graph, verified live on both legs:/model/infoand/model_group/info(registry fields and router capability flags),/utils/supported_openai_params(the multi-agent alias gainsreasoning_effortfromsupports_reasoning: true,moonshot/kimi-k2.7-codeunchanged),/v1/models(unchanged),/health(the multi-agent probe moves to the Responses API and passes; both Gemini Omni entries still fail withThis model only supports Interactions API.on both legs), and the cost headers plus spend rows on the interactions, chat, responses, and image routes above. Untested:GET /model/deprecations(read-only listing ofdeprecation_date). Not verified:zai/glm-5.3-flashanddatabricks/databricks-glm-5-3-flashrequests (no credentials on this box; both entries are checked through/model/infoonly), Mistral cached-token billing (Mistral reportedcached_tokens: 0on both calls of both legs),dall-e-3/healthfor the xAI image model on the same two proxies (issue #26184's symptom):Verdict: PASS
Type
🆕 New Feature
🐛 Bug Fix
Caveats (if any)
Low
databricks/databricks-glm-5-3-flashlogs $0 spend until Databricks publishes a rate: the Databricks pricing page lists GLM 5.2 only, so there is nothing to encode, the gap exists with or without this PR, and a deployment can setinput_cost_per_token/output_cost_per_tokeninlitellm_paramsin the meantimemode: chateven though only the Interactions API serves them, so a/healthcheck on them fails (observed on both QA legs); a dedicated mode needs a schema change and health-check support, out of scope for a registry PRgrok-imagine-image-2.02K tiers ($0.06 low, $0.08 medium) and the $0.01 per input image rate are not encoded, since neither theresolutionparameter nor image inputs reach the image cost calculator; the older xAI image models' $0.002 image-input rate is left out for the same reasondall-e-3and any other provider whose image response omitsquality/size: aquality: hdor1792x1024request is now looked up under itshd/1024-x-1024/dall-e-3orstandard/1792-x-1024/dall-e-3registry key ($0.08, OpenAI's list price) instead of the standard 1024x1024 price ($0.04) it was under-billed at; requests that set neither are unchanged. Not exercised live: OpenAI shut DALL-E 2 and 3 down on 2026-05-12 (the API now answersThe model 'dall-e-3' does not exist) anddall-e-3is no longer in Azure's model catalog, so only a lingering Azure deployment can still reach this pathxai/grok-imagine-image-progetsdeprecation_date: 2026-05-15, which is already in the past; the proxy only surfaces the field throughGET /model/deprecationsand never blocks routing on it/utils/supported_openai_params?model=xai/grok-4.20-multi-agentnow listsreasoning_effort, derived from thesupports_reasoning: truethe alias shares with the dated entryFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/0c7c623504ce4aacab9c05f3d974140a
Open in Devin Desktop: https://app.devin.ai/desktop/session/0c7c623504ce4aacab9c05f3d974140a?variant=devin