Repository navigation
fix(model-prices): align Azure, Bedrock, Copilot, Gemini, Groq, OpenAI and OpenRouter entries with official docs - #43598
Conversation
Co-authored-by: rain <1504569896@qq.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…l docs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| "max_input_tokens": 1048576, | ||
| "max_output_tokens": 65536, | ||
| "max_tokens": 65536, | ||
| "mode": "chat", |
There was a problem hiding this comment.
When a caller selects Deep Research Pro through /v1/chat/completions, this chat entry sends the request to Gemini's generateContent endpoint. Deep Research Pro requires the separate Interactions API, so the caller cannot obtain a research report through the advertised chat route.
| "cache_creation_input_token_cost": 1.5e-05, | ||
| "cache_creation_input_token_cost_above_272k_tokens": 3e-05, | ||
| "cache_read_input_token_cost": 1.2e-06, | ||
| "cache_read_input_token_cost_above_272k_tokens": 2.4e-06, | ||
| "input_cost_per_token": 1.2e-05, | ||
| "input_cost_per_token_above_272k_tokens": 2.4e-05, |
There was a problem hiding this comment.
The revised EU rates include separate cache prices and prices above 272K tokens, but no cost calculation test checks this Azure entry at either tier. A value or tier-selection regression could therefore silently bring back the invoice discrepancy this change is meant to fix. Please add a test covering ordinary and above-threshold usage, including cached tokens.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ure AI registry fixes Absorbs the fields from #43609, #43666, #43671 and #43644 that match the provider's own docs or price API today, and adds a cost test for the azure/eu/gpt-6-astra Data Zone tiers Co-authored-by: bunnysayzz <stfuazzo@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
|
1 similar comment
|
|
…audit_1790623011 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> # Conflicts: # litellm/model_prices_and_context_window_backup.json # model_prices_and_context_window.json
…OpenRouter values from official sources Co-authored-by: Michal Formanek <michal.formanek@generaliceska.cz> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
1 participant
TLDR
Problem this solves:
gpt-4o-mini-tts-2025-03-20has no entry, so its calls go unpricedmoonshotai.kimi-k3bills at the cheaper Global rateopenrouter/~x-ai/grok-latestdrifted from the Grok 4.7 prices it redirects toHow it solves it:
gpt-4o-mini-tts-2025-03-20, cloned from its aliasUser Flow
Before: a team on the EU Data Zone GPT-6 Astra deployment sees spend below what Azure bills
azure/eu/gpt-6-astrato the model listx-litellm-response-costreads $7.70 while the Azure invoice shows $8.40After: the same request is priced at the Data Zone rate
x-litellm-response-costreads $8.40, matching AzureChanges by provider
Azure
azure/eu/gpt-6-astra, absorbed from #43579 by @Rainmemery (credited as co-author)serviceName eq 'Foundry Models' and armRegionName eq 'swedencentral', productAzure OpenAI GPT6, meters6-astra ShortCo/LongCo ... Std DZ(Inp 12, Cd Inp 1.2, Cd Wr 15, Opt 60 short; 24, 2.4, 30, 90 long, per 1M) https://prices.azure.com/api/retail/prices?$filter=serviceName%20eq%20'Foundry%20Models'%20and%20armRegionName%20eq%20'swedencentral'%20and%20priceType%20eq%20'Consumption'Gemini
gemini/deep-research-pro-preview-12-2025image_generationtochat, matching the othergemini/deep-research-*entries["text"], supported_modalities adds audio and video, supports_pdf_input trueoutput_cost_per_imageandoutput_cost_per_image_token, since the model outputs text reports onlygemini/gemini-3-pro-image: supports_reasoning true, supports_prompt_caching falsegemini/gemini-3.1-flash-image: supports_reasoning true, supports_prompt_caching false, supports_pdf_input truegemini/gemini-3.1-flash-lite-image: supports_reasoning true, supports_pdf_input trueGroq
groq/qwen/qwen3.8-27b: max_input_tokens 131042 to 131072OpenAI
New
gpt-4o-mini-tts-2025-03-20, identical togpt-4o-mini-ttsVertex AI (absorbed from #43671)
gemini-live-2.5-flash-native-audio: input_cost_per_video_token 3e-06vertex_ai/gemini-2.5-flash-native-audio: input_cost_per_image_token and input_cost_per_video_token 3e-06vertex_ai/claude-sonnet-4-5andvertex_ai/claude-sonnet-4-5@20250929: cache_creation_input_token_cost_batches 1.88e-06Gemini Robotics ER 2 (absorbed from #43666 by @bunnysayzz, credited as co-author)
gemini/gemini-robotics-er-2-preview: output_cost_per_reasoning_token 1e-05 to 5e-06Azure AI (absorbed from #43609)
azure_ai/deepseek-v3.2: max_input_tokens 163840 to 128000azure_ai/grok-4: max_input_tokens 131072 to 262000azure_ai/grok-code-fast-1: max_input_tokens 131072 to 256000OpenRouter (verified part of #43644)
Every value below was re-read from the live
GET https://openrouter.ai/api/v1/modelsresponse on 2026-09-29 (pricing.prompt,pricing.completion,pricing.input_cache_read,context_length,top_provider.max_completion_tokens). Source for all rows: https://openrouter.ai/api/v1/modelsopenrouter/anthropic/claude-sonnet-5.5:batch: 1e-06 in, 5e-06 out, 1e-07 cache read, 1.25e-06 cache write, 2e-06 1h cache write, 1,000,000 in, 128,000 outdeepseek/deepseek-v3.2-exp: max_output_tokens and max_tokens 65536 to 147456deepseek/deepseek-v4-flash-0731: input 2.1e-08 to 1.8e-08deepseek/deepseek-v4-flash-vision-exp: input 2.156e-07, output 6.468e-07, cache read 6.86e-09, max output 262144deepseek/deepseek-v4-pro-0813: output 3.5e-06 to 4.2e-06deepseek/deepseek-v4.1-flash: input 3e-07, output 1.2e-06, cache read 6e-09, max output 943718minimax/minimax-m2: input 3e-07, output 1.2e-06, max output 176947moonshotai/kimi-k2.6: input 6.5e-07, output 3.41e-06, cache read 1.5e-07nvidia/nemotron-3.5-lightning: input 6e-08, output 1.6e-07, cache read 3e-08, max output 32768openai/gpt-5.6-sol-pro: input 4e-06, output 2e-05, cache read 4e-07, cache write 5e-06; above 272K 8e-06, 3e-05, 8e-07, 1e-05openai/gpt-oss-20b: new cache read 9e-09prism-ml/ternary-bonsai-2-27b: new cache read 3.75e-08qwen/qwen3-14b: input 1.2e-07, output 2.4e-07qwen/qwen3-30b-a3b: input 1.2e-07, output 5e-07, max output 16384x-ai/grok-4.7: input 2e-06, output 6e-06, cache read 5e-07; above 200K 4e-06, 1.2e-05, 1e-06z-ai/glm-5.3: input 1.4e-06, output 4.4e-06, cache read 2.6e-07, max output 943717GitHub Copilot (pricing on existing entries absorbed from #43702 by @michalformanek, credited as co-author)
Source for every row: https://docs.github.com/en/copilot/reference/copilot-billing/models-and-pricing, read from its data file https://raw.githubusercontent.com/github/docs/main/data/tables/copilot/models-and-pricing.yml (prices per 1M tokens)
github_copilot/claude-haiku-4.5: input 1e-06, output 5e-06, cache read 1e-07, cache write 1.25e-06 ($1, $5, $0.10, $1.25)github_copilot/claude-sonnet-4: input 3e-06, output 1.5e-05, cache read 3e-07, cache write 3.75e-06 ($3, $15, $0.30, $3.75)github_copilot/gpt-5-mini: input 2.5e-07, output 2e-06, cache read 2.5e-08 ($0.25, $2, $0.025)github_copilot/gpt-5.3-codex: input 1.75e-06, output 1.4e-05, cache read 1.75e-07 ($1.75, $14, $0.175)Bedrock
moonshotai.kimi-k3(the in-Region ID) now bills at the in-Region standard rate instead of the Global rateus.stays at 3.3e-06 andglobal.stays at 3e-06, both already matchingUSE1-moonshotai.kimi-k3-mantle-{input,output,cache-read}-tokens-standard(0.0033, 0.0165, 0.00033 per 1K) and-cache-write-tokens-30m-standard(0.004125 per 1K), against-global-standardat 0.003, 0.015, 0.0003, 0.00375bedrock-runtimeID for Kimi K3 https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-moonshot-ai-kimi-k3.htmlGemini Robotics ER 2 Streaming
gemini/gemini-robotics-er-2-streaming-preview: adds max_input_tokens 131072, max_output_tokens and max_tokens 65536, supports_reasoning trueOpenRouter (this run)
Source for all rows: https://openrouter.ai/api/v1/models, read 2026-09-29 (
context_lengthandtop_provider.context_lengthagree on each)openrouter/inclusionai/ling-3.0-flash-vl: max_input_tokens 131072 to 262144openrouter/thinkingmachines/inklingandinkling-small: max_input_tokens 1048576 to 524288openrouter/~x-ai/grok-latest: input 1.6e-06 to 2e-06, output 4.8e-06 to 6e-06, cache read 4e-07 to 5e-07; above 200K 4e-06, 1.2e-05, 1e-06, now matchingx-ai/grok-4.7which the alias redirects toAbsorbed and superseded PRs
prompt_cache_min_tokens: 512andsupports_forced_tool_use: falsealready landed on main via fix(model_prices): correct Claude Sonnet 5.5 capabilities and provider keys #43587. Itssupports_web_search: trueis dropped, since the Sonnet 5.5 pages do not list it and no other first-party Sonnet 5 entry sets it; its source swap is cosmetic and droppeddeepseek/deepseek-v3.2-expprices and cache fields (API: 2.7e-07 in, 4.1e-07 out, no cache read)deepseek/deepseek-v4-flash-0731cache read (API 1.8e-08, PR 1e-08)deepseek/deepseek-v4-proprices and cache read,deepseek/deepseek-v4-pro-0813input and cache readgoogle/gemma-4-26b-a4b-itprices and cache readnvidia/nemotron-3.5-lightningmax_input_tokens (API 262144, PR 1000000)modeandsupported_endpointsappear on no official GitHub page, so they cannot be confirmed. Its test file is not carried over, since it asserts on the dropped keys. Closed as supersededbedrock_mantle/google.gemma-4-e4b): dropped, no Bedrock model card or price list row for itbedrock-mantleonly https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-xai-grok-4-3.htmlGET https://api.fireworks.ai/inference/v1/modelsdisagrees with the proposed valueembed-multilingual-light-v3.0Checked and not changed
Deprecations (feat(proxy): proactive model deprecation alerts and
/model/deprecationsendpoint #26900deprecation_date): no new dates added this rungpt-4o-audio-preview-2024-12-17and similar) keep 2027-01-20; the 2026-05-07 row on https://developers.openai.com/api/docs/deprecations names only the alias, not the dated snapshotOpenAI image models: https://developers.openai.com/api/docs/deprecations now lists
gpt-image-1shutdown 2026-10-23 andgpt-image-1.5,gpt-image-1-mini,chatgpt-image-latest2026-12-01; the registry already carries these datesAnthropic, xAI, DeepSeek: every model in the official model list API already has an entry
OpenAI: other list-API IDs without entries (
gpt-audio-mini-2025-10-06,gpt-4o-search-preview-2025-03-11,gpt-4o-mini-search-preview-2025-03-11) are marked Deprecated on their model pages, so not addedPerplexity and Voyage: pricing pages match the registry for every model checked
Gemini:
antigravity-preview-*is priced at "standard Gemini list rates" of whatever model runs, with no per-token price, so not added;aqahas no published priceGemini: the
vertex_ai/deep-research-pro-preview-12-2025entry has the same shape as the Gemini one but was not checked against Vertex docs, so it is untouchedFireworks:
accounts/fireworks/models/qwen3p8-2p4t-a95bis on-demand only ("Serverless Not supported" at https://fireworks.ai/models/fireworks/qwen3p8-2p4t-a95b), and neitherGET https://api.fireworks.ai/inference/v1/modelsnorGET https://api.fireworks.ai/v1/serverless/modelscarries a per-token price, so not addedFireworks: the
minimax-m3supports_vision: truepins inci_cd/cost_map_pins.jsonstill disagree with the models API (supports_image_input: false); the pinned values are left as isCohere: 12 model IDs from
GET https://api.cohere.com/v1/modelshave no entry (Command A Reasoning, Translate and Vision, Command R7B Arabic, Aya Vision 32B, Transcribe, North Mini Code, North Small Translate, Tiny Aya), but https://cohere.com/pricing publishes per-token prices only for legacy Command, Command R and Aya Expanse, checked as raw HTML, through r.jina.ai and in a browser, so they are not addedMistral: API-only differences on
codestral-2508andlabs-leanstral-1-5do not match the official model pages' naming or values, so nothing changedTogether: priced models missing from the registry were found in the models API, but the pricing page could not be matched row by row, so not added this run
Nebius and Groq model APIs need keys not available to this run; Groq was checked through its docs pages
Bedrock: Gemma 4 and Grok 4.3 Mantle prices and windows match the Price List API and model cards; Grok 4.7 is
bedrock-runtimeonly per its card, so no Mantle entry is missingOpenRouter:
xiaomi/mimo-v2.6-proreports 1050000 at the top level and 1048576 for its top provider, so left alone; router IDs such asopenrouter/auto-betacarry -1 or zero prices and are not addedGemini: official pages give 1,048,576 in and 65,536 out for
lyria-3-clip-previewandlyria-3-pro-preview, but these are music models and the registry's 131072/8192 may reflect the generation request limit, so left for a reviewerTogether: eight priced audio/TTS models from
GET https://api.together.xyz/v1/models(Kokoro 82M, Orpheus 3B, Whisper Large v3, Cartesia Sonic 1/2/3, Nemotron 3 ASR, Rime Arcana v2) are not added, since the API gives no per-character or per-second unit the registry can storeGitHub Copilot:
gpt-5.4-nano,claude-sonnet-4.6andclaude-fable-5*are in GitHub's release status data but not in its pricing table, so not addedPre-Submission checklist
test_generic_cost_per_token_azure_eu_gpt_6_astra_tierspins the Data Zone rates below and above 272K; the other edits are registry data covered by the schema and alias teststests/unit/test_model_prices_schema.py,tests/unit/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py,tests/unit/test_openai_service_tier_long_context_pricing.py,tests/unit/test_utils.py,tests/unit/llms/azure_ai,tests/unit/llms/deepseekand others (1250 passed), pluspython3 ci_cd/check_files_match.pyandgenerate_model_prices_schema.py --checkType
Bug Fix
Caveats (if any)
Medium
/v1/chat/completionsendpoint list and token prices from mainmoonshotai.kimi-k3in-Region spend rises 10% to match the AWS in-Region ratesupports_prompt_cachinggoes from true to false on two Gemini image models, so model info no longer advertises caching for themLink to Devin session: https://app.devin.ai/sessions/6f722597ad00408b9457e9ac2dabffb2
Open in Devin Desktop: https://app.devin.ai/desktop/session/6f722597ad00408b9457e9ac2dabffb2?variant=devin