fix(model_prices): verified registry audit, Databricks Sep-2026 catalog, realtime image pricing, deprecation dates - #39388
Conversation
…nd gpt-daybreak aliases, fix gpt-5.5 Fast and W&B pricing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ect watsonx list prices Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…itellm_registry_audit_2026_09_02
…uld record zero spend Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
PR #39388 (BerriAI/litellm, author devin-ai-integration[bot]) has no labels — the |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR performs a broad model-registry audit, adding provider catalog entries and correcting pricing, capabilities, limits, modes, and retirement dates.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds and corrects model pricing, limits, capabilities, endpoint modes, and deprecation metadata across numerous providers. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the root model registry changes in the runtime backup map. |
| model_prices_and_context_window.schema.json | Extends recognized tiered cache-pricing fields used by new registry entries. |
| tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py | Adds regression coverage for realtime image-token pricing. |
| tests/test_litellm/litellm_core_utils/test_get_model_cost_map.py | Expands validation of corrected model prices and guards against stale registry values. |
| tests/test_litellm/llms/databricks/test_databricks_cost_calculator.py | Covers cost calculation for newly priced Databricks catalog entries. |
| tests/test_litellm/test_baseten_glm_5_3_model_metadata.py | Validates Baseten GLM-5.3 pricing, limits, modalities, and supported capabilities. |
| tests/test_litellm/test_gpt_image_cost_calculator.py | Verifies GPT Image 2 output is billed at the image-token rate with and without detailed token breakdowns. |
Reviews (13): Last reviewed commit: "fix(registry): mark baseten GLM-5.3 as v..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
… fast-mode rate Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…per registry convention Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…d maximum new tokens Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e148868. Configure here.
…itellm_registry_audit_2026_09_02
…ing-3.5, sync Azure and Vertex deprecation dates, fix novita gpt-oss vision flags Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…and Foundry docs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…es from Azure Retail Prices Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
devin-ai-integration
Bot
changed the title
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit c707f2f. Configure here.
…imax-m3 and qwen3.7-plus Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 0c29f51. Configure here.
…itellm_registry_audit_2026_09_02
…ash Vision, Together MiniMax M2.7 and Qwen2.5 7B Turbo pricing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
… pricing, both non-serverless Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ion docs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 50d6b26. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 50d6b26. Configure here.
922659f
into
litellm_internal_staging
1 participant
TLDR
Problem this solves:
How it solves it:
User Flow
Before: a proxy admin who added
azure_ai/Codestral-2501andazure/us/gpt-4.1-nano-2025-04-14to their config sees $0 spend for one and a retirement date a year too early for the othercodestral-2501row comes back withinput_cost_per_token: 0andmax_input_tokens: null, so every request to it is logged at $0 spendgpt-4.1-nano-azure-usrow saysdeprecation_date: 2026-10-14, a year before the date Azure publishesAfter: the same calls return Azure's list price and retirement schedule
codestral-2501row comes back withinput_cost_per_token: 3e-07,output_cost_per_token: 9e-07andmax_input_tokens: 256000, matching the Azure Mistral pricing page and Retail Prices APIgpt-4.1-nano-azure-usrow saysdeprecation_date: 2027-04-14, matching the Foundry retirement scheduleRelevant issues
Rolling registry PR for the 2026-09-02, 2026-09-03 (two runs) and 2026-09-04 (three runs) audits. Continues #39170 (merged). Deprecation dates use the
deprecation_datefield shipped for #26900Linear ticket
Changes by provider
Every value below was read from the raw HTML or JSON of the linked official page. Root and backup maps are identical (
ci_cd/check_files_match.pypasses) and the schema is unchanged.Azure Foundry (2026-09-03 run)
Sources: https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule (raw HTML table rows), https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure, https://prices.azure.com/api/retail/prices (official Retail Prices API,
serviceName eq 'Foundry Models'), https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/mistral/, https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/fireworks/azure/us/gpt-4.1-nano-2025-04-14:deprecation_date2026-10-14 -> 2027-04-14. The schedule row forgpt-4.1-nanoversion 2025-04-14 reads "Model retirement 2027-04-14, deployment retirement 2027-10-14"; the old value matched neitherazure_ai/claude-fable-5-1: addeddeprecation_date: 2027-12-05, the schedule row forclaude-fable-5-1. Before this the proxy fell back to Anthropic's own 2027-09-01 date from the bareclaude-fable-5-1entryazure_ai/Codestral-2501: $0.30 / $0.90 per 1M (Retail PricesCodestral Inp glbl0.0003 per 1K andCodestral Outp glbl0.0009 per 1K), 256K context and 4,096 output per the Foundry catalog page, chat mode, text only. The schedule lists it as GA with no retirement date, so none is recordedazure_ai/FW-Nemotron-Lightning-3.5-30B-A3B: $0.06 / $0.22 per 1M with $0.01 cached input (Retail PricesFW NVIDIA Nemotron 3.5 LN In/Opt/Cd In DZmeters, the only meters Azure publishes for it), 262,144 context and text-only modalities per the catalog page, reasoning and tool calling onazure_ai/MAI-Thinking-1: $2 / $8 per 1M with $0.20 cached input (Retail PricesMAI-Thinking-1 Inp/Opt/Cd Inp glbl 1M Tokens), 256,000 context, 64,000 output, text only, tool calling and reasoning content, per the models-sold-directly-by-Azure pageazure_ai/mistral-ocr-4-0: $4 per 1K pages OCR and $5 per 1K pages annotation (Retail PricesOCR 4 glbl PagesandOCR 4 annot glbl Pages), same shape as the existingmistral/mistral-ocr-4-0entryDatabricks (2026-09-04 run; https://www.databricks.com/product/pricing/foundation-model-serving and https://www.databricks.com/product/pricing/proprietary-foundation-model-serving for DBU rates, https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models for endpoint names, input modalities, the limits it prints and the Responses API notes, the vendors' own model pages for limits Databricks does not print (https://developers.openai.com/api/docs/models/gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.5-pro, gpt-5.4; https://ai.google.dev/gemini-api/docs/models/gemini-3.8-flash and the other Gemini 3.x Flash, Flash Lite and Image pages), https://docs.databricks.com/aws/en/machine-learning/model-serving/function-calling and https://docs.databricks.com/aws/en/machine-learning/model-serving/query-reason-models for tool and reasoning support, https://docs.databricks.com/aws/en/machine-learning/retired-models-policy for retirement dates; all read from browser-rendered raw HTML because the pricing pages sit behind a JavaScript challenge)
databricks/databricks-claude-fable-5-1,gpt-5-6-sol,gpt-5-6-terra,gpt-5-6-luna,gpt-5-5,gpt-5-5-pro,gemini-3-8-flash,gemini-3-7-flash,gemini-3-6-flash,gemini-3-5-flash,gemini-3-5-flash-lite,gemini-3-1-flash-image,gemini-3-pro-image,glm-5-3,grok-4-6,inkling,qwen35-122b-a10b,qwen3-next-80b-a3b-instruct,qwen3-embedding-0-6b. Dollar prices are the published Global DBU per 1M rates times $0.070, the same convention and*_dbu_cost_per_tokenreference fields the existing Databricks entries use. Input modalities come from the supported-models page, and limits from the same page where it prints them (Inkling 1,000,000 in, Grok 4.6 500,000 in text only, Qwen3.5 122B 256K in / 25K out, GLM-5.3 1,048,576 in / 65,536 out) or from the vendor's model page otherwise (GPT-5.5, 5.5 Pro and the three GPT-5.6 rows: OpenAI prints a 1,050,000 context and 128,000 max output, stored as 922,000 in / 128,000 out like the directgpt-5.6-solentry; Gemini 3.x Flash rows 1,048,576 in / 65,536 out; Gemini 3.1 Flash Image 131,072 in / 32,768 out; Gemini 3 Pro Image 65,536 in / 32,768 out).gpt-5-5andgpt-5-5-procarrymode: responsesbecause the Databricks page says "Use the Responses API to interact with this model".supports_reasoningandsupports_function_callingare set only where the reasoning and function-calling pages list the modeldatabricks/databricks-glm-5-3-flash: now priced at 2.143 / 7.143 DBU per 1M with 0.429 cached input (the pricing page publishes the row this run; the entry previously said rates were not published), plus the 1,048,576 context, image input and reasoning flags from the supported-models tabledatabricks/databricks-gpt-5-4:max_input_tokens272,000 -> 922,000 (OpenAI's GPT-5.4 page prints a 1,050,000 context and 128,000 max output; Databricks prints no limit), andsupports_vision,supports_reasoning,supports_function_calling,supports_tool_choiceand the text+image modalities that were missing.gpt-5-4-miniandgpt-5-4-nanoget the same flags;gpt-5,gpt-5-1,gpt-5-2,gpt-5-3-codex,gpt-5-miniandgpt-5-nanogetsupports_function_callingandsupports_tool_choice, all listed on the function-calling page.gpt-5-3-codexalso moves frommode: chattomode: responses, the same "Use the Responses API" note the supported-models page puts on GPT-5.5databricks/databricks-claude-fable-5-1:prompt_cache_min_tokens: 512, the minimum Anthropic publishes for Fable 5.1 at https://platform.claude.com/docs/en/build-with-claude/prompt-caching; the cost-map-wide Fable 5 test requires it on every variantdeprecation_date:claude-3-7-sonnet2026-04-12,claude-sonnet-42026-10-09,gemini-2-5-flash2026-10-02,gpt-5-1-codex-max,gpt-5-1-codex-miniandgpt-5-2-codex2026-07-16,meta-llama-3-1-405b-instruct2026-02-15,mixtral-8x7b-instruct2025-04-30,llama-2-70b-chat2024-10-30,mpt-30b-instructandmpt-7b-instruct2024-08-30,meta-llama-3-70b-instruct2024-07-23. No other Databricks entry got a date because the page lists no other retirementn/a: the registry follows the convention already on the base branch forglm-5-2and every GPT row, cache write equal to the input rate (no surcharge) and cache read equal to the input rate (no discount) when the column isn/a. Rows with a published cached-input rate (GLM-5.3 3.714, GLM-5.3 Flash 0.429, Inkling 2.429) carry that rate.test_every_priced_databricks_model_declares_cache_rateson the base branch requires both fields on every priced Databricks entry, so omitting them was not an option without changing that guaranteegemini-3-8-flash,gemini-3-1-flash-imageandgemini-3-pro-imagecarry limits and modalities but no price, with ametadata.notessaying the DBU rate is not published yet, the same shapeglm-5-3-flashhad on the base branch until its rate appeared. See CaveatsAzure OpenAI realtime (https://learn.microsoft.com/en-us/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure for the model rows, https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule for retirement dates, https://developers.openai.com/api/docs/pricing for the per-token rates Azure charges at OpenAI parity; raw HTML)
azure/gpt-realtime-2,azure/gpt-realtime-2.1andazure/gpt-realtime-2.1-miniinmode: realtimewith the OpenAI rates ($4 / $24 text, $32 / $64 audio, $5 image, $0.40 cached for the two full models; $0.60 / $2.40 text, $10 / $20 audio, $0.80 image, $0.06 cached for mini), the 32,000 input and 4,096 output limits the Azure catalog rows print,/v1/realtimeas the only endpoint, text, image and audio input, text and audio output, function calling ondeprecation_date:azure/gpt-realtime-22026-08-31 (already retired on the schedule),azure/gpt-realtime-2.1andazure/gpt-realtime-2.1-mini2027-06-25, the retirement dates the schedule rows printOpenAI realtime (https://developers.openai.com/api/docs/pricing, https://developers.openai.com/api/docs/models/gpt-realtime-2, https://developers.openai.com/api/docs/models/gpt-realtime-2.1, https://developers.openai.com/api/docs/models/gpt-realtime-2.1-mini; raw HTML)
gpt-realtime,gpt-realtime-2025-08-28,gpt-realtime-1.5,gpt-realtime-2,gpt-realtime-2.1,gpt-realtime-2.1-mini,gpt-realtime-mini-2025-10-06,gpt-realtime-mini-2025-12-15and the Azure twinsazure/gpt-realtime-2025-08-28,azure/gpt-realtime-1.5-2026-02-23,azure/gpt-realtime-mini,azure/gpt-realtime-mini-2025-10-06) moves its image rate frominput_cost_per_imagetoinput_cost_per_image_token. OpenAI prices realtime image input per 1M tokens ($5.00 and $0.80 per 1M on the pricing page) and the API reportsinput_token_details.image_tokens;input_cost_per_imageis the per-image field the calculator multiplies byimage_count, so those tokens were being charged at the text rate. Verified with a real call in the proof section (194 image tokens billed at $0.80 per 1M instead of $0.60) and pinned bytest_realtime_image_tokens_priced_per_tokengpt-realtime-2,gpt-realtime-2.1andgpt-realtime-2.1-minilimits (128,000 in / 32,000 out) and prices were checked against the model pages and match, no changeVertex AI (https://cloud.google.com/gemini-enterprise-agent-platform/models/partner-models/claude/use-claude, raw HTML)
vertex_ai/claude-fable-5-1andvertex_ai/claude-fable-5-1@default: addeddeprecation_date: 2027-03-01. The Vertex page says "Retirement date: No sooner than March 1, 2027". Neither entry had a date, so the proxy was reporting Anthropic's direct-API date instead. All other Vertex Claude dates already match the pageNovita (https://developers.openai.com/api/docs/models/gpt-oss-120b, https://developers.openai.com/api/docs/models/gpt-oss-20b)
novita/openai/gpt-oss-120bandnovita/openai/gpt-oss-20b:supports_visiontrue -> false. OpenAI's model pages list text as the only input modality for both open-weight models; the hosting provider cannot add image inputOpenAI (https://developers.openai.com/api/docs/pricing, https://developers.openai.com/api/docs/models)
gpt-daybreak-blue-latestandgpt-daybreak-red-latest, the model ids OpenAI now documents for the Daybreak aliases. They point atgpt-5.6-sol/gpt-5.6-cyber, are Responses-only per the model pages (Chat Completions is struck through in the raw endpoint list, the same way it is ongpt-5.5-pro), so they usemode: responseslike the other Responses-only OpenAI entries andcompletion()bridges them to/v1/responses. They carry the same prices as those snapshots. The older baredaybreak-*-latestaliases are kept and only theirsourceis repointed at the pages that existgpt-5.5andgpt-5.5-2026-04-23: Fast (priority) tier corrected to $12.50 / $75 per 1M with $1.25 cached input, per the Fast mode row on the pricing page (was $10 / $60 / $1).tests/test_litellm/proxy/spend_tracking/test_savings.pyhard-coded the old 2x priority ratio forgpt-5.5, so its expected multiplier is now the published 2.5xgpt-image-2,gpt-image-2-2026-04-21,azure/gpt-image-2andazure/gpt-image-2-2026-04-21: removedoutput_cost_per_token: 1e-05. The pricing page's GPT Image 2 row has text input $5, cached text $1.25, image input $8, cached image $2 and image output $30 per 1M, and no text output price at all. With the $10 field present,completion_cost()for a usage that reports onlycompletion_tokens(the shape the chat and/spend/calculatepaths produce) billed the generated image at $10 per 1M instead of the published $30, since the calculator only falls back tooutput_cost_per_image_tokenwhen the text output price is absent (generic_cost_per_tokeninlitellm_core_utils/llm_cost_calc/utils.py). Usages with anoutput_tokens_details.image_tokensbreakdown, like the live/v1/images/generationsresponse, already used $30 and are unchanged.tests/test_litellm/test_gpt_image_cost_calculator.pynow pins the image rate for both usage shapes, andtests/test_litellm/test_utils.pyexpectsget_model_infoto reportoutput_cost_per_token: 0for the two entriesAnthropic (https://platform.claude.com/docs/en/models/mythos-5-1/overview, https://platform.claude.com/docs/en/about-claude/pricing)
claude-mythos-5-1with the documented prices, 1M context, 128K output and the capability flags listed on the overview pageNebius Token Factory (https://tokenfactory.nebius.com/api/public/models_info, catalog pages linked per entry)
max_model_lencome from the public models_info endpoint;supports_reasoning,supports_function_callingandsupports_visionare set only where the catalog lists that use casenebius/google/gemma-3-27b-it($0.10 / $0.30, 110K),nebius/Qwen/Qwen3-32B(40,960 context) andnebius/Qwen/Qwen2.5-VL-72B-Instruct($0.25 / $0.75, 32K), and moved theirsourceoff the retired nebius.com/prices pageIBM watsonx (https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-models.html?context=wx, cross-checked with https://www.ibm.com/products/watsonx-ai/pricing)
watsonx/bigscience/mt0-xxl-13b: $0.001908 per 1K tokens both ways (was 0.0005 / 0.002 per token, a 1K to 1 unit slip), context 4,096; also addedwatsonx/bigscience/mt0-xxl, the id IBM actually documentswatsonx/ibm/granite-4-h-small$0.0636 / $0.265 per 1M and 131,072 context,watsonx/openai/gpt-oss-120b$0.159 / $0.636 and 131,072,watsonx/meta-llama/llama-3-3-70b-instruct$0.7526 both ways and 131,072,watsonx/mistralai/mistral-small-3-1-24b-instruct-2503$0.106 / $0.318 and 131,072,watsonx/meta-llama/llama-4-maverick-17b$0.371 / $1.484 and 131,072watsonx/meta-llama/llama-4-maverick-17b-128e-instruct-fp8, the documented API id for Maverickmax_output_tokens(andmax_tokens, whichtest_max_tokens_consistencyrequires to match) follows IBM's model detail pages: the Maverick rows cap new tokens at 8,192 and Mistral Small 3.1 at 16,384, while granite-4-h-small, gpt-oss-120b and llama-3-3-70b-instruct publish only the 131,072 combined window, so output is bounded by thatW&B Inference (https://wandb.ai/site/pricing/inference)
wandb/Qwen/Qwen3-235B-A22B-Instruct-2507,wandb/Qwen/Qwen3-235B-A22B-Thinking-2507,wandb/deepseek-ai/DeepSeek-R1-0528,wandb/deepseek-ai/DeepSeek-V3-0324,wandb/meta-llama/Llama-4-Scout-17B-16E-InstructGemini API (https://ai.google.dev/gemini-api/docs/pricing, https://ai.google.dev/gemini-api/docs/models, raw HTML; 2026-09-03 second run)
gemini/lyria-3.5-clip-preview($0.04 per song) andgemini/lyria-3.5-pro-preview($0.08 per song), the two Lyria 3.5 rows on the pricing page, same shape as the existing Lyria 3 entriesgemini/lyria-3-pro-preview: added the missingoutput_cost_per_image: 0.08; the pricing page lists $0.08 per song and the entry recorded $0gemini-2.5-flash-native-audio-latest,gemini-2.5-flash-native-audio-preview-09-2025,gemini-2.5-flash-native-audio-preview-12-2025,gemini-3.1-flash-live-previewand theirgemini/twins:modechat -> realtime. Every one of them lists only/v1/realtimeinsupported_endpoints, so chat mode contradicted the endpoint list (from fix(pricing): add the Vertex AI Gemini Live GA models and correct the preview #36886)gemini/gemini-3.5-live-translate-preview:modechat -> realtime, plusmax_input_tokens: 131072,max_output_tokens: 65536,max_tokens: 65536andtextadded tosupported_output_modalities. The model page (https://ai.google.dev/gemini-api/docs/models/gemini-3.5-live-translate-preview, raw HTML) lists Live API as its only supported surface, audio in, audio plus text transcript out, 131,072 / 65,536 token limits; the entry had/v1/realtimeas its only endpoint but still said chatVertex AI Live (https://cloud.google.com/vertex-ai/generative-ai/pricing, https://cloud.google.com/vertex-ai/generative-ai/docs/live-api, raw HTML)
gemini-live-2.5-flash-native-audio: addedinput_cost_per_image_token: 3e-06. The Vertex pricing row for the GA Live model prices image and video input at $3.00 per 1M alongside audio; the entry already had the audio and text rates (from fix(pricing): add the Vertex AI Gemini Live GA models and correct the preview #36886)Mistral (https://docs.mistral.ai/inference/pricing, raw HTML)
mistral/ministral-14b-latest,mistral/ministral-14b-2512: addedcache_read_input_token_cost: 2e-08($0.02 cached input per 1M)mistral/ministral-3b-latest,mistral/ministral-3b-2512: addedcache_read_input_token_cost: 1e-08($0.01 cached input per 1M)Nebius Token Factory (https://tokenfactory.nebius.com/api/public/models_info)
nebius/meta-llama/Llama-3.3-70B-Instruct: limits 128,000 -> 131,072, thecontext_lengththe catalog API reportsnebius/moonshotai/Kimi-K3: limits 1,048,576 -> 1,024,000, thecontext_lengththe catalog API reportsPerplexity Agent API (https://docs.perplexity.ai/docs/agent-api/models, model list embedded as JSON in the raw page)
perplexity/<vendor>/<model>Responses-mode entries the page lists and the registry lacked:anthropic/claude-fable-5,claude-opus-5,claude-opus-4-8,claude-sonnet-5,claude-sonnet-4-6;openai/gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5,gpt-5.4,gpt-5.4-mini,gpt-5.4-nano,gpt-5;google/gemini-3.1-pro-preview,gemini-3.1-flash-lite,gemini-3.5-flash,gemini-3.5-flash-lite,gemini-3.6-flash,gemini-3.7-flash;xai/grok-4.6,grok-4.5,grok-4.3,grok-4.20-reasoning,grok-4.20-non-reasoning,grok-4.20-multi-agent;perplexity/glm-5.3,glm-5.3-flash,nemotron-3.5-lightning-30b-a3b,nemotron-3-ultra-550b-a55b. Each carries the page's input, output and cached-input prices, and the*_above_272k_tokenstier where the page publishes a long-context rate. They follow theperplexity/openai/gpt-5.2naming already on the base branchperplexity/openai/gpt-5.2,gpt-5.1,gpt-5-mini,perplexity/anthropic/claude-opus-4-6,claude-opus-4-7,claude-opus-4-5,claude-sonnet-4-5,claude-haiku-4-5,perplexity/google/gemini-3-flash-preview,perplexity/perplexity/sonar) now carry the page's input, output and cached-input prices and asourcesupports_adaptive_thinking(Fable 5 alsothinking_always_on, Opus 5 alsoprompt_cache_min_tokens: 512), matching theperplexity/anthropic/claude-opus-4-7row already on the base branch. The cost-map-wide tests intest_claude_fable_5_config.py,test_claude_opus_5_config.pyandtest_claude_sonnet_5_config.pyrequire the flag on every variantOpenRouter (https://openrouter.ai/api/v1/models, OpenRouter's own catalog API, cross-checked with the upstream provider entries already in the registry)
openrouter/minimax/minimax-m3($0.30 / $1.20 per 1M, $0.06 cached input, 1,048,576 context, 512,000 output, text, image and video input, tools, tool_choice, reasoning and structured outputs persupported_parameters) andopenrouter/qwen/qwen3.7-plus($0.32 / $1.28, $0.064 cache read, $0.40 cache write, 1,000,000 context, 131,072 output, text and image input), the two catalog models that have no upstream entry in the registry, straight from the catalog API's pricing,context_length,top_provider.max_completion_tokens, modalities andsupported_parametersopenrouter/anthropic/claude-fable-5,claude-fable-5.1,claude-opus-4.8,claude-sonnet-5;openrouter/google/gemini-2.5-flash-lite,gemini-3.5-flash,gemini-3.5-flash-lite,gemini-3.6-flash,gemini-3.7-flash,gemini-3.8-flash;openrouter/openai/gpt-4o-mini,gpt-5.1,gpt-5.3-codex,gpt-5.4,gpt-5.4-mini,gpt-5.4-nano,gpt-5.5,gpt-5.6-luna,gpt-5.6-terra,o3,o4-mini;openrouter/x-ai/grok-4.20,grok-4.20-multi-agent,grok-4.3,grok-4.5,grok-4.6,grok-build-0.1. Prices are the catalog API'sprompt,completionandinput_cache_readvalues, which for these first-party models equal the upstream provider list price already in the registry. Token limits are copied from the matching upstream entry (for exampleopenrouter/openai/gpt-5.5usesgpt-5.5's 1,050,000 / 128,000), not from OpenRouter's rounded context figure. Capability flags come from the API'ssupported_parametersand modalities. The four Anthropic entries also carry the bare entry'ssupports_adaptive_thinking: trueandsupports_sampling_params: false(Fable alsothinking_always_onandprompt_cache_min_tokens: 512), which the Fable, Opus 4.8 and Sonnet 5 cost-map sweep tests require on every variant so LiteLLM never sends the legacy thinking shape or rejected sampling params to those modelsdeepseek/deepseek-chat,deepseek-chat-v3-0324,deepseek-r1,deepseek-v3.2,deepseek-v3.2-exp;gryphe/mythomax-l2-13b;mancer/weaver;minimax/minimax-m2.1,minimax-m2.5;mistralai/devstral-2512,mistral-large,mistral-small-3.1-24b-instruct,mistral-small-3.2-24b-instruct,mixtral-8x22b-instruct;moonshotai/kimi-k2.5;nvidia/nemotron-3.5-lightning;openai/gpt-3.5-turbo,gpt-oss-120b,gpt-oss-20b;qwen/qwen-2.5-coder-32b-instruct,qwen3-235b-a22b-2507,qwen3-235b-a22b-thinking-2507,qwen3-coder,qwen3-coder-plus,qwen3.5-122b-a10b,qwen3.5-27b,qwen3.5-397b-a17b,qwen3.5-flash-02-23,qwen3.5-plus-02-15;undi95/remm-slerp-l2-13b;xiaomi/mimo-v2.5,mimo-v2.5-pro;z-ai/glm-4.6,glm-4.7-flash,glm-5,glm-5.1. Several were off by a large factor (for examplegpt-oss-120b$0.05 -> $0.037 in, $0.25 -> $0.17 out;qwen3.5-plus-02-15$0.40 -> $0.26 in). Cached-input prices were added where the API publishesinput_cache_readanthropic/claude-opus-4.5output 32,000 -> 64,000;anthropic/claude-sonnet-4.51,000,000 / 1,000,000 -> 200,000 / 64,000;anthropic/claude-haiku-4.5output 200,000 -> 64,000;google/gemini-2.5-flashandgemini-2.5-prooutput 8,192 -> 65,535;openai/gpt-4ooutput 4,096 -> 16,384;openai/o3-miniando3-mini-high128,000 / 65,536 -> 200,000 / 100,000. Each of these was internally inconsistent with the upstream entry it proxiesprompt,completion,input_cache_readandinput_cache_writevalues, plus the audio and image output rates where it publishes them; limits arecontext_lengthandtop_provider.max_completion_tokens; capability flags come only from the row's ownsupported_parametersandarchitecture.input_modalities, withsupports_pdf_inputset where the row lists thefileinput modality. Skipped from the same pass:openai/gpt-3.5-turbo-0613(retired upstream), theopenai/gpt-5-image*,gpt-5.4-image-2,gpt-5.6-luna-pro,gpt-5.6-sol,gpt-5.6-sol-pro,gpt-5.6-terra-pro,gpt-6-astra-proandgpt-chat-latestrows, and the twogoogle/lyria-3-*rows, none of which LiteLLM's OpenRouter path can serve as a chat entrytests/test_litellm/litellm_core_utils/test_get_model_cost_map.pygains a regression test pinning ten of the corrected prices and a stale-value guard (from fix(catalog): align openrouter prices with live openrouter.ai rates #39591, values re-derived from the live API);tests/test_litellm/test_utils.pyupdates the Kimi K2.5 expectations to the corrected $0.45 / $2.25 / $0.07AWS Bedrock (https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonBedrock/current/us-west-2/index.json, the official AWS Price List JSON; https://aws.amazon.com/bedrock/pricing/ Standard tier Qwen3 table; https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-qwen-qwen3-coder-480b-a35b-instruct.html)
qwen.qwen3-coder-480b-a35b-v1:0:input_cost_per_token2.2e-07 -> 4.5e-07. The us-west-2 offer SKUUSW2-Qwen3Coder-480B-A35B-input-tokens(featureOn-demand Inference, effective 2026-08-01) is $0.00045 per 1K, and the pricing page's Standard tier Qwen3 row resolves to the same $0.45 / $1.80 per 1M in us-west-2 and us-east-2. The output price ($1.80) already matched.sourcepoints at the us-west-2 offer file because the us-east-1 offer only lists the Mantle SKUs for this model and the Runtime id returnsThe provided model identifier is invalidin us-east-1, even though the model card lists that regionFireworks AI (https://docs.fireworks.ai/serverless/pricing, raw HTML; the model detail response from https://api.fireworks.ai/v1/accounts/fireworks/models/deepseek-v4-flash-vision-exp)
fireworks_ai/accounts/fireworks/models/deepseek-v4-flash-vision-expand the shortfireworks_ai/deepseek-v4-flash-vision-expalias, matching the two-key shape the other Fireworks entries use. The serverless pricing row reads $0.22 / $0.007 / $0.66 per 1M, so $2.2e-07 input, $7e-09 cache read and $6.6e-07 output, with the 1,048,576 context and image input the model detail response reports, plus tool calling and tool choice. Nomax_output_tokens, because neither surface publishes one, and nosupports_reasoning, because neither states it even though the live response returnsreasoning_contentBaseten (https://www.baseten.co/pricing/, https://docs.baseten.co/inference/model-apis/overview, https://docs.baseten.co/inference/model-apis/vision)
baseten/zai-org/GLM-5.3: $1.40 / $4.40 per 1M with $0.14 cached input, 1,048,576 context and 262,144 output per the Model APIs overview, tool calling, prompt caching and response schema on.supports_vision: truewithtext, imageinput modalities: the overview's feature-support table ticks Vision for GLM 5.3 and the vision page listszai-org/GLM-5.3under supported models withimage_urlinput (images only, novideo_url).tests/test_litellm/test_baseten_glm_5_3_model_metadata.pycovers the entry (from fix(pricing): add baseten/zai-org/GLM-5.3 to the model pricing registry #39603).supports_reasoningis deliberately absent because the Baseten chat path does not acceptreasoning_effortAbsorbed from other open registry PRs
Field-level merge only: each source PR was diffed against its own merge base and only the fields it changed were re-verified and applied, so nothing newer on
litellm_internal_stagingis droppedsupports_visionflips are in, verified against OpenAI's model pagesmax_output_tokens/max_tokens32,768 anddeprecation_date2027-08-23, which neither the catalog page nor the retirement schedule publishesCodestral-2501,MAI-Thinking-1andmistral-ocr-4-0are in with the fields Azure publishes (MAI-Thinking-1 drops the 2026-11-04 date and mistral-ocr-4-0 drops the 2027-10-01 date and token limits; none appear on the schedule or catalog page). Not absorbed:FW-GLM-5.3,FW-DeepSeek-V4-Flash-0731andgrok-4.6(no Azure meter exists for any of them in the Retail Prices API and the pricing pages have no row),Kimi-K3(the only Azure meters areFW Kimi K3 ... DZat $3.30 / $16.50 / $0.33 per 1M, which do not match the PR's $3 / $15 / $0.30),MAI-Image-2.5-Pro(the PR's 2026-10-31 date conflicts with the schedule's 2026-10-01 and the price shape does not match the sibling MAI-Image-2.5 entries)litellm_internal_stagingon its own (29ea2bd) and picked up here through the base merge, nothing to absorbgpt-5-4row and the directgpt-5.6-solentry),mode: chatongpt-5-5andgpt-5-5-pro(the supported-models page says "Use the Responses API to interact with this model", soresponses; the same note sits on GPT-5.3 Codex, so the base branch'sgpt-5-3-codexrow moves toresponsestoo),max_output_tokensoninkling,grok-4-6andqwen3-next-80b-a3b-instructand the 262,144 context on Qwen3-Next (neither Databricks nor the vendor pages print a limit for them), text-only modalities oninklingandclaude-fable-5-1(Databricks liststext, imagefor both), and the 1,048,576 context, 65,536 output, prompt caching and tool flags on the two Gemini image models (Databricks says they do not support function calling, and Google's model pages print 131,072 / 65,536 input and 32,768 output, https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-image and https://ai.google.dev/gemini-api/docs/models/gemini-3-pro-image)watsonx/bigscience/mt0-xxl-13bare in. The watsonx row is re-verified against the IBM page above; the W&B pricing page fetched on 2026-09-04 no longer prints those five models, so their rows keep the digits the registry already had and only correct the divisor (per 1M, not per 100). IBM lists $0.001908, not the $0.0018 in that PR, so the official number is used. Not absorbed:azure_ai/jais-30b-chat(no public per-token price found on an official page),watsonx/core42/jais-13b-chat(no longer in IBM's model table), and the other W&B rows whose values already match the pricing pagelitellm_internal_staging, and the rest of the PR is a Python typing change, so it is left alonesupports_vision: false(Baseten's overview table and vision page both say GLM 5.3 takes images, so the entry says true),supports_reasoningandsupports_parallel_function_calling(the Baseten chat path dropsreasoning_effortandparallel_tool_calls, see the metadata test), and thedocs.baseten.co/inference/model-apis/overviewsource URL, since the pricing page is where the numbers come fromsourceand the retirement dates the PR lacks: the threeazure/gpt-realtime-2*entries, theinput_cost_per_imagetoinput_cost_per_image_tokenmove on the Azure realtime rows, and thegpt-realtime-2128,000 / 32,000 limits and $24 per 1M output. Nothing left to absorbmode: realtimeflips and theinput_cost_per_image_tokenongemini-live-2.5-flash-native-audioare in. Not absorbed: the newgemini-live-2.5-flashandvertex_ai/gemini-live-2.5-flash*entries (the Vertex pricing table lists only the native-audio GA model), the 2.5 Flash Native Audio price cuts to $0.30 / $1.00 / $2.50 (the raw Gemini pricing table still shows $0.50 / $3.00 / $2.00), the audio-onlysupported_output_modalitiesand the region lists, none of which the linked pages stateLeft alone as code or behavior PRs that only incidentally touch the JSON, or as registry changes with no official source yet: #39610 (Bedrock cost calculation code), #39607 (GPT-6 Astra OpenAI and Azure transformation code), #39427 (Azure AI OCR implementation), #39414 (Vertex DeepSeek OCR cost code), #39395 (Meta realtime handler), #38165 (Dashscope types and utils), #37991 (Vertex implementation), #39115, #35617, #34609 (Bedrock strict-tools flags, not in Bedrock docs), #37972 (EmpirioLabs, no official pricing page), #36385 and #33930 (Bedrock Mantle limits and Gemma, not in Bedrock docs), #36279 and #36274 (Azure GPT-5.6 prices, no Azure meter yet), #35720 (ChatGPT models, no OpenAI pricing row), #36084 and #34494 (deletion-only), #35446 (Kimi K3 output limit, not on Fireworks page), #33765 (Azure GPT-5.5 snapshot deletions), #33920 (Anthropic web-search flags, behavior change), #33618 (Azure audio aliases, one remaining
modechange is a behavior change), #34941 (MAI image cost calculator code), #34620 (WaveSpeed provider code), #36422 and #36416 (fallback_generalizations file layout, no model values), #39732, #37103 and #34623 (new provider code with their own catalog rows), #35737 (MiniMax image generation code), #36233 (Vertex Chirp 3 STT/TTS provider code), #29898 (Bedrock invoke output_format behavior), #30383 (router cost backfill code)Checked and deliberately left unchanged
deprecation_datevalues match the raw pagesmistral-document-ai-2512: registry has $3 per 1K pages, Retail Prices API shows $2 / $2.50 forOCR 2512 glbl Pages. Left as is because the catalog page it cites cannot be read without JavaScript, so the two official surfaces cannot be reconciled hereMiniMaxAI/MiniMax-M2.7andQwen/Qwen2.5-7B-Instruct-Turbo: both appear in https://api.together.xyz/v1/models with per-token prices, but neither is on https://docs.together.ai/docs/serverless-models and a live call to each returnsUnable to access non-serverless model ... create and start a new dedicated endpoint. Dedicated endpoints bill per GPU-minute, so the catalog's per-token figures are not a serverless list price. The MiniMax entry was pulled back out and the existing Qwen2.5 7B Turbo entry keeps its flags with no priceQwen/Qwen3.8-2.4T-A95Bcached input: docs say $0.50, pricing page says $0.25; the registry matches the pricing page, so no change until Together agrees with itselfdeep-research-*andantigravity-preview-*: billed at the underlying model's list rate, no per-model price to recordcommand-a-plus-05-2026,rerank-v4.0-*and the Mistral additions were skipped because the pricing pages do not publish per-token prices for them yetgrok-4.20-multi-agentkeepssupports_function_calling: falseeven though the catalog says true, because the API rejects client-side tools (02f7873). Video, TTS and STT rows are skipped, LiteLLM has no xAI path for themminimaxai/minimax-m2.7is "Contact Sales" pricing, so it is not addeddeprecation_datebeyond the retired-models page, in particular none forgemini-2-5-proor the Claude 4.x rows still listed as supported. The workspace used for the Databricks proof below hasdatabricks-gpt-oss-20bprovisioned and neither GPT-5 Responses endpoint, so the catalog itself was read from the pages, not from a live workspacePre-Submission checklist
test_realtime_image_tokens_priced_per_tokenfails on the merge base where image tokens bill at the text rate; the Databricks cost calculator tests pin the 19 new DBU rows)LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest -q tests/test_litellm/test_model_prices_schema.py tests/test_litellm/test_model_cost_aliases.py tests/test_litellm/litellm_core_utils/test_get_model_cost_map.py tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py tests/test_litellm/llms/databricks/ tests/test_litellm/llms/azure_ai/test_azure_ai_cost_calculator.py tests/test_litellm/llms/azure_ai/test_azure_ai_fw_models_metadata.py tests/test_litellm/llms/mistral/ocr/test_mistral_ocr_cost.py tests/test_litellm/proxy/spend_tracking/test_savings.py tests/test_litellm/test_baseten_glm_5_3_model_metadata.py tests/test_litellm/test_cost_calculator.py tests/test_litellm/test_daybreak_model_metadata.py tests/test_litellm/test_gpt_image_cost_calculator.py tests/test_litellm/test_utils.py(every test file this PR touches: 1,131 passed, 13 skipped at f5157a6, and the Baseten metadata test passes again at 50d6b26 with its newsupports_visionandtext, imageassertions; at 1f0611atest_aaamodel_prices_and_context_window_json_is_validfailed because the OpenRouter Qwen rows introducedcache_*_input_token_cost_above_128k_tokensand_above_256k_tokens, which the last commit adds to that test's field list),python3 ci_cd/check_files_match.py(root and backup identical, 3,814 entries each),uv run --no-sync python ci_cd/generate_model_prices_schema.py(schema unchanged),make lint(exit 0 at 8e83d6d, 0c29f51, 1f0611a and 50d6b26),make check(PASS at f5157a6; 50d6b26 on top only changes two registry flags and the test that pins them)local_testing_part1/part2fail onbedrock/cohere.command-r-plus-v1:0with an AWS 404 for the us-west-2 invoke URL, identically on staging's own pipeline (jobs 2153247 and 2153213), and this PR does not touch that entry. The non-requiredosv-scanjob fails on every open PR on a gitpython advisory inuv.lock, which this PR does not touchDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Two proxies, one from the merge base dd01abc on port 42131 (detached worktree, database
litellm_qa_base) and one from the tip 50d6b26 on port 42132 (databaselitellm_qa_head), booted withLITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config qa_config.yaml --port $PORT --num_workers 2 --use_v2_migration_resolver. Sameqa_config.yamlfor both,master_key: sk-qa-1234,drop_params: true:The realtime case connects to
ws://localhost:$PORT/v1/realtime?model=gpt-realtime-2.1-mini, sends oneconversation.item.createwithinput_text"What color is this square? One word." plus a generated 64x64 solid blue PNG asinput_image, thenresponse.createwithoutput_modalities: ["text"], and prints theresponse.doneusage. The Fireworks case postsreq_fw.json: the same question with a generated 64x64 solid red PNG as a base64image_urlandmax_tokens: 16. Spend rows are read back through the proxy's ownGET /spend/logs. The placeholder-key deployments are only read through/model/info,/model_group/info,/model/deprecationsand/spend/calculate, which never call the providerBefore (dd01abc)
/model/info
curl -s -H "Authorization: Bearer sk-qa-1234" localhost:42131/model/info | jq -r '.data[] | "\(.model_name)\tmode=\(.model_info.mode)\tinput=\(.model_info.input_cost_per_token)\toutput=\(.model_info.output_cost_per_token)\tcache_write=\(.model_info.cache_creation_input_token_cost)\tcache_read=\(.model_info.cache_read_input_token_cost)\timg_tok=\(.model_info.input_cost_per_image_token)\timg=\(.model_info.input_cost_per_image)\tmax_input=\(.model_info.max_input_tokens)\tmax_output=\(.model_info.max_output_tokens)\tvision=\(.model_info.supports_vision)"' | sort/model/deprecations
curl -s -H "Authorization: Bearer sk-qa-1234" localhost:42131/model/deprecations | jq -c '{deprecated: [.deprecated[] | {model_name, deprecation_date}], upcoming: [.upcoming[] | {model_name, deprecation_date}]}'Real OpenAI realtime call with an image, gpt-realtime-2.1-mini
response.doneusage:Real Mistral call, ministral-14b
curl -s -D - -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42131/v1/chat/completions -d '{"model":"ministral-14b","messages":[{"role":"user","content":"Reply with the single word: pong"}],"max_tokens":8,"temperature":0}'Real Fireworks call with an image, deepseek-v4-flash-vision-exp
curl -s -D - -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42131/v1/chat/completions -d @req_fw.jsonx-litellm-response-costheader for it because the registry has no row for the modelReal OpenAI image generation, gpt-image-2
curl -s -D - -o img.json -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42131/v1/images/generations -d '{"model":"gpt-image-2","prompt":"a plain red square","n":1,"size":"1024x1024","quality":"low"}' | grep -i "^HTTP\|x-litellm-response-cost:"; jq -c '{data_len: (.data|length), usage}' img.json/spend/calculate for a gpt-image-2 usage with no output token breakdown
curl -s -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42131/spend/calculate -d '{"completion_response":{"id":"img-1","object":"chat.completion","created":1,"model":"gpt-image-2","choices":[{"index":0,"message":{"role":"assistant","content":""},"finish_reason":"stop"}],"usage":{"prompt_tokens":100,"completion_tokens":5000,"total_tokens":5100}}}'/spend/calculate for openrouter/qwen/qwen3.7-flash above and below the 256k tier
curl -s -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42131/spend/calculate -d '{"completion_response":{"id":"t1","object":"chat.completion","created":1,"model":"openrouter/qwen/qwen3.7-flash","choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":300000,"completion_tokens":1000,"total_tokens":301000,"prompt_tokens_details":{"cached_tokens":100000}}}}', then the same call with"prompt_tokens":10000,"completion_tokens":100,"total_tokens":10100and"cached_tokens":4000/spend/calculate for the placeholder deployments
for m in dbx-glm-5.3-flash dbx-gpt-5.6-sol dbx-qwen3.5-122b bedrock-qwen3-coder-480b codestral-2501 openrouter-minimax-m2.7 together-qwen2.5-7b; do printf "%s " "$m"; curl -s -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42131/spend/calculate -d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply with the single word: pong\"}]}"; echo; done/model_group/info
for g in dbx-gpt-5.6-sol dbx-gpt-5.3-codex azure-realtime-2.1-mini fw-dsv4-vision openrouter-minimax-m2.7 baseten-glm-5.3; do curl -s -H "Authorization: Bearer sk-qa-1234" "localhost:42131/model_group/info?model_group=$g" | jq -c '.data[] | {model_group, mode, supports_vision, supports_function_calling, supports_reasoning, max_input_tokens, max_output_tokens, input_cost_per_token, output_cost_per_token}'; done/spend/logs for the four real calls
curl -s -H "Authorization: Bearer sk-qa-1234" localhost:42131/spend/logs | jq -c 'sort_by(.startTime) | .[] | {model, call_type, prompt_tokens, completion_tokens, spend}'After (50d6b26)
/model/info
curl -s -H "Authorization: Bearer sk-qa-1234" localhost:42132/model/info | jq -r '.data[] | "\(.model_name)\tmode=\(.model_info.mode)\tinput=\(.model_info.input_cost_per_token)\toutput=\(.model_info.output_cost_per_token)\tcache_write=\(.model_info.cache_creation_input_token_cost)\tcache_read=\(.model_info.cache_read_input_token_cost)\timg_tok=\(.model_info.input_cost_per_image_token)\timg=\(.model_info.input_cost_per_image)\tmax_input=\(.model_info.max_input_tokens)\tmax_output=\(.model_info.max_output_tokens)\tvision=\(.model_info.supports_vision)"' | sort/model/deprecations
curl -s -H "Authorization: Bearer sk-qa-1234" localhost:42132/model/deprecations | jq -c '{deprecated: [.deprecated[] | {model_name, deprecation_date}], upcoming: [.upcoming[] | {model_name, deprecation_date}]}'Real OpenAI realtime call with an image, gpt-realtime-2.1-mini
response.doneusage:Real Mistral call, ministral-14b
curl -s -D - -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42132/v1/chat/completions -d '{"model":"ministral-14b","messages":[{"role":"user","content":"Reply with the single word: pong"}],"max_tokens":8,"temperature":0}'/model/infoabove now shows thecache_readrate of 2e-08 that the base was missingReal Fireworks call with an image, deepseek-v4-flash-vision-exp
curl -s -D - -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42132/v1/chat/completions -d @req_fw.jsonReal OpenAI image generation, gpt-image-2
curl -s -D - -o img.json -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42132/v1/images/generations -d '{"model":"gpt-image-2","prompt":"a plain red square","n":1,"size":"1024x1024","quality":"low"}' | grep -i "^HTTP\|x-litellm-response-cost:"; jq -c '{data_len: (.data|length), usage}' img.jsonoutput_tokens_details.image_tokens, so this path already used the $30 image rate on both sides/spend/calculate for a gpt-image-2 usage with no output token breakdown
curl -s -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42132/spend/calculate -d '{"completion_response":{"id":"img-1","object":"chat.completion","created":1,"model":"gpt-image-2","choices":[{"index":0,"message":{"role":"assistant","content":""},"finish_reason":"stop"}],"usage":{"prompt_tokens":100,"completion_tokens":5000,"total_tokens":5100}}}'output_cost_per_token: 1e-05is gone/spend/calculate for openrouter/qwen/qwen3.7-flash above and below the 256k tier
curl -s -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42132/spend/calculate -d '{"completion_response":{"id":"t1","object":"chat.completion","created":1,"model":"openrouter/qwen/qwen3.7-flash","choices":[{"index":0,"message":{"role":"assistant","content":"ok"},"finish_reason":"stop"}],"usage":{"prompt_tokens":300000,"completion_tokens":1000,"total_tokens":301000,"prompt_tokens_details":{"cached_tokens":100000}}}}', then the same call with"prompt_tokens":10000,"completion_tokens":100,"total_tokens":10100and"cached_tokens":4000cache_read_input_token_cost_above_256k_tokensfield is picked up by the tier loop ingeneric_cost_per_token. Below the tier, 6,000 x 3e-8 + 4,000 x 6e-9 + 100 x 1.3e-7 = 0.000217, the base rates/spend/calculate for the placeholder deployments
for m in dbx-glm-5.3-flash dbx-gpt-5.6-sol dbx-qwen3.5-122b bedrock-qwen3-coder-480b codestral-2501 openrouter-minimax-m2.7 together-qwen2.5-7b; do printf "%s " "$m"; curl -s -H "Authorization: Bearer sk-qa-1234" -H 'Content-Type: application/json' localhost:42132/spend/calculate -d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply with the single word: pong\"}]}"; echo; donetogether-qwen2.5-7bis identical on both sides: its Together catalog price was reverted before this tip because the live endpoint reports the model as non-serverless (see the unchanged section above)/model_group/info
for g in dbx-gpt-5.6-sol dbx-gpt-5.3-codex azure-realtime-2.1-mini fw-dsv4-vision openrouter-minimax-m2.7 baseten-glm-5.3; do curl -s -H "Authorization: Bearer sk-qa-1234" "localhost:42132/model_group/info?model_group=$g" | jq -c '.data[] | {model_group, mode, supports_vision, supports_function_calling, supports_reasoning, max_input_tokens, max_output_tokens, input_cost_per_token, output_cost_per_token}'; donebaseten-glm-5.3now reportssupports_vision: truewith the Baseten limits and prices; the base has no row for the model at all/spend/logs for the four real calls
curl -s -H "Authorization: Bearer sk-qa-1234" localhost:42132/spend/logs | jq -c 'sort_by(.startTime) | .[] | {model, call_type, prompt_tokens, completion_tokens, spend}'Observed on both sides, none caused or changed by this PR:
Databricks Responses-API mode against a real workspace (2026-09-04)
The same two commits, booted again with
LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config dbx_config.yaml --port $PORT --num_workers 2and no database: merge base dd01abc on port 49137, tip 50d6b26 on port 57918, a real Databricks workspace token inDATABRICKS_API_KEY,master_key: sk-qa-1234,drop_params: true. Every proxy call below also sends-H "Authorization: Bearer sk-qa-1234" -H "Content-Type: application/json"What the workspace itself answers, with no proxy in between (it has
databricks-gpt-oss-20bprovisioned and no GPT-5 Responses endpoint):Before (dd01abc)
Databricks rejects the proxy's
/chat/completionscall for both GPT rows with its "only supports the Responses API" 400, while the provisioned control model answersAfter (50d6b26)
Both GPT rows now report
mode: responses, the same/v1/chat/completionscalls are sent to the workspace's/responsesendpoint (the 404 is the workspace's own ENDPOINT_NOT_FOUND from the raw calls above, not a routing error),/v1/responsesbehaves the same on both legs, and the control model still answers. A bridged reply end to end needs a workspace with the GPT-5 Responses endpoint provisionedType
🐛 Bug Fix
Caveats (if any)
Low
gpt-5-5,gpt-5-5-proandgpt-5-3-codexflip tomode: responses/chat/completionscalls with "only supports the Responses API" (proof above)/responses; the QA workspace has no such endpoint, so a full bridged reply still needs one that doesgemini-3-8-flash,gemini-3-1-flash-imageandgemini-3-pro-imagecarry no pricemetadata.notessays so, and a per-deployment cost override works meanwhiledeprecation_datevalues are already in the past/model/deprecationsreads the fieldqwen/qwen3.5-35b-a3boutput is $1.25 here, $0.75 now;undi95/remm-slerp-l2-13binput is $0.45 here, $0.35 now (both were further off on the base)n/acache columns are stored as the input rate (no surcharge, no discount), the conventionglm-5-2and the GPT rows already follow on the basereasoning_tokenson top ofoutput_tokenson both legs; pre-existing calculator behavior, not touched hereazure/gpt-realtime-2is past its 2026-08-31 retirement, so/model/deprecationslists it as deprecated on day onegpt-image-2budget reservation is $0 before the call and reconciles from real spend after it, the shapegpt-image-1and the Azure GPT Image rows already have; not exercised liveproxy-infra / Run tests (Python 3.10)is red ontest_qualifiers_and_optionality_are_unwrappedci/circleci: proxy_store_model_in_db_testsis red ontest_chat_completion_bad_model_with_spend_logs(model_groupcomes back empty)litellm_internal_stagingpipeline today (#89085 through #89104) and eleven of the last twelve PR pipelines; not a required check, not touched hereFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/38fb47b2a0e448df985f90f107e4939c
Open in Devin Desktop: https://app.devin.ai/desktop/session/38fb47b2a0e448df985f90f107e4939c?variant=devin
Requested by: @mateo-berri
Note
Medium Risk
Changes are mostly registry data but directly drive global spend, deprecations, and API mode (e.g. Databricks Responses bridge); incorrect rows would misbill customers or route chat calls unexpectedly.
Overview
This PR refreshes
model_prices_and_context_window.json(and the matching backup) from provider pricing pages: hundreds of new model rows, corrected rates/limits/modes/deprecation dates, and absorbed registry-only work from several open PRs. Proxy/model/info,/model/deprecations, and/spend/calculatetherefore stop reporting $0 or wrong retirement dates for models that were missing or stale.Spend calculation fixes move realtime image input from
input_cost_per_imagetoinput_cost_per_image_tokensoinput_token_details.image_tokensbill at the published per‑1M image rate (pinned bytest_realtime_image_tokens_priced_per_token). For GPT Image 2, it drops a strayoutput_cost_per_tokenso completions without an image-token breakdown useoutput_cost_per_image_token($30/1M) instead of the old $10/1M text rate. Other notable registry-driven billing fixes include W&B entries stored per-token instead of per‑1M (100,000× overcount), Bedrock Qwen3 Coder 480B input price, and OpenRouter headline rate corrections with a regression test intest_get_model_cost_map.py.Metadata / routing changes include Gemini Live models set to
mode: realtime, new Azure realtime 2.x rows, Databricks catalog additions with DBU pricing and somemode: responsesrows (e.g. GPT‑5.3 Codex, GPT‑5.5 on Databricks),gpt-5.5Fast/priority tier rates, and new Daybreak / Perplexity Agent / Nebius / Baseten GLM‑5.3 entries. Tests across cost calculators, Databricks, Azure AI, savings, and model schema are updated to match.Reviewed by Cursor Bugbot for commit 50d6b26. Bugbot is set up for automated code reviews on this repo. Configure here.