fix(models): registry audit 2026-09-01: openai realtime and long-context tiers, mistral aliases, voyage, xai, fireworks, together, scaleway, azure ai, govcloud, azure gov, cloudflare whisper, deprecation dates - #39170
Conversation
…liases, voyage, xai, fireworks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
|
PR #39170 (BerriAI/litellm, author devin-ai-integration[bot]) has labels |
Greptile SummaryUpdates the model registry and its packaged backup with current provider pricing, context limits, capabilities, aliases, and deprecation dates.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Updates the canonical model registry with provider pricing, capabilities, limits, aliases, and retirement metadata; no eligible follow-up issue was identified. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the canonical registry updates in the packaged runtime fallback. |
| tests/test_litellm/test_openai_service_tier_long_context_pricing.py | Adds local coverage for OpenAI flex and priority pricing above the 272k-token threshold. |
| tests/test_litellm/test_bedrock_usgov_pricing.py | Pins GovCloud regional and geo-inference-profile pricing and provider mappings. |
| tests/test_litellm/test_cloudflare_workers_ai_model_metadata.py | Adds metadata and per-second pricing coverage for Cloudflare Whisper models. |
| tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py | Updates cost-field and service-tier expectations to cover the expanded registry schema. |
| whitelisted_bedrock_models.txt | Adds the new regional GovCloud Bedrock identifiers required by model discovery. |
Reviews (4): Last reviewed commit: "fix(models): add the us-gov. geo inferen..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…itellm_registry_audit_2026_09_01
…itellm_registry_audit_2026_09_01
…zure gov, openai tiered long-context, scaleway, together qwen3.8, azure ai cache and kimi k2.7 code, azure mai deprecations Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…s pending a Government schedule source Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…cloud pricing tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit da23e02. Configure here.
…itellm_registry_audit_2026_09_01
|
HI, coming from one of the closed prs #38789 - devin flagged the us-gov govcloud inference profiles as unverifiable, but this is because amazon's docs are inconsistent and out of date (or maybe intentional, who knows) (i do have access to govlcoud, would be happy to open a separate pr) One way to check is if the region availability for bedrock-runtime has GEO listed for us-gov-west-1 and us-gov-east-1, then there's an inference profile (with us-gov. prefix instead of us. ) And sometimes they just list it under its own section on the model card |
…nnet 5 and Opus 4.8
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 7a35c34. Configure here.
|
if you can get us-gov.anthropic.claude-opus-5 in here too..it's not listed, but it does exist! |
|
Thanks, the raw model card tables confirm Geo support in both GovCloud regions. Both |
3 participants
TLDR
Problem this solves:
magistral-*-latestandmistral-mediumprice retired modelsus-gov.Claude geo profiles bill at the commercial rateHow it solves it:
deprecation_dateadded only where the provider publishes the date (feat(proxy): proactive model deprecation alerts and/model/deprecationsendpoint #26900)User Flow
Before: a team routing Together Qwen3.8, GPT-5.6 Luna priority long prompts, Cloudflare Whisper or GovCloud Bedrock through the proxy gets wrong or zero spend, and nothing tells them Kimi K2.7 Code retires in a month
together_ai/Qwen/Qwen3.8-2.4T-A95B,openai/gpt-5.6-luna,azure_ai/kimi-k2.7-code,azure_ai/MAI-Image-2.5,bedrock/us-gov-west-1/meta.llama3-8b-instruct-v1:0,bedrock/us-gov.anthropic.claude-sonnet-5,azure/us-gov/gpt-5.1andcloudflare/@cf/openai/whispertomodel_listand starts the proxyimminentlist is emptyqwen3.8at $2.50 / $6.25 per 1M,kimi-k2.7-code,azure-gov-gpt-5.1andcf-whisperat $0,gov-llama3-8boutput at $2.65 per 1M,gov-claude-sonnet-5-geoat the commercial $2.00 / $10.00,gpt-5.6-lunawith no priority rate above 272k"model": "qwen3.8"and get 200 withx-litellm-response-cost: 0.000245"service_tier": "priority"to POST http://localhost:4000/v1/chat/completions, POST http://localhost:4000/v1/responses and POST http://localhost:4000/v1/messages forgpt-5.6-luna; each returns 200 withx-litellm-response-costaround 0.01174, the standard priority rate, half what OpenAI bills above 272kAfter: the same calls return the provider's real prices, and the retirement dates show up
model_listand starts the proxyimminent: [mai-image-2.5 2026-10-01, kimi-k2.7-code 2026-10-03]qwen3.8at $2.00 / $6.00 per 1M,kimi-k2.7-codeat $0.95 / $4.00,azure-gov-gpt-5.1at $1.71875 / $13.75,cf-whisperat 7.5e-06 per second,gov-llama3-8boutput at $0.60 per 1M,gov-claude-sonnet-5-geoat $2.40 / $12.00,gpt-5.6-lunawith priority rates above 272k"model": "qwen3.8"returns 200 withx-litellm-response-cost: 0.000212x-litellm-response-costaround 0.02345, the priority long-context rateChanges by provider
Every number was read from the provider's raw HTML, JSON pricing feed or model API, not from a summarizer
OpenAI
Sources: https://developers.openai.com/api/docs/pricing, https://developers.openai.com/api/docs/guides/fast-mode, https://developers.openai.com/api/docs/guides/flex-processing, model pages for gpt-realtime-2, gpt-realtime-2.1-mini and gpt-realtime-mini
"gpt-realtime-2": { + "cache_read_input_audio_token_cost": 4e-07, - "max_input_tokens": 32000, "max_output_tokens": 4096, "max_tokens": 4096, + "max_input_tokens": 128000, "max_output_tokens": 32000, "max_tokens": 32000, - "output_cost_per_token": 1.6e-05, + "output_cost_per_token": 2.4e-05, "gpt-realtime-2.1-mini": { - "max_output_tokens": 4096, "max_tokens": 4096, + "max_output_tokens": 32000, "max_tokens": 32000, "gpt-realtime-mini": { - "max_input_tokens": 128000, + "max_input_tokens": 32000,Long-context service tiers (absorbed from #39245). The pricing page lists a separate row for prompts above 272k tokens on each tier. Priority rates above 272k are added for
gpt-5.6,gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-luna; flex rates above 272k forgpt-5.5,gpt-5.5-2026-04-23,gpt-5.4,gpt-5.4-2026-03-05,gpt-5.4-pro,gpt-5.4-pro-2026-03-05"gpt-5.6": { + "input_cost_per_token_above_272k_tokens_priority": 1.6e-05, + "cache_read_input_token_cost_above_272k_tokens_priority": 1.6e-06, + "cache_creation_input_token_cost_above_272k_tokens_priority": 2e-05, + "output_cost_per_token_above_272k_tokens_priority": 6e-05, "gpt-5.6-luna": { + "input_cost_per_token_above_272k_tokens_priority": 8e-07, + "cache_read_input_token_cost_above_272k_tokens_priority": 8e-08, + "cache_creation_input_token_cost_above_272k_tokens_priority": 1e-06, + "output_cost_per_token_above_272k_tokens_priority": 3.6e-06, "gpt-5.4": { + "input_cost_per_token_above_272k_tokens_flex": 2.5e-06, + "cache_read_input_token_cost_above_272k_tokens_flex": 2.5e-07, + "output_cost_per_token_above_272k_tokens_flex": 1.125e-05,Tests:
tests/test_litellm/test_openai_service_tier_long_context_pricing.py(new, from #39245) pins the tier fields, the flex and priority ratios, root and backup parity, and the cost at ~300k prompt tokens.test_llm_cost_calc_utils.pygets the updated cost-field count (27) and the fast-tier rates that now existMistral
Sources:
GET https://api.mistral.ai/v1/models, https://docs.mistral.ai/models/model-cards/mistral-medium-3-5-26-04, https://docs.mistral.ai/models/model-cards/mistral-small-4-0-26-03Mistral's model API lists
magistral-medium-latestandmistral-mediumas aliases ofmistral-medium-latest(Medium 3.5), andmagistral-small-latestas an alias ofmistral-small-2603(Small 4). Magistral 1.2 retired on 2026-07-31. The three alias entries are now copies of the entries they resolve to"mistral/magistral-medium-latest": { - "input_cost_per_token": 2e-06, "output_cost_per_token": 5e-06, "max_input_tokens": 40000, + "input_cost_per_token": 1.5e-06, "output_cost_per_token": 7.5e-06, "max_input_tokens": 262144, "cache_read_input_token_cost": 1.5e-07, "supports_vision": true, "mistral/mistral-medium": { - "input_cost_per_token": 2.7e-06, "output_cost_per_token": 8.1e-06, "max_input_tokens": 32000, "max_output_tokens": 8191, + "input_cost_per_token": 1.5e-06, "output_cost_per_token": 7.5e-06, "max_input_tokens": 262144, "max_output_tokens": 262144, "supports_function_calling": true, "supports_reasoning": true, "supports_vision": true, "mistral/magistral-small-latest": { - "input_cost_per_token": 5e-07, "output_cost_per_token": 1.5e-06, "max_input_tokens": 40000, + "input_cost_per_token": 1.5e-07, "output_cost_per_token": 6e-07, "max_input_tokens": 262144, "cache_read_input_token_cost": 1.5e-08, "supports_vision": true,Voyage, xAI, Fireworks
Sources: https://docs.voyageai.com/docs/pricing and https://docs.voyageai.com/docs/reranker (rerank-3 $0.05, rerank-3-lite $0.02 per 1M, 32k context);
GET https://api.x.ai/v1/language-models(grok-build-latestaliasesgrok-4.5) with prices at https://docs.x.ai/developers/models; https://docs.fireworks.ai/serverless/pricing (GLM 5.3 Flash $0.15 / $0.03 cached / $0.50, Inkling $1.00 / $0.17 cached / $4.05 per 1M)New entries
voyage/rerank-3,voyage/rerank-3-lite,xai/grok-build-latest(copy ofxai/grok-4.5),fireworks_ai/accounts/fireworks/models/glm-5p3-flash,fireworks_ai/accounts/fireworks/models/inklingTogether (absorbed from #39278)
Source: https://docs.together.ai/docs/serverless-models, Qwen3.8 2.4T-A95B row reads $2.00 input, $6.00 output, $0.25 cached input per 1M
"together_ai/Qwen/Qwen3.8-2.4T-A95B": { - "input_cost_per_token": 2.5e-06, "output_cost_per_token": 6.25e-06, "cache_read_input_token_cost": 5e-07, + "input_cost_per_token": 2e-06, "output_cost_per_token": 6e-06, "cache_read_input_token_cost": 2.5e-07,Scaleway (absorbed from #38502)
Sources: https://www.scaleway.com/en/pricing/model-as-a-service/ (raw price table) and https://www.scaleway.com/en/docs/generative-apis/reference-content/supported-models/ (context, capabilities)
New
scaleway/glm-5.2($1.80 / $5.50 per 1M) andscaleway/deepseek-v4-flash-0731($0.40 / $0.80, $0.08 cached input per 1M). The source PR only edited the root file; the backup copy is updated here tooAzure AI Foundry (absorbed from #38976)
Sources: https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/deepseek/ and https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/kimi/ (the embedded pricing JSON, read row by row), https://ai.azure.com/catalog/models/Kimi-K2.7-Code, https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule
"azure_ai/deepseek-v4-pro": { + "cache_read_input_token_cost": 1.45e-07, + "supports_prompt_caching": true } "azure_ai/deepseek-v4-flash": { + "cache_read_input_token_cost": 2.8e-08, + "supports_prompt_caching": true } "azure_ai/deepseek-v4-flash-0731": { - "input_cost_per_token": 1.9e-07, "output_cost_per_token": 5.1e-07, "cache_read_input_token_cost": 2.8e-08, + "input_cost_per_token": 4.4e-07, "output_cost_per_token": 1.32e-06, "cache_read_input_token_cost": 1.4e-08, "azure_ai/kimi-k2.5": { + "cache_read_input_token_cost": 1e-07, + "supports_prompt_caching": true, source -> kimi pricing page } "azure_ai/kimi-k2.6": { + "cache_read_input_token_cost": 1.6e-07, + "supports_prompt_caching": true, source -> kimi pricing page } +"azure_ai/kimi-k2.7-code": { "input_cost_per_token": 9.5e-07, "output_cost_per_token": 4e-06, "cache_read_input_token_cost": 1.9e-07, "deprecation_date": "2026-10-03", ... } "azure_ai/MAI-Image-2.5": { + "deprecation_date": "2026-10-01" } "azure_ai/MAI-Image-2.5-Flash": { + "deprecation_date": "2026-10-01" }The DeepSeek pricing JSON has distinct
0731rows, which is where the correcteddeepseek-v4-flash-0731prices come from. The source PR added a second keyazure_ai/DeepSeek-V4-Flash-0731; the lowercase key already exists on the base branch, so the existing key is corrected instead of adding a duplicate. The three retirement dates are the rows in the raw retirement schedule table (Kimi K2.7 Code 2026-10-03, MAI Image 2.5 and 2.5 Flash 2026-10-01)AWS GovCloud Bedrock and Bedrock Mantle (absorbed from #38789)
Sources: https://aws.amazon.com/bedrock/pricing/ (GovCloud tab, Claude rows), the AWS price list offer files for
us-gov-west-1andus-gov-east-1(usage types prefixedUGW1-/UGE1-, converted from per-1K and per-1M units), https://docs.aws.amazon.com/bedrock/latest/userguide/models-region-compatibility.html, and the Claude Sonnet 5 and Opus 4.8 model cards (https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-sonnet-5.html, https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-opus-4-8.html)New
bedrock/us-gov-{west,east}-1/entries foranthropic.claude-sonnet-5($2.40 / $12.00, cache write $3.00 5m and $4.80 1h, cache read $0.24 per 1M),anthropic.claude-opus-4-8($6.00 / $30.00, $7.50 / $12.00 cache write, $0.60 cache read),nvidia.nemotron-nano-3-30b,nvidia.nemotron-nano-12b-v2,nvidia.nemotron-super-3-120b,openai.gpt-oss-20b-1:0,openai.gpt-oss-120b-1:0. All 14 keys are appended towhitelisted_bedrock_models.txt"bedrock/us-gov-west-1/meta.llama3-8b-instruct-v1:0": { - "output_cost_per_token": 2.65e-06, + "output_cost_per_token": 6e-07,New
us-gov.anthropic.claude-sonnet-5andus-gov.anthropic.claude-opus-4-8(bedrock_converse) at the same GovCloud rates. The model cards' "Availability using the bedrock-runtime endpoint" table lists us-gov-west-1 and us-gov-east-1 as Geo yes and In-Region no for both models, so theus-gov.geo profile is the id GovCloud callers actually send, and the pricing page's GovCloud "Geo and In-region Cross-region Inference" table prices it the same as in-region. On the base branch those ids fall through to the commercial global entry and bill $2.00 / $10.00 (Sonnet 5) and $5.00 / $25.00 (Opus 4.8), 20% under the GovCloud rateNew
bedrock_mantle/us-gov-west-1/entries foropenai.gpt-5.6-terra,openai.gpt-5.6-luna(standard plus_above_272k_tokenstiers),openai.gpt-5.4,xai.grok-4.3, andbedrock_mantle/us-gov-east-1/openai.gpt-5.4. Every value is the matching offer-file SKUTests:
tests/test_litellm/test_bedrock_usgov_pricing.pygains the pinned-value tests from #38789, cut down to the entries that are in this PR, including negative checks that the dropped east keys stay absent, plus aus-gov.key template so the geo profile entries are pinned the same wayAzure Government (absorbed from #38789)
Source: Azure Retail Prices API,
serviceName eq 'Foundry Models', regionsusgovvirginia/usgovarizona, cross-checked with https://learn.microsoft.com/en-us/azure/azure-government/documentation-government-cognitiveservicesNew
azure/us-gov/gpt-5.1($1.71875 / $13.75, $0.171875 cached per 1M),azure/us-gov/o3-mini($1.513 / $6.05, $0.757 cached per 1M),azure/us-gov/text-embedding-3-large($0.163 per 1K),azure/us-gov/text-embedding-3-small($0.025 per 1K)Cloudflare Workers AI (absorbed from #38384)
Sources: https://developers.cloudflare.com/workers-ai/models/whisper/ ($0.00045 per audio minute) and https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/ ($0.00051 per audio minute), consistent with the neuron rates on https://developers.cloudflare.com/workers-ai/platform/pricing/
New
cloudflare/@cf/openai/whisper(input_cost_per_second7.5e-06) andcloudflare/@cf/openai/whisper-large-v3-turbo(8.5e-06), modeaudio_transcription. Tests: two whisper cases added totests/test_litellm/test_cloudflare_workers_ai_model_metadata.pyDropped as unverifiable
bedrock/us-gov-east-1/meta.llama3-8b-instruct-v1:0output price: the east offer file has no Llama 3 8B row and the region table lists the model in us-gov-west-1 onlybedrock_mantle/us-gov-east-1/openai.gpt-5.6-terra,openai.gpt-5.6-luna,xai.grok-4.3: the east offer file has no SKUs for them (it carries grok-4.6 instead)deprecation_dateon the fourazure/us-gov/*entries: Azure publishes no Government-specific retirement schedule, and the commercial schedule does not state it applies to Government regions, so no date is writtensourceswaps onazure_ai/deepseek-v3.2,deepseek-v3.2-speciale,deepseek-r1,deepseek-v3,deepseek-v3-0324: metadata only, no value changed, left outcloudflare/@cf/openai/whisper-tiny-enat $0: Cloudflare publishes no price, a $0 placeholder would report free usageSource PRs
Superseded and closed with a pointer here: #39245 (OpenAI tiers and tests, fully absorbed), #39278 (Together Qwen3.8, fully absorbed), #38502 (Scaleway, fully absorbed), #38789 (GovCloud, the
us-gov.Claude geo profiles and Azure Government, verifiable subset absorbed), #38384 (Cloudflare Whisper, two priced models absorbed)Already closed before this update: #38976 (Azure AI Foundry draft, its values are absorbed here with the source-only edits dropped), #39148 (Claude Fable 5.1, merged, a transformation PR), #39079 (together sync 2026-09-01, on the base branch through #38990)
Left alone: #39242 (Morph) changes
litellm/llms/morph/chat/transformation.py, so it is a behavior PR; #38516 (Volcengine) has no verifiable USD sourceChecked and deliberately not changed
gpt-5.6*and Azure copies keepmax_input_tokens: 922000. The model pages say 1,050,000 but commit 6bb677d set the 922k cap on purpose, so this needs a human callcodestral-2508/codestral-latest: model API says 256000 context, model card says 128k. Two official sources disagree, so the value staysmax_output_tokens: 500000on grok-4.5 / 4.6 is not documented by xAI and is left as isqwen3p8-2p4t-a95banddeepseek-v4-flash-vision-exp: no price row on the pricing page/api/v1/modelson ~15 entries but is an aggregator, not the providerRelevant issues
Supports #26900 (deprecation metadata for
/model/deprecations)Linear ticket
Pre-Submission checklist
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or moreosv-scan(not required) flagspypdf 6.15.0andtornado 6.5.7inuv.lock; that file is byte-identical to the base branch and the base fails the same scan@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup: two worktrees, merge base 2ce4e3f on port 23401 and tip 7a35c34 on port 52852, one proxy each with
--num_workers 2,PYTHONPATHpointed at its worktree (the boot log prints whichlitellm/__init__.pyloaded),LITELLM_LOCAL_MODEL_COST_MAP=Trueso each serves its own checkout's registry, and its own Postgres database (lit6696_before/lit6696_after, schema pushed withprisma db push) so the spend rows can be read back. Real calls to Together, xAI, Mistral, Fireworks and OpenAI. The ~290k token prompt (292,870 tokens by tiktoken o200k) is a 1.6 MB text file with "Reply with the single word pong" appended, stored inchat_big.json,responses_big.jsonandmessages_big.jsonwith"service_tier": "priority". The/v1/messagesbody usesmax_tokens: 64because at 16 the whole budget goes to reasoning and the response carries zero usage on both sides (pre-existing, see the risk section)Before (2ce4e3f)
GET /model/deprecations
GET /model/info
/model/infocurl against port 23401, printing the cost fieldsPOST /v1/chat/completions qwen3.8
POST /v1/chat/completions grok-build
POST /v1/chat/completions mistral-medium
POST /v1/chat/completions glm-5.3-flash
POST /v1/chat/completions gpt-5.6-luna priority ~290k
POST /v1/responses gpt-5.6-luna priority ~290k
POST /v1/messages gpt-5.6-luna priority ~290k
LiteLLM_SpendLogs
After (7a35c34)
GET /model/deprecations
GET /model/info
/model/infocurl against port 52852, printing the cost fieldsPOST /v1/chat/completions qwen3.8
POST /v1/chat/completions grok-build
POST /v1/chat/completions mistral-medium
POST /v1/chat/completions glm-5.3-flash
POST /v1/chat/completions gpt-5.6-luna priority ~290k
POST /v1/responses gpt-5.6-luna priority ~290k
POST /v1/messages gpt-5.6-luna priority ~290k
LiteLLM_SpendLogs
Local checks at 7a35c34:
python3 ci_cd/check_files_match.pypasses,uv run --no-sync python ci_cd/generate_model_prices_schema.pyproduces no schema diff,LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/test_bedrock_usgov_pricing.py tests/test_litellm/test_cloudflare_workers_ai_model_metadata.py tests/test_litellm/test_openai_service_tier_long_context_pricing.py tests/test_litellm/test_model_prices_schema.py tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py(323 passed),tests/local_testing/test_get_model_info.py -k bedrock_converse(the whitelist gate, 2 passed),make checkPASSLive PR risk (7a35c34 vs merge base 2ce4e3f)
Breaking behavior: none. Every touched key is either new or gets re-priced tokens, cache fields, tier fields or a
deprecation_date; no key,mode,litellm_providerorsupported_endpointsvalue is removed or renamedBackward compatibility: the same
model_listboots on base and tip against the same Postgres schema, all live calls return 200 on both sides, only the reported cost, the spend rows and the deprecations buckets changeRegression risk: cost figures move for Together Qwen3.8 (down 20% input, 0.000245 to 0.000212 on the same call), Azure AI
deepseek-v4-flash-0731(up 2.3x, corrected to the 0731 row),mistral/mistral-medium(8.37e-05 to 5.55e-05) and the magistral aliases,gpt-realtime-2output, GPT-5.4 / 5.5 / 5.6 flex or priority prompts above 272k tokens (2x on that slice: the ~290k priority prompt goes from 0.01174 to 0.02345), xAIgrok-build-latest(the generic xai fallback billed 0.0004872 on the base call and its own row bills 0.0005352 on the tip call; each equals xAI's owncost_in_usd_ticksin that response, 4872000 ($0.0004872) and 5352000 ($0.0005352), the two calls differing only in how many prompt tokens hit xAI's cache) and Fireworks GLM 5.3 Flash (was $0, now 1.07e-05). GovCloud Llama 3 8B output drops from $2.65 to $0.60 per 1M, and theus-gov.Claude geo profiles rise 20% from the commercial rate they fell through toDependency graph: the two JSON files feed
/model/info,/model/deprecations, the cost headers andLiteLLM_SpendLogsrows on/v1/chat/completions,/v1/responsesand/v1/messages, the audio transcription cost path (input_cost_per_second), and the router's context-window checks. Tested above on both sides:/model/info,/model/deprecations, four small chats (Together, xAI, Mistral, Fireworks), all three text endpoints at ~290k priority tokens, and the spend rows read back from Postgres. Covered by tests: schema, GovCloud regional and geo profile values, Azure Government pinned values, Cloudflare Whisper per-second values, OpenAI tier ratios and cost at 300k, the Bedrock whitelist gate, cost-calc field countUnverified surfaces: no live call was made to Scaleway, Azure AI Foundry, GovCloud Bedrock, Azure Government or Cloudflare (no credentials in this rig), so those entries are proven through
/model/infoand the pinned tests, not through a billed request. Two pre-existing/v1/messagesbehaviors on OpenAI models are identical on base and tip and untouched here: the usage block reportsinput_tokens: 3pluscache_read_input_tokenswhile the cost header bills the whole prompt, and withmax_tokens: 16on this prompt every output token is reasoning, the response carries zero usage, and the cost header and spend row are 0Type
🐛 Bug Fix
Caveats (if any)
Medium
deepseek-v4-flash-0731more than doubles ($0.19 / $0.51 to $0.44 / $1.32 per 1M); anyone budgeting on the old copy-paste values sees a jumpmistral/mistral-mediummoves from Medium 1.0 pricing ($2.70 / $8.10) to Medium 3.5 ($1.50 / $7.50)gpt-realtime-2text output cost rises 50% ($16 to $24 per 1M)us-gov.Claude Sonnet 5 and Opus 4.8 calls report 20% more spend than before, because the base branch billed them at the commercial rateLow
azure/us-gov/*entries carry nodeprecation_date; add one when Azure publishes a Government retirement scheduleglm-5p3-flashandinklinghave nomax_output_tokens; Fireworks does not publish one/v1/messageson OpenAI models returns zero usage and zero cost when the wholemax_tokensbudget goes to reasoning; pre-existing on the base branch, not a registry problemosv-scanfails onpypdfandtornadopins inuv.lock, identical to the base branchFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/8b29e469c48246ab9a0f00fc49eba159
Open in Devin Desktop: https://app.devin.ai/desktop/session/8b29e469c48246ab9a0f00fc49eba159?variant=devin
Requested by: @mateo-berri
Note
Medium Risk
Changes reported spend and budgets for several high-traffic models (Together Qwen, Azure DeepSeek 0731, Mistral aliases, GPT long-context tiers, realtime output); no runtime routing logic changes, but incorrect figures would have been user-visible billing bugs.
Overview
Updates
model_prices_and_context_window.jsonand its backup so proxy spend,/model/info, and/model/deprecationsmatch current provider pricing.OpenAI: Adds flex long-context (above 272k) rates for GPT-5.4/5.5 families and priority long-context rates for GPT-5.6 variants; refreshes realtime token limits, text output pricing, and audio cache read on
gpt-realtime-2.Mistral: Re-points
magistral-*-latestandmistral/mistral-mediumaliases to Medium 3.5 / Small 4 pricing, context windows, cache read, and vision flags.New / corrected entries: Voyage rerank-3 models, xAI grok-build-latest, Fireworks GLM 5.3 Flash and Inkling, Scaleway GLM 5.2 and DeepSeek 0731, Azure AI Kimi K2.7 Code plus prompt-cache fields and MAI Image retirement dates; GovCloud Bedrock, Bedrock Mantle, and Azure Government model rows; Cloudflare Whisper transcription per-second costs. Fixes Together Qwen3.8 rates, Azure AI deepseek-v4-flash-0731 pricing, and Gov Llama3-8B output on Bedrock.
Tests & allowlist: New/expanded pinned pricing tests (OpenAI tiers, GovCloud, Cloudflare Whisper); cost-calc expectations for GPT-5.6 priority long context; 14 Gov Bedrock keys added to
whitelisted_bedrock_models.txt.Reviewed by Cursor Bugbot for commit da23e02. Bugbot is set up for automated code reviews on this repo. Configure here.