feat(pricing): add AWS GovCloud and Azure Government pricing for missing models - #38789
feat(pricing): add AWS GovCloud and Azure Government pricing for missing models#38789mateo-berri wants to merge 2 commits into
Conversation
…ing models Adds GovCloud (us-gov-west-1 / us-gov-east-1) pricing entries for Claude Sonnet 5, Claude Opus 4.8, Nemotron Nano 3 30B, Nemotron Nano 12B v2, Nemotron Super 3 120B, gpt-oss-20b, gpt-oss-120b, GPT-5.6 Terra, GPT-5.6 Luna, GPT-5.4, and grok-4.3 (west only), plus Azure Government entries for gpt-5.1, o3-mini, text-embedding-3-large, and text-embedding-3-small. Also fixes the existing gov llama3-8b entries, which carried the 70B output rate ($2.65/MTok) instead of the $0.60/MTok the AWS offer file publishes.
Greptile SummaryThis PR expands government-cloud pricing coverage and corrects an existing GovCloud Llama output rate.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds the primary Azure Government and AWS GovCloud pricing metadata and corrects the Llama 3 8B output rate. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the government-cloud pricing additions and rate correction in the backup cost map. |
| tests/test_litellm/test_bedrock_usgov_pricing.py | Adds local, network-free assertions for the new pricing entries, regional coverage, and long-context tiers. |
| whitelisted_bedrock_models.txt | Whitelists the newly introduced region-qualified Bedrock GovCloud chat-model identifiers. |
Reviews (2): Last reviewed commit: "fix(pricing): whitelist the new GovCloud..." | Re-trigger Greptile
…rock_converse guard
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 7be56c6. Configure here.
|
Closing as superseded by the rolling registry PR #39170, which carries the offer-file verified GovCloud and Azure Government subset and lists the dropped east and profile keys |
2 participants
TLDR
Problem this solves:
bedrock/us-gov-*/meta.llama3-8b-instruct-v1:0output price was 4.4x too highHow it solves it:
tests/test_litellm/test_bedrock_usgov_pricing.pywhitelisted_bedrock_models.txt, the same way every existingbedrock/us-gov-*chat key is, so the legacytest_model_info_bedrock_converseguard keeps passingUser Flow
Before: a developer whose gateway routes Claude Sonnet 5 through AWS GovCloud sees it logged at the commercial rate, and the GovCloud Mantle models cannot be priced at all
model: bedrock/anthropic.claude-sonnet-5andaws_region_name: us-gov-west-1(config or POST https://litellm-domain/model/new)"model": "claude-sonnet-5"and gets a 200model: bedrock_mantle/openai.gpt-5.6-terrawithaws_region_name: us-gov-west-1, and every request to it logs at $0.00 because the cost lookup fails with "This model isn't mapped yet"model: azure/<deployment>withmodel_info: {base_model: azure/us-gov/gpt-5.1}for their Azure Government deployment, and requests log at $1.25 and $10.00 per million instead of the Azure Government $1.72 and $13.75meta.llama3-8b-instruct-v1:0GovCloud requests log at $2.65 per million output tokens, 4.4x the real $0.60After: the same requests log at the GovCloud and Azure Government rates
model: bedrock/anthropic.claude-sonnet-5andaws_region_name: us-gov-west-1(config or POST https://litellm-domain/model/new)"model": "claude-sonnet-5"and gets a 200bedrock_mantle/openai.gpt-5.6-terrainus-gov-west-1log at $2.64 per million input and $15.84 per million output (the 272k long-context tier applies above that prompt size)gpt-5.1deployment logs at $1.72 per million input and $13.75 per million outputmeta.llama3-8b-instruct-v1:0GovCloud requests log at $0.60 per million output tokensPricing evidence
Every AWS price is the
20260826181919AmazonBedrock offer file for the region (pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonBedrock/current/us-gov-west-1/index.jsonand theus-gov-east-1sibling), which puts every GovCloud SKU at exactly 1.2x the commercial price. The Claude models are Marketplace-billed and have no offer file rows, so their prices come from the AWS Bedrock pricing page's GovCloud (US) and GovCloud (US-East) tables, which are identical. Azure Government prices come from the Azure retail prices API (prices.azure.com/api/retail/prices,serviceName eq 'Foundry Models',armRegionNameusgovvirginiaandusgovarizona), Data Zone meters for gpt-5.1 and o3-miniRegion coverage was decided per model from the AWS model cards plus the offer files, and where the two disagreed the more recent source won. Terra and Luna are GA in both GovCloud regions (the 2026-08-24 What's New post
aws.amazon.com/about-aws/whats-new/2026/08/openai-gpt-terra-luna-govcloud/and both model cards listus-gov-east-1as in-region with an identical price table) while the east offer file has not published their SKUs yet, so both region keys ship with the west prices.openai.gpt-5.4is the reverse: its model card lists west only, but the east offer file already publishesUGE1-openai.gpt-5.4-mantlerows at the same prices, so both region keys ship.xai.grok-4.3is west only on both sources, so only the west key ships. The gov offer files carry no long-context SKUs for gpt-5.4, so its gov entries have noabove_272ktierRelevant issues
Linear ticket
Resolves LIT-6421
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
No GovCloud or Azure Government credentials were reachable for a real provider call (none in
.env, and the 1Password desktop app was locked), so the proof drives the proxy's own cost engine throughPOST /spend/calculate, the same path spend logs go through. Both legs boot the same config on the same port with 2 uvicorn workers and send the same 15 requests; only the checked-out commit differs. Each request carries 1,000,000 prompt and 1,000,000 completion tokens so the returned cost reads as dollars per million on each sideShared setup
Before (06852ca)
Llama 3 8B GovCloud output price
cost bedrock/us-gov-west-1/meta.llama3-8b-instruct-v1:0{"cost":2.9499999999999997}($0.30 input + $2.65 output; the output rate is wrong)Claude Sonnet 5 and Opus 4.8 (region keys and us-gov. profiles)
cost bedrock/us-gov-west-1/anthropic.claude-sonnet-5{"cost":12.0}(commercial $2.00 + $10.00, no GovCloud entry)cost bedrock/us-gov-east-1/anthropic.claude-opus-4-8{"cost":30.0}(commercial $5.00 + $25.00)cost us-gov.anthropic.claude-sonnet-5{"cost":12.0}cost us-gov.anthropic.claude-opus-4-8{"cost":30.0}Nemotron and gpt-oss
cost bedrock/us-gov-west-1/nvidia.nemotron-super-3-120b{"cost":0.8}(commercial $0.15 + $0.65)cost bedrock/us-gov-east-1/openai.gpt-oss-120b-1:0{"cost":0.75}(commercial $0.15 + $0.60)Mantle: Terra, Luna, gpt-5.4, grok-4.3
cost bedrock_mantle/us-gov-west-1/openai.gpt-5.6-terra{"error":{"message":"This model isn't mapped yet. model=bedrock_mantle/us-gov-west-1/openai.gpt-5.6-terra, custom_llm_provider=bedrock_mantle. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.","type":"None","param":"None","code":"500"}}cost bedrock_mantle/us-gov-east-1/openai.gpt-5.6-luna{"error":{"message":"This model isn't mapped yet. model=bedrock_mantle/us-gov-east-1/openai.gpt-5.6-luna, custom_llm_provider=bedrock_mantle. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.","type":"None","param":"None","code":"500"}}cost bedrock_mantle/us-gov-east-1/openai.gpt-5.4{"error":{"message":"This model isn't mapped yet. model=bedrock_mantle/us-gov-east-1/openai.gpt-5.4, custom_llm_provider=bedrock_mantle. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.","type":"None","param":"None","code":"500"}}cost bedrock_mantle/us-gov-west-1/xai.grok-4.3{"error":{"message":"This model isn't mapped yet. model=bedrock_mantle/us-gov-west-1/xai.grok-4.3, custom_llm_provider=bedrock_mantle. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.","type":"None","param":"None","code":"500"}}cost bedrock_mantle/us-gov-east-1/xai.grok-4.3{"error":{"message":"This model isn't mapped yet. model=bedrock_mantle/us-gov-east-1/xai.grok-4.3, custom_llm_provider=bedrock_mantle. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.","type":"None","param":"None","code":"500"}}Azure Government gpt-5.1, o3-mini, text-embedding-3-large
cost azure/us-gov/gpt-5.1{"cost":11.25}(commercial $1.25 + $10.00)cost azure/us-gov/o3-mini{"cost":5.5}(commercial $1.10 + $4.40)cost azure/us-gov/text-embedding-3-large 1000000 0{"cost":0.13}(commercial $0.13)After (7be56c6)
Llama 3 8B GovCloud output price
cost bedrock/us-gov-west-1/meta.llama3-8b-instruct-v1:0{"cost":0.8999999999999999}($0.30 input + $0.60 output)Claude Sonnet 5 and Opus 4.8 (region keys and us-gov. profiles)
cost bedrock/us-gov-west-1/anthropic.claude-sonnet-5{"cost":14.4}($2.40 + $12.00)cost bedrock/us-gov-east-1/anthropic.claude-opus-4-8{"cost":36.0}($6.00 + $30.00)cost us-gov.anthropic.claude-sonnet-5{"cost":14.4}cost us-gov.anthropic.claude-opus-4-8{"cost":36.0}Nemotron and gpt-oss
cost bedrock/us-gov-west-1/nvidia.nemotron-super-3-120b{"cost":0.96}($0.18 + $0.78)cost bedrock/us-gov-east-1/openai.gpt-oss-120b-1:0{"cost":0.8999999999999999}($0.18 + $0.72)Mantle: Terra, Luna, gpt-5.4, grok-4.3
cost bedrock_mantle/us-gov-west-1/openai.gpt-5.6-terra{"cost":29.04}(1M prompt tokens crosses the 272k tier: $5.28 + $23.76)cost bedrock_mantle/us-gov-east-1/openai.gpt-5.6-luna{"cost":2.9040000000000004}(exactly one tenth of Terra, tier included)cost bedrock_mantle/us-gov-east-1/openai.gpt-5.4{"cost":23.1}($3.30 + $19.80, no long-context tier in GovCloud)cost bedrock_mantle/us-gov-west-1/xai.grok-4.3{"cost":4.5}($1.50 + $3.00)cost bedrock_mantle/us-gov-east-1/xai.grok-4.3{"error":{"message":"This model isn't mapped yet. model=bedrock_mantle/us-gov-east-1/xai.grok-4.3, custom_llm_provider=bedrock_mantle. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.","type":"None","param":"None","code":"500"}}(still unmapped on purpose: grok-4.3 is not offered in us-gov-east-1)Azure Government gpt-5.1, o3-mini, text-embedding-3-large
cost azure/us-gov/gpt-5.1{"cost":15.46875}($1.71875 + $13.75)cost azure/us-gov/o3-mini{"cost":7.563}($1.513 + $6.05)cost azure/us-gov/text-embedding-3-large 1000000 0{"cost":0.16299999999999998}($0.163)Type
🆕 New Feature
🐛 Bug Fix
Caveats (if any)
Medium
/spend/calculate, not a real GovCloud completion.env, 1Password desktop app locked); a real completion per family is queued on the ticket for when they landus-gov-*Claude entries already resolve through in productionLow
bedrock_mantle/us-gov-east-1/openai.gpt-5.4ships on the east offer file alone while its model card still lists west only; dropping the key would leave east requests with no price at all, so it staysusgovarizonameters becauseusgovvirginiapublishes no text-embedding-3 meters yet; every other gov model prices identically in both regions, so they staybedrock_mantle/us-gov-*: the map carries no commercial Mantle Claude precedent to mirrorxai.grok-4.3stays west only, where the model card and the east offer file agreebedrock/us-gov-*chat keys carrylitellm_provider: bedrockplus a whitelist entry, the same shape as every existing gov region key; converse routing keys off the base model, so nothing changes at request timetests/llm_translation/test_bedrock_govcloud.py::test_govcloud_completion_with_cost_trackingfails locally at the merge base and at this tip with the samebotocore[crt]missing-dependency error, unrelated to this changeFinal Attestation
Note
Low Risk
Data-only pricing and whitelist updates plus tests; no runtime routing or auth changes, though incorrect rates would affect billing visibility until corrected.
Overview
Extends
model_prices_and_context_window.json(and its backup) so spend/cost lookup works for government-cloud SKUs instead of falling back to commercial rates or failing with “model isn't mapped.”Azure Government adds
azure/us-gov/*entries for gpt-5.1, o3-mini, and text-embedding-3-small/large with gov-specific token prices.AWS GovCloud Bedrock adds region-prefixed keys for Claude Sonnet 5 and Opus 4.8 (
bedrock/us-gov-*andus-gov.*converse profiles), Nemotron and gpt-oss chat models in both gov regions, and correctsmeta.llama3-8b-instruct-v1:0output cost from the erroneous 70B rate to $0.60/MTok.Bedrock Mantle adds gov entries for gpt-5.6-terra/luna (including 272k tiers), gpt-5.4 (standard tier only), and xai.grok-4.3 (west only).
tests/test_litellm/test_bedrock_usgov_pricing.pygains parametrized regressions for each family (1.2× commercial where applicable).whitelisted_bedrock_models.txtlists the newbedrock/us-gov-*chat keys so existing Bedrock converse model-info guards keep passing.Reviewed by Cursor Bugbot for commit 7be56c6. Bugbot is set up for automated code reviews on this repo. Configure here.