feat(pricing): add azure data-zone and long-context pricing for gpt-5.4/5.5 by devin-ai-integration[bot] · Pull Request #32279 · BerriAI/litellm

devin-ai-integration · 2026-07-06T21:23:05Z

Relevant issues

Linear ticket

Resolves LIT-4225

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

These models have no reachable public endpoint and a real Data Zone call needs the customer's Azure deployment credentials, so the proxy runs against this PR's bundled cost map (LITELLM_LOCAL_MODEL_COST_MAP=True) in front of a tiny local OpenAI-compatible upstream that returns fixed token usage (1000 prompt / 500 completion for the base tier on port 8801, and 300000 prompt / 1000 completion for long context on port 8802), so the emitted x-litellm-response-cost header is deterministic and hand-checkable. That header is the exact value written to spend logs, so it is what the customer sees

python fake_upstream.py 8801 1000 500 &      # base tier usage
python fake_upstream.py 8802 300000 1000 &   # long-context usage
LITELLM_LOCAL_MODEL_COST_MAP=True python -m litellm.proxy.proxy_cli --config config.yaml --port 4000 &

for m in gpt-5.4-global gpt-5.4-us gpt-5.5-global gpt-5.5-us \
         lc-gpt-5.4-global lc-gpt-5.4-us lc-gpt-5.5-global lc-gpt-5.5-us; do
  curl -s -i http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}" \
    | grep -i "x-litellm-response-cost:"
done

Output:

### base tier (1000 prompt / 500 completion) ###
gpt-5.4-global      x-litellm-response-cost: 0.01     # 1000*2.5e-6 + 500*1.5e-5
gpt-5.4-us          x-litellm-response-cost: 0.011    # 1.1x Global Data Zone
gpt-5.5-global      x-litellm-response-cost: 0.02
gpt-5.5-us          x-litellm-response-cost: 0.022    # 1.1x Global Data Zone

### long context (300000 prompt / 1000 completion) ###
lc-gpt-5.4-global   x-litellm-response-cost: 1.5225   # Global has >272k surcharge (300000*5e-6)
lc-gpt-5.4-us       x-litellm-response-cost: 0.8415   # Data Zone flat rate, no surcharge (300000*2.75e-6)
lc-gpt-5.5-global   x-litellm-response-cost: 3.045    # 300000*1e-5 + 1000*4.5e-5
lc-gpt-5.5-us       x-litellm-response-cost: 3.3495   # Data Zone >272k, 300000*1.1e-5 + 1000*4.95e-5

Before this change the -us and -eu rows resolved to the base entry and billed the Global rate

Type

🆕 New Feature

Changes

Azure customers on the gpt-5.4 and gpt-5.5 families were seeing spend discrepancies because the Data Zone deployment variants were missing from the cost map, so a request to azure/us/gpt-5.5 fell through to the base entry and was billed at the Global rate instead of the Data Zone rate

Every number here was transcribed field by field from the Azure OpenAI pricing page rather than derived from a flat multiplier, which matters because the Azure Data Zone surface is not uniform across the family. Azure only publishes a Data Zone column for gpt-5.4 (under 272k context) and gpt-5.5, and it only publishes a Long Context (over 272k) Data Zone row for gpt-5.5. The mini, nano and pro members are Global only, and mini/nano have no long-context tier at all. An earlier revision of this branch had wrongly generated Data Zone entries for mini/nano/pro and a long-context tier for mini/nano by scaling everything by 1.1x; verifying against the page removed those invented entries

Concretely this adds eight entries to both model_prices_and_context_window.json and the bundled litellm/model_prices_and_context_window_backup.json: the us and eu Data Zone variants of azure/gpt-5.4, azure/gpt-5.4-2026-03-05, azure/gpt-5.5 and azure/gpt-5.5-2026-04-23. Each carries the standard on-demand and priority processing token costs from the page, plus the over-272k tier for the gpt-5.5 pair only. The gpt-5.4 Data Zone entries deliberately omit any above_272k keys, so a prompt past 272k on a gpt-5.4 Data Zone deployment bills the flat Data Zone rate rather than a surcharge that Azure does not offer

The Global entries and the mini/nano/pro entries already matched the page and are left untouched

Sources

Prices verified against the Azure OpenAI pricing page at https://azure.microsoft.com/en-us/pricing/details/azure-openai/ (Standard on-demand and Priority Processing columns; the Long Context rows for the over-272k tier). The relevant rows, per 1M tokens:

GPT-5.5 Data Zone                  Input $5.50  Cached $0.55  Output $33
GPT-5.5 Long Context Data Zone     Input $11    Cached $1.10  Output $49.50
GPT-5.5 Data Zone (priority)       Input $13.75 Cached $1.38  Output $82.50
GPT-5.4 (<272k) Data Zone          Input $2.75  Cached $0.28  Output $16.50
GPT-5.4 (<272k) Data Zone priority Input $5.50  Cached $0.55  Output $33
(no GPT-5.4 Data Zone >272k row; no Data Zone row for gpt-5.4 mini/nano/pro or gpt-5.5-pro)

For the resolution logic these entries rely on, cost lookup walks the provider/model then model then stripped variants in litellm/litellm_core_utils/get_model_cost_map.py and litellm/cost_calculator.py, and the over-272k tier is selected by _parse_above_token_threshold when prompt_tokens crosses 272000

Link to Devin session: https://app.devin.ai/sessions/c00b3c0e15884513955dfc03d3e25a7b
Requested by: @mubashir1osmani

Note

Low Risk
Data-only cost-map additions with no runtime logic changes; main risk is incorrect published rates affecting billing estimates for those model IDs.

Overview
Adds eight new model cost-map entries in model_prices_and_context_window.json and the bundled backup so Azure US/EU Data Zone deployments (azure/us/..., azure/eu/...) for gpt-5.4, gpt-5.4-2026-03-05, gpt-5.5, and gpt-5.5-2026-04-23 resolve to Azure’s Data Zone rates instead of falling through to the Global azure/gpt-5.* rows.

Each new entry includes on-demand and priority token pricing (and cache-read where applicable). gpt-5.5 Data Zone rows also define >272k input/output/cache tiers; gpt-5.4 Data Zone rows intentionally omit above_272k fields so long prompts bill at the flat Data Zone rate Azure publishes for that family.

^{Reviewed by Cursor Bugbot for commit 99f2887. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

New Features
- Added region-specific model entries for Azure in the pricing and capability data.
- Expanded support details for several GPT models, including availability, limits, supported endpoints, and capabilities.
- Included additional support for web search and reasoning options on newer model variants.

….4/5.5 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration · 2026-07-06T21:23:09Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

CLAassistant · 2026-07-06T21:23:14Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

Mubashir Osmani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

greptile-apps · 2026-07-06T21:28:36Z

Greptile Summary

Adds Azure Data Zone (us/eu) pricing entries for the gpt-5.4 and gpt-5.5 model families so requests routed through Data Zone deployments are billed at the correct regional rate instead of falling through to the Global rate.

Eight new entries cover the un-dated aliases and dated variants (gpt-5.4-2026-03-05, gpt-5.5-2026-04-23) for both us and eu Data Zones. All per-token costs are consistent with the values stated in the PR description.
above_272k_tokens pricing keys are correctly present on gpt-5.5 Data Zone entries and deliberately omitted from gpt-5.4 Data Zone entries, reflecting Azure's published pricing structure (no long-context surcharge row for gpt-5.4 Data Zone).
Capability flags (supports_reasoning, supports_web_search, supports_none_reasoning_effort, etc.) on each new entry match their corresponding base Global entry, maintaining consistency.

Confidence Score: 5/5

Pure pricing data addition — no logic changes, no regressions possible from existing model resolution paths.

The change is limited to two JSON data files. All eight new entries have been cross-checked: token costs match the values documented in the PR description, above-threshold keys are included exactly where Azure publishes a corresponding row (gpt-5.5 only), and capability flags align with the existing base entries for each model. The backup file is an exact mirror of the main file.

No files require special attention.

Important Files Changed

Filename	Overview
model_prices_and_context_window.json	Adds 8 new Data Zone pricing entries (us/eu variants of gpt-5.4, gpt-5.4-2026-03-05, gpt-5.5, gpt-5.5-2026-04-23); prices match stated Azure source, above_272k fields correctly included for gpt-5.5 and omitted for gpt-5.4, and capability flags are consistent with their corresponding base entries.
litellm/model_prices_and_context_window_backup.json	Mirrors the same 8 new Data Zone entries as the main JSON; diff confirms both files received identical additions.

_{Reviews (2): Last reviewed commit: "chore(pricing): drop azure data-zone pri..." | Re-trigger Greptile}

codecov · 2026-07-06T21:29:07Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Verified against the Azure OpenAI pricing page. Azure only offers Data Zone for gpt-5.4 (<272k) and gpt-5.5, and a Long Context Data Zone tier only for gpt-5.5; mini, nano and pro are Global-only and mini/nano have no long-context tier. Drop the incorrectly-added mini/nano/pro data-zone variants and the mini/nano long-context keys, and set the gpt-5.4/5.5 us and eu entries to the exact published standard and priority prices (gpt-5.4 has no data-zone long context, gpt-5.5 does). Tests now assert the published values rather than a flat 1.1x multiplier. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

mateo-berri · 2026-07-06T22:03:44Z

@greptileai

mateo-berri

Missing test coverage for dated long-context variants

test_regional_long_context_surcharge_present only parametrizes the un-dated aliases (azure/gpt-5.4-mini, azure/gpt-5.4-nano), but the PR description explicitly calls out four entries that were corrected — including azure/gpt-5.4-mini-2026-03-17 and azure/gpt-5.4-nano-2026-03-17. The dated variants do carry the right values in the JSON (verified), but no test directly asserts the specific numeric rates for them; they are only indirectly covered by test_backup_matches_main, which only checks that main and backup agree — not that the values are correct.

@mubashir1osmani why did you resolve this. Is this not a real concern? I just want to know why. Going forward, please leave your explanation as a reply in thread

mateo-berri · 2026-07-06T23:13:43Z

bugbot run

mateo-berri · 2026-07-06T23:14:00Z

@coderabbitai review (just testing coderabbit; don't treat its concerns as blocking)

coderabbitai · 2026-07-06T23:14:13Z

@mateo-berri, sounds good — I'll go ahead and run a review now. Since you mentioned this is just a test, I'll post my findings as usual, but no worries if you'd like to disregard any non-blocking suggestions.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai · 2026-07-06T23:14:20Z

@mateo-berri Got it — kicking off a review now. Noted that this is a test run, so no worries about treating any findings as blocking.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 99f2887. Configure here.}

mubashir1osmani · 2026-07-06T23:18:11Z

Missing test coverage for dated long-context variants
test_regional_long_context_surcharge_present only parametrizes the un-dated aliases (azure/gpt-5.4-mini, azure/gpt-5.4-nano), but the PR description explicitly calls out four entries that were corrected — including azure/gpt-5.4-mini-2026-03-17 and azure/gpt-5.4-nano-2026-03-17. The dated variants do carry the right values in the JSON (verified), but no test directly asserts the specific numeric rates for them; they are only indirectly covered by test_backup_matches_main, which only checks that main and backup agree — not that the values are correct.

@mubashir1osmani why did you resolve this. Is this not a real concern? I just want to know why. Going forward, please leave your explanation as a reply in thread

its just a sanity check if dated model variants assert an accurate price per token. i dont think we write tests for model cost map because verifying it from the provider would suffice

coderabbitai · 2026-07-06T23:20:05Z

📝 Walkthrough

Walkthrough

Updates two model pricing/config JSON files to replace single Azure model entries for gpt-5.4, gpt-5.4-2026-03-05, gpt-5.5, and gpt-5.5-2026-04-23 with region-specific azure/us and azure/eu entries, each carrying pricing, token limits, endpoints, modalities, and capability flags.

Changes

Azure region-scoped pricing split

Layer / File(s)	Summary
gpt-5.4 region entries `litellm/model_prices_and_context_window_backup.json`, `model_prices_and_context_window.json`	Splits gpt-5.4 and gpt-5.4-2026-03-05 Azure entries into azure/us and azure/eu keys, each with cache/input/output priority pricing fields, token limits, supported endpoints/modalities, and capability flags.
gpt-5.5 region entries `litellm/model_prices_and_context_window_backup.json`, `model_prices_and_context_window.json`	Splits gpt-5.5 and gpt-5.5-2026-04-23 Azure entries into azure/us and azure/eu keys, adding above_272k_tokens pricing fields, web search support, and reasoning-effort capability flags alongside the standard pricing/endpoint schema.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: Azure Data Zone and long-context pricing for gpt-5.4/5.5.
Description check	✅ Passed	The description follows the template and includes ticket, checklist, proof, type, and changes; only Relevant issues is left blank.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch litellm_azure_long_context_datazone_pricing

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@litellm/model_prices_and_context_window_backup.json`:
- Around line 6186-6263: The dated Azure gpt-5.5-2026-04-23 model entries are
missing the same reasoning-effort capability flags exposed by the gpt-5.5 alias,
which causes inconsistent capability checks. Update both
azure/us/gpt-5.5-2026-04-23 and azure/eu/gpt-5.5-2026-04-23 in
model_prices_and_context_window_backup.json to include
supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and
supports_minimal_reasoning_effort alongside the existing capability fields so
the pinned snapshot matches the alias behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: cc2ca8bf-1767-44e3-9ce6-27bffc3227d5

📥 Commits

Reviewing files that changed from the base of the PR and between b4a10fb and 99f2887.

📒 Files selected for processing (2)

litellm/model_prices_and_context_window_backup.json
model_prices_and_context_window.json

mateo-berri

LGTM; thanks!

feat(pricing): add azure data-zone and long-context pricing for gpt-5…

556636c

….4/5.5 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

mubashir1osmani self-assigned this Jul 6, 2026

devin-ai-integration Bot requested a review from mateo-berri July 6, 2026 21:23

greptile-apps Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread tests/test_litellm/test_azure_datazone_long_context_pricing.py Outdated

Mubashir Osmani and others added 2 commits July 6, 2026 21:39

chore(pricing): drop azure data-zone pricing test per review

99f2887

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

mateo-berri requested changes Jul 6, 2026

View reviewed changes

cursor Bot reviewed Jul 6, 2026

View reviewed changes

coderabbitai Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/model_prices_and_context_window_backup.json

mateo-berri approved these changes Jul 6, 2026

View reviewed changes

mubashir1osmani merged commit 5cb0721 into litellm_internal_staging Jul 6, 2026
127 checks passed

mubashir1osmani deleted the litellm_azure_long_context_datazone_pricing branch July 6, 2026 23:32

Mattchewone mentioned this pull request Jul 20, 2026

chore: sync upstream v1.93.0 bitovi/litellm#14

Merged

Uh oh!

Uh oh!

Conversation

devin-ai-integration Bot commented Jul 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Type

Changes

Sources

Summary by CodeRabbit

Uh oh!

devin-ai-integration Bot commented Jul 6, 2026

🤖 Devin AI Engineer

Uh oh!

CLAassistant commented Jul 6, 2026

Uh oh!

greptile-apps Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Jul 6, 2026

Codecov Report

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

mateo-berri commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mubashir1osmani commented Jul 6, 2026

Uh oh!

coderabbitai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

devin-ai-integration Bot commented Jul 6, 2026 •

edited by coderabbitai Bot

Loading

greptile-apps Bot commented Jul 6, 2026 •

edited

Loading

mateo-berri commented Jul 6, 2026 •

edited

Loading

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading