Skip to content

fix(pricing): restore Azure gpt-5.6 terra/luna rates to Azure's published prices - #36274

Open
olifarhaan wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
olifarhaan:litellm_azure_gpt56_pricing
Open

fix(pricing): restore Azure gpt-5.6 terra/luna rates to Azure's published prices#36274
olifarhaan wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
olifarhaan:litellm_azure_gpt56_pricing

Conversation

@olifarhaan

@olifarhaan olifarhaan commented Aug 8, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • Azure gpt-5.6 luna bills 5x more than LiteLLM reports, terra 20% more
  • Every azure/ gpt-5.6 deployment under-reports spend

How it solves it:

  • Restore the six azure terra/luna rows to Azure's published prices
  • Pin the two azure/us/ rows in the existing regression test

Relevant issues

Fixes #36094

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

There are no Azure credentials in this environment, so the live run below is the exact command list rather than captured output. Add a luna deployment to dev_config.yaml first:

  - model_name: azure-luna
    litellm_params:
      model: azure/gpt-5.6-luna
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2026-05-01-preview"
  1. Start the proxy: python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload 2>&1 | tee litellm.log
  2. Send a request and read back the cost header:
curl -s -D /dev/stderr http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"model": "azure-luna", "messages": [{"role": "user", "content": "Write one sentence about the sea"}]}' \
  2>&1 >/dev/null | grep -i "x-litellm-response-cost"
  1. Cross-check against what Azure billed for the same token counts on the Azure OpenAI pricing page

Before this PR the reported cost is 1/5 of the Azure rate for luna. After, they match. The same holds for azure/gpt-5.6-terra at 1.25x, and for the azure/us/ and azure/eu/ data-zone variants at their 10% uplift

The spend log at http://localhost:4000/ui/?page=logs shows the same number if you prefer reading it there

Type

🐛 Bug Fix

Changes

OpenAI cut gpt-5.6 terra 20% and luna 80% on 2026-07-30. Commit 4d43080 mirrored that cut onto the azure/ rows, reasoning that "openai and bedrock_mantle entries already match". Azure did not follow. Microsoft still publishes $1.00/1M input and $6.00/1M output for luna global and $1.10/$6.60 for the us/eu data zones, and there are open Microsoft Q&A threads asking why the Azure prices were never updated (one, two). So LiteLLM has been reporting 5x less than Azure bills for luna, and 20% less for terra, on every azure/ gpt-5.6 deployment since 2026-08-01

This restores the 60 fields that commit changed, across the six global/us/eu terra and luna rows, in both cost maps. Input, output, cache-read, priority, and above-272k variants all move together. sol is untouched, and so are the openai/ and bedrock_mantle rows, whose cut is correct

Two corrections to the issue as filed, both worth a reviewer's attention. It blames PR #35258, but that PR left the azure rows alone: at its merge commit azure/gpt-5.6-luna was still 1e-06, and only the openai keys moved. The actual change was 4d43080 two days later, which was deliberate rather than collateral. The issue also reports luna only, while terra regressed identically and is included here

Because this reverses a considered decision rather than an obvious slip, it is worth a maintainer confirming the pricing call before merge. Happy to drop the terra half if you would rather keep the scope to what was reported

The parametrized cost test already pinned the global and eu rows, so it fails without this change. It had no coverage for the azure/us/ rows, which is where the report actually pointed, so those two are added

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…shed prices

OpenAI cut terra 20% and luna 80% on 2026-07-30. 4d43080 mirrored that cut
onto the azure/ rows on the assumption that Azure follows OpenAI's list price.
Azure has not adopted the cut: Microsoft still publishes $1.00/1M input and
$6.00/1M output for luna global, and $1.10/$6.60 for the us/eu data zones.

That made LiteLLM report 5x less than Azure actually bills for luna and 20%
less for terra, on every azure/ gpt-5.6 deployment.

Restore the 60 affected fields across the six azure global/us/eu terra and luna
rows to their pre-4d43080a7 values, in both cost maps. Input, output,
cache-read, priority, and above-272k variants all scale together. sol is
untouched, as are the openai/ and bedrock_mantle rows, whose cut is correct.

The regression test parametrization already covered the global and eu rows;
add the two us rows so azure/us/gpt-5.6-luna, the deployment named in the
report, is pinned too.

Fixes BerriAI#36094
@greptile-apps

greptile-apps Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR restores Azure GPT-5.6 terra and luna pricing across global, US, and EU entries while keeping the canonical and packaged backup maps synchronized.

  • Updates base, cache-read, priority, and long-context rates.
  • Expands cost-calculation regression coverage for selected priority and long-context paths.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Restores the canonical Azure GPT-5.6 terra/luna rates and preserves the documented regional uplift relationships.
litellm/model_prices_and_context_window_backup.json Mirrors the canonical pricing changes exactly for packaged and offline lookup paths.
tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py Adds representative regression cases for base, priority, and long-context cost selection.

Reviews (2): Last reviewed commit: "test(pricing): cover the azure gpt-5.6 p..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing olifarhaan:litellm_azure_gpt56_pricing (506adb3) with litellm_internal_staging (c28cbb8)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (e24a914) during the generation of this report, so c28cbb8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@olifarhaan

Copy link
Copy Markdown
Author

@greptileai tier-rate coverage added in 925c669, please take another look

The existing parametrization only exercises base input/output/cache-read, so
40 of the 60 restored fields were unpinned: a typo in a priority or
above-272k rate, or a key the calculator never reads, would fall back to base
rates and misreport spend for those tiers without failing anything.

Drive generic_cost_per_token across both tiers and both context bands for the
global terra/luna rows, plus one us and one eu row. All nine cases fail
against the pre-fix cost map.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: azure/gpt-5.6-luna under-reports cost by 5x on main (Regression after v1.95.0)

1 participant