Repository navigation
feat(azure): price GPT-6 Sol and GPT-6 Luna on the azure route - #42704
Closed
marty-sullivan wants to merge 1 commit into
Closed
marty-sullivan wants to merge 1 commit into
marty-sullivan wants to merge 1 commit into
Conversation
An Azure deployment of either model had no azure/ entry, so the cost map answered two different wrong ways. azure/gpt-6-sol and the dated snapshot raised ModelNotMappedError, and azure/us/gpt-6-sol fell through to 0.0, which a proxy records as a real $0.00 spend row. Adds six keys, the Standard Global, dated snapshot and US data zone shapes for each model, following the azure/gpt-6-astra rows PR BerriAI#39827 landed. Rates are derived from the OpenAI direct rows: parity on the global shape, since Foundry resells generation 6 at OpenAI's list price, and the standard 1.1x on the US data zone. Flex, priority and batch rates are dropped along with /v1/batch, because Azure publishes Std meters only for this generation. supports_max_reasoning_effort is false rather than the true the OpenAI rows carry. A live call to a real deployment of each model answers reasoning_effort max with a 400, "Unsupported value: 'reasoning_effort' does not support 'max' with this model. Supported values are: 'none', 'low', 'medium', 'high', and 'xhigh'.", and xhigh returns 200. That is the same evidence e79f3ec acted on for azure/gpt-6-astra, so all three shapes per model now match those rows. No EU data zone row. Microsoft's own DZ meter splits by region for generation 6, $11.00 across seven US regions against $12.00 across fourteen that include nine EU ones, so an EU row is 1.2x rather than the 1.1x azure/eu/gpt-6-astra carries. With no published sol or luna meter there is nothing to cite at either value. Azure's Retail Prices API publishes no meter for either model as of 2026-09-23, so these rates are derived rather than cited, and the PR stays a draft until they can be reconciled against published figures.
Contributor
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced Sep 23, 2026
Merged
Contributor
|
Closing: these rows already landed on main with the same prices, and the rolling registry PR #42951 tracks the remaining unverified fields. Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
gpt-6-solandgpt-6-lunatodayazure/cost map rowx-litellm-response-costheader comes back eitherreasoning_effort: max, which Azure rejectsHow it solves it:
azure/gpt-6-astrarowsreasoning_effortmax marked unsupported, from a live probeUser Flow
Before: a developer calling an Azure
gpt-6-soldeployment through the gateway gets no spend data atall, and nothing in the response says so
model: azure/gpt-6-soland anapi_basepointing at theirAzure resource, then restarts the proxy
POST https://litellm-domain/v1/chat/completionswith"model": "gpt-6-sol"and a short prompt
200comes back with real content and ausageobject carrying real token countsx-litellm-response-costheader arrives at all, so a client reading that header to track costsilently gets nothing
https://litellm-domain/ui/?page=logs, find the request, and the row reads $0model: azure/us/gpt-6-solfor the US data zone and sends thesame request. It behaves the same way, no cost header and a $0 log row
deployment cannot fire
GET https://litellm-domain/model_group/infowhich reasoning efforts the deployment offerslists
max, so the developer tries"reasoning_effort": "max"and Azure answers400After: both deployments bill, the data zone one bills more than the global one, and the effort list
matches what Azure accepts
model: azure/gpt-6-soland anapi_basepointing at theirAzure resource, then restarts the proxy
POST https://litellm-domain/v1/chat/completionswith"model": "gpt-6-sol"and a short prompt
200comes back with real content and ausageobject carrying real token countsx-litellm-response-costnow reads a dollar amount matching the token counts at $2.00 per millioninput and $10.00 per million output
https://litellm-domain/ui/?page=logs, find the request, and it shows that same non-zerospend
model: azure/us/gpt-6-soland sends the same request. The cost comes back10% higher, which is the US data zone uplift
GET https://litellm-domain/model_group/infono longer listsmax, so the developer sees the fiveefforts Azure actually accepts before sending anything
Relevant issues
No issue asks for these entries. Related but not duplicates: #42515 added the two bare OpenAI rows and
nothing Azure, #39827 added the
azure/gpt-6-astrarows this follows, and #42594 syncs Azure priceswithout touching either of these models
Affected release
Linear ticket
Pre-Submission checklist
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Captured 2026-09-23 against a real Azure resource serving both models, so every call below is a real
billed call. Nothing is mocked and no pytest run stands in for any of it
Shared setup, used for both Before and After.
One note on how the six cost map rows get exercised, because it changes the config. The real Azure
deployments are named
gpt-6-solandgpt-6-luna, and Azure's deployment path is literal: adeployment string of
us/gpt-6-solorgpt-6-sol-2026-09-22makes the proxy call/openai/deployments/us/gpt-6-sol/..., which Azure answers404 Resource not found. So the data zoneand dated rows are priced by pointing
base_modelat them whilemodelkeeps the deployment thatexists. That is the documented way to steer an Azure cost lookup and the repo's own docs name this exact
case:
terraform/provider/docs/resources/model.mddescribes an Azure deployment routed asazure/gpt-4.1whose real tier is Data Zone, priced by setting the key tous/gpt-4.1-2025-04-14. Itis also what a real operator does, since an Azure deployment is not named after its data zone either
config.yaml:Env vars the proxy ran with:
AZURE_API_BASE(the resource endpoint),AZURE_API_KEY(redacted),LITELLM_MASTER_KEY(redacted, shown assk-REDACTEDin the curls below),DATABASE_URL(a localPostgres, so the spend rows are readable),
STORE_MODEL_IN_DB=False, andLITELLM_LOCAL_MODEL_COST_MAP=True. That last one matters: without it the proxy pulls the cost map offthe network and the Before run would read the published map rather than the checked-out one, which
would prove nothing
The driver loop, reused verbatim at both commits. It sends the same prompt to all six model groups and
prints the status line, the cost header, and the token counts:
Before (
721d39f476)Spend on all six shapes
Start the proxy at the merge base:
Run the driver loop. Every call returns
200with real content and real token counts, and not oneof them carries an
x-litellm-response-costheader:The
grepprints nox-litellm-response-costline for any of the six, so those lines are genuinelyabsent rather than trimmed from this paste. The full header block on the first call shows what does
come back, a breakdown of zeros with the total missing:
Show the proxy log naming the models it could not price:
Six such lines in total, one per model group
The proxy also warns that it billed a real request at zero, which is the clearest single line in the
log:
Four such warnings fire, for
gpt-6-sol,gpt-6-sol-dated,gpt-6-lunaandgpt-6-luna-dated.Worth flagging that the two
usshapes get no warning at all despite billing $0 the same way,because the diagnostic only speaks up when the pricing entry it found declares some non-zero rate,
and the data zone spelling resolves to a synthesized entry that is all zeros. So on the data zone
rows the $0 is silent in the log as well as in the response
Read the spend rows back:
Six real billed calls, six rows, $0.00 on every one
Reasoning efforts the proxy advertises
Ask the proxy what reasoning efforts it offers:
With no
azure/row of their own these fall back to the bare OpenAI rows, which do supportmax,so all six advertise it. The Azure probe in the next case shows Azure answering
400to exactlythat value
Azure's own answer on reasoning_effort max
This case probes Azure directly rather than the proxy, so it reads the same at either commit and the
output is identical under Before and After. It is shown in full under After, since that is where it
lines up with the flag this PR sets. The short version:
gpt-6-solandgpt-6-lunaboth answer400to
reasoning_effort: maxand200toxhigh, at this commit and at the tip alikeAfter (
d07d31c8e1)Spend on all six shapes
Start the proxy at the PR tip, same config, same env:
Run the same driver loop. Every call now carries a cost header, and the model answered with the
same 13 in / 4 out on all six, which makes the six numbers directly comparable:
The mapping errors are gone as well. The same
grepthat printed six lines before prints none:The zero-cost warning is gone too. The same
grepthat matched four times before matches nothing:Reconcile each cost against its row's rates and the reported 13 in / 4 out. Expected is
13 * input_cost_per_token + 4 * output_cost_per_token:gpt-6-solazure/gpt-6-solgpt-6-sol-datedazure/gpt-6-sol-2026-09-22gpt-6-sol-usazure/us/gpt-6-solgpt-6-lunaazure/gpt-6-lunagpt-6-luna-datedazure/gpt-6-luna-2026-09-22gpt-6-luna-usazure/us/gpt-6-lunaEvery one matches to the last bit, with no rounding applied on either side. The two luna figures
carry a float tail because
13 * 1e-07 + 4 * 5e-07is not exactly representable in binary; theexpected value carries the identical tail, which is why they compare equal rather than approximately
equal. The uplift lands where it should too:
7.26e-05 / 6.6e-05and3.63e-06 / 3.3e-06are both1.1, so each data zone row is exactly 1.1x its global twin on the same token counts
The per-direction headers reconcile independently, which separates the input rate from the output
rate rather than only checking their sum. On
gpt-6-sol:Read the spend rows back with the same command as Before:
Each row equals its cost header, so the number a client reads and the number a budget spends against
agree
Reasoning efforts the proxy advertises
Ask the proxy what reasoning efforts it offers, same command as Before:
maxis gone from all six and the remaining five match Azure's own list in the400below, wordfor word
Azure's own answer on reasoning_effort max
This one runs straight at the Azure endpoint rather than through the proxy, because it establishes a
vendor behaviour rather than a gateway one, so it reads the same at either commit and is where the
supports_max_reasoning_effort: falseflag comes from. Run on 2026-09-23 against a real deployment ofeach model.
Ask each deployment for
reasoning_effort: max:gpt-6-solandgpt-6-lunaboth return:gpt-6-astrareturns the same rejection, with'low', 'medium', 'high', and 'xhigh'as its list.The lists differ on whether
noneappears, which is a real difference between the models ratherthan noise, and no conclusion about
nonesupport is drawn from it here. What matters for this PRis that all three refuse
maxAsk each for
reasoning_effort: xhigh, the next level down:So
xhighstays flagged and onlymaxcomes offType
🆕 New Feature
Caveats (if any)
Severe
The rates are derived, not published. That is why this is a draft
Microsoft publishes no retail meter for either model as of 2026-09-23, and I checked that
positively rather than assuming it. The Azure Retail Prices API under
serviceName eq 'Foundry Models'returns 34,448 rows over 1,863 distinct meter names, and exactly 16 of those names belongto generation 6. All 16 are astra, and none is sol or luna. Two controls say the query works:
6-astra ShortCo Inp Std Gl 1M Tokensreads $10.00, matchingazure/gpt-6-astra, and5.6 sol ShortCo Inp Std Gl 1M Tokensreads $4.00, matchingazure/gpt-5.6-sol. Worth noting for anyone re-checking twoways to get a false zero here: the service was renamed from
Cognitive ServicestoFoundry Models, and no meter name in this family contains the substringgpt, so acontains(meterName, 'gpt-6')filter returns nothingSo the numbers here are OpenAI's published direct price at parity on the global shape, plus the
standard 1.1x on the US data zone shape. I am not presenting that as a citation. It is the same
derivation
azure/gpt-6-astraalready encodes, and it matches the map's dominant convention: of the88
azure/<model>rows that have a bare OpenAI twin, 83 sit at exact parity on every shared rate,and of the 108 data-zone rows with a global twin, 91 sit at exactly 1.1x with the rest differing only
by rounding. The deployments serve live today, which is what motivated this. Reconcile against
whatever Microsoft publishes before this is marked ready
Medium
No EU data zone row, on purpose, and copying astra's would be wrong
Microsoft's generation-6 data zone meter is not one price, it splits by region.
6-astra ShortCo Inp Std DZ 1M Tokensreads $11.00 across 7 regions, none of them EU, and $12.00 across 14 regions ofwhich 10 are EU. So gen-6 US data zone is 1.1x and the bucket holding the EU regions is 1.2x.
azure/eu/gpt-6-astracarries1.1e-05, the US bucket's number, which reads low against its ownmeter. That row came from the automated price-sync bot in
41caaa301c, not from either astra PR,which is some evidence it was never weighed by hand
With no published sol or luna meter there is nothing to cite at 1.1x or at 1.2x, so shipping an EU
row would mean either guessing a second number or propagating astra's. Note this also differs from
generation 5.6, whose DZ meter is a flat 1.1x for US and EU alike, so the multiplier cannot be
carried forward from that generation either. An operator on an EU data zone deployment gets no
pricing from this PR and still needs one
Fixing
azure/eu/gpt-6-astrais left out to keep the scope to one problemThe finding above is actionable on its own and I would rather not fold a repricing of a merged row
into a PR that adds new ones. Happy to send it separately
Low
Flex, priority and batch rates are deliberately absent, and
/v1/batchwith themAll 16 generation-6 meters are
Std. There is noFl, noPPand no batch SKU anywhere in thefamily, where 5.6 does have
PP. Carrying OpenAI's flex, priority and batch numbers would price atier Azure does not sell, so these rows match
azure/gpt-6-astra's reduced field set rather thanthe OpenAI row's full one. A test asserts that, so a later sync cannot quietly re-add them
The dated snapshot rows are a spelling, not a requirement
azure/gpt-6-astra-2026-09-03is one of 89 datedazure/keys on this branch, so these follow anestablished convention and specifically the one the sibling gen-6 model already has. Being straight
about what they buy, though:
_strip_dated_snapshot_suffix(67778cfe26) already resolves a datedmodel string back to its undated row, so a call would price correctly without them. They are belt
and braces plus consistency with the astra precedent, and the test on them asserts the only thing
that can go wrong, which is the dated row drifting from the undated one
supports_prompt_cache_breakpointis kept on all six rows, which is a small divergenceNone of the 277 pre-existing
azure/rows carries this flag, andazure/gpt-6-astradoes not,while both bare gen-6 rows and
azure_ai/gpt-6-astrado. Keeping it makes the six rows agree witheach other, because the flag resolves through a bare-twin fallback that only fires for the exact
azure/<model>spelling: with the flag dropped,azure/gpt-6-solreads true whileazure/us/gpt-6-soland the dated key read false, which is the state astra is in today. Droppingit here would be closer to the
azure/convention at the price of three rows disagreeing about onemodel's capability
No
azure_ai/rowsazure_ai/gpt-6-astraexists, so there is a precedent, but it arrived in its own later PR (feat(cost-map): add azure_ai/gpt-6-astra Foundry pricing #39983)rather than in either astra PR, it cites the Foundry catalog rather than the retail price sheet, and
only 16 of the 131
azure/<model>rows have anazure_ai/twin at all. Adding one here would meaninventing a second unpublished citation for the same unpublished rate. Straightforward to add if
someone is serving these through a Foundry project
No
sourcefield on any of the six rowsEvery
azure/gen-6 sibling pointssourceat a retail-prices query that contains no sol or lunameter, so reusing it would dress a derivation up as a citation. 97 of the 277
azure/rows carry nosourceat all, so omitting it is within convention. Add the real URL when Microsoft publishes oneFinal Attestation