Skip to content

fix(model-costs): apply GPT-5.6 Sol promotional pricing cut - #37880

Merged
mateo-berri merged 4 commits into
litellm_internal_stagingfrom
litellm_gpt56_sol_promo_pricing
Aug 22, 2026
Merged

fix(model-costs): apply GPT-5.6 Sol promotional pricing cut#37880
mateo-berri merged 4 commits into
litellm_internal_stagingfrom
litellm_gpt56_sol_promo_pricing

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • gpt-5.6-sol got a promotional price cut, the map still had the old rates
  • Every Sol request was billed 25% over on input and 50% over on output
  • daybreak-blue-latest and the bare gpt-5.6 alias both route to Sol, so they overbilled the same way

How it solves it:

  • Repriced gpt-5.6-sol standard, batch, flex and fast keys
  • Long-context (>272K) rates repriced along with short context
  • Same cut applied to daybreak-blue-latest and to the bare gpt-5.6 alias
  • Added a regression test pinning the gpt-5.6 alias to gpt-5.6-sol on all 23 cost fields
  • Updated the pinning tests to the new published rates

User Flow

Before: a developer calling GPT-5.6 Sol through the gateway, by any of its three names, sees spend well above what OpenAI actually invoices

  1. They send POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5.6-sol", ...} and get back 13 prompt tokens, 4 completion tokens
  2. The response carries x-litellm-response-cost: 0.000185, priced at $5.00 in and $30.00 out per 1M
  3. The same request with "service_tier": "fast" comes back at x-litellm-response-cost: 0.00037, priced at $10.00 in and $60.00 out
  4. They switch to {"model": "gpt-5.6", ...}, which OpenAI routes to the same Sol model, and get the same inflated 0.000185 and 0.00037
  5. https://litellm-domain/ui/?page=logs shows that inflated spend, so budgets burn down faster than the real bill

After: the same calls are billed at the promotional rates OpenAI publishes, whichever name the caller used

  1. They send the same POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5.6-sol", ...} for the same token counts
  2. The response carries x-litellm-response-cost: 0.000132, priced at $4.00 in and $20.00 out per 1M
  3. The "service_tier": "fast" variant comes back at x-litellm-response-cost: 0.000264, priced at $8.00 in and $40.00 out
  4. The {"model": "gpt-5.6", ...} call now matches at 0.000132 and 0.000264 instead of billing the pre-cut rate
  5. https://litellm-domain/ui/?page=logs shows spend matching the OpenAI invoice

Relevant issues

Linear ticket

LIT-5985

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Rates come from https://developers.openai.com/api/docs/pricing (Standard, Batch, Flex and Fast mode tables), which also notes the promo runs at least through November 21, 2026. Sol short context is now $4.00 in, $0.40 cached, $5.00 cache write, $20.00 out, and long context is $8.00 / $0.80 / $10.00 / $30.00

Why the bare gpt-5.6 alias is in scope

OpenAI's model docs state the gpt-5.6 alias routes requests to GPT-5.6 Sol, and a live call confirms it: sending {"model": "gpt-5.6"} straight to OpenAI comes back with "model": "gpt-5.6-sol" in the response body. The alias entry held all 23 cost fields at the pre-cut Sol values, so anyone calling the short name was overbilled by 25% on input and 50% on output while the Sol entry itself was correct. The new test_gpt_5_6_alias_prices_match_sol pins the two entries together so they cannot drift apart again

Shared setup, a live proxy against real OpenAI with both deployments:

model_list:
  - model_name: gpt-5.6
    litellm_params:
      model: openai/gpt-5.6
      api_key: os.environ/OPENAI_API_KEY
  - model_name: gpt-5.6-sol
    litellm_params:
      model: openai/gpt-5.6-sol
      api_key: os.environ/OPENAI_API_KEY
general_settings:
  master_key: sk-1234

Each leg ran in its own worktree with its own venv, its own random free port and LITELLM_LOCAL_MODEL_COST_MAP=True, so each proxy loaded the cost map committed at that leg's own commit. Before ran at bb99f5774e, this PR's merge base against litellm_internal_staging, and After at the tip. Real OpenAI, no mocks. Because the change touches shared cost code, every leg covers all three unified endpoints rather than just the one the User Flow narrates

The call, with "service_tier":"fast" added to the body for the fast legs:

curl -sS -D - -X POST http://localhost:$PORT/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Reply with the single word: ok"}],"max_completion_tokens":16}'

Before (bb99f57, the merge base)

Model Endpoint Tier Tokens x-litellm-response-cost Expected
gpt-5.6-sol /v1/chat/completions standard 13 / 4 0.000185 0.000185
gpt-5.6-sol /v1/responses standard 13 / 5 0.000215 0.000215
gpt-5.6-sol /v1/messages standard 13 / 5 0.000215 0.000215
gpt-5.6-sol /v1/chat/completions fast 13 / 4 0.00037 0.00037
gpt-5.6 /v1/chat/completions standard 13 / 4 0.000185 0.000185
gpt-5.6 /v1/responses standard 13 / 5 0.000215 0.000215
gpt-5.6 /v1/messages standard 13 / 5 0.000215 0.000215
gpt-5.6 /v1/chat/completions fast 13 / 4 0.00037 0.00037

Expected is the pre-cut rate, $5.00 in and $30.00 out per 1M, doubled for fast. Split headers matched independently: the Sol standard chat leg returned x-litellm-response-cost-input: 6.500000000000001e-05 and x-litellm-response-cost-output: 0.00012. /v1/model/info at this commit reported input_cost_per_token 5e-06 and output_cost_per_token 3e-05 for both names

After (1a55418, the tip)

Model Endpoint Tier Tokens x-litellm-response-cost Expected
gpt-5.6-sol /v1/chat/completions standard 13 / 4 0.000132 0.000132
gpt-5.6-sol /v1/responses standard 13 / 5 0.000152 0.000152
gpt-5.6-sol /v1/messages standard 13 / 5 0.000152 0.000152
gpt-5.6-sol /v1/chat/completions fast 13 / 4 0.000264 0.000264
gpt-5.6 /v1/chat/completions standard 13 / 4 0.000132 0.000132
gpt-5.6 /v1/responses standard 13 / 5 0.000152 0.000152
gpt-5.6 /v1/messages standard 13 / 5 0.000152 0.000152
gpt-5.6 /v1/chat/completions fast 13 / 4 0.000264 0.000264

Expected is the promotional rate, $4.00 in and $20.00 out per 1M, doubled for fast. Split headers matched here too: 5.2e-05 and 8e-05 on the Sol standard chat leg. The two model names now agree on every endpoint, which is the whole point of the alias half of the fix. The QA legs ran at 71400e1029; 1a55418ea2 only swaps the alias test onto the conftest local_model_cost_map fixture and cannot change billing

Scope check on the map itself

Diffing both cost maps across bb99f5774e..1a55418ea2 shows exactly 3 of 3165 entries changed (gpt-5.6, gpt-5.6-sol, daybreak-blue-latest), no entries added or removed, and no key added or removed on any entry, so nothing reading the map can break structurally. Every delta is a decrease: x0.8 on every input-side field and x0.6667 on every output-side field. All the family's internal rate ratios still hold afterwards (cache read 0.1x input, cache write 1.25x, batch and flex 0.5x, priority 2x, long-context input 2x and output 1.5x)

Observations from the run

  • Fast tier echoes back as service_tier: "priority", not "fast"
  • Fast billing draws on the _priority cost keys, no _fast key exists
  • Responses and messages bill 5 output tokens where chat bills 4
  • Responses body carries usage.cost: null despite a correct cost header
  • daybreak-blue-latest has no flex, priority or batch keys at all
  • The gpt-5.6 alias lacks supports_computer_use that Sol carries

The last two are preexisting and untouched by this PR, split out as LIT-5986 and LIT-5992

Type

🐛 Bug Fix

Caveats (if any)

  • Promo pricing, revisit after November 21, 2026. Tracked as LIT-5988 with that due date
  • Azure Sol entries are deliberately left at the old rates. Azure publishes no GPT-5.6 Sol price at all right now, its pricing page renders placeholders and the Retail Prices API returns no GPT-5.6 meters, so there is nothing to cut them to
  • Bedrock Sol entries are deliberately left alone too. The AWS model card still lists $5.50 in and $33.00 out in-region, and the launch note says pricing for GPT-5.6 Sol remains unchanged. Our bedrock_mantle entry already matches that card on all eight fields
  • Fast and priority requests above 272K still bill the standard long-context rate across this family, because the _above_272k_tokens_priority fields are missing from twelve entries. That is a preexisting gap unrelated to the promo, so it is split out as LIT-5986
  • Four entries hold a context-window number in max_input_tokens (daybreak-blue-latest at 1050000, the three bedrock_mantle gpt-5.6 entries at 1000000, all of which should be 922000). Also preexisting and unrelated to pricing, split out as LIT-5987
  • The gpt-5.6 alias is missing supports_computer_use, which both gpt-5.6-sol and daybreak-blue-latest carry. Same alias-drift shape as the pricing bug, but it is a capability claim rather than a rate, so it wants its own check against OpenAI's docs. Split out as LIT-5992

Link to Devin session: https://app.devin.ai/sessions/49ef693b0c6647599ce2f2dac8c9ab9b


Note

Medium Risk
Changes billed token rates for a live model family, so incorrect numbers would under/overcharge spend tracking. Scope is limited to three OpenAI cost-map entries plus tests; Azure/Bedrock are unchanged.

Overview
Lowers OpenAI GPT-5.6 Sol billing to the published promo rates so spend matches the invoice instead of overcharging ~25% on input and ~50% on output.

Updates gpt-5.6-sol, the gpt-5.6 alias (OpenAI routes it to Sol), and daybreak-blue-latest across standard, batch, flex, priority/fast, cache, and >272K keys. Azure and Bedrock Sol entries are left at their own published rates.

Adds test_gpt_5_6_alias_prices_match_sol so the alias cannot drift from Sol on cost fields again, and retargets existing cost tests to the new rates.

Reviewed by Cursor Bugbot for commit 1a55418. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

CLAassistant commented Aug 21, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ Mubashir Osmani


Mubashir Osmani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR applies GPT-5.6 Sol promotional pricing across the canonical and backup cost maps and updates regression coverage.

  • Reprices standard, batch, flex, priority, cache, and long-context fields for gpt-5.6, gpt-5.6-sol, and daybreak-blue-latest.
  • Keeps the canonical and backup pricing maps synchronized.
  • Updates cost-calculation expectations and adds alias-to-Sol pricing parity coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Updates the affected OpenAI GPT-5.6 family pricing fields consistently across tiers, caching modes, and context thresholds.
litellm/model_prices_and_context_window_backup.json Mirrors the canonical pricing-map changes without detected divergence.
tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py Updates tier-specific expectations and adds regression coverage requiring the bare GPT-5.6 alias to match Sol pricing.
tests/test_litellm/test_cost_calculator.py Updates the end-to-end cache-aware completion-cost expectation to the promotional rates.

Reviews (3): Last reviewed commit: "test(model-costs): use the local_model_c..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_gpt56_sol_promo_pricing (1a55418) with litellm_internal_staging (9c558df)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (2cb85da) during the generation of this report, so 9c558df was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

OpenAI's model page for gpt-5.6 serves the GPT-5.6 Sol page and states
that the gpt-5.6 alias routes requests to GPT-5.6 Sol, so the alias bills
at Sol's rates. The registry entry was left on the pre-cut rates while
gpt-5.6-sol took the cut, overbilling gpt-5.6 callers by 25 percent on
input and 50 percent on output.

All 23 cost fields on gpt-5.6 now match gpt-5.6-sol, and a regression
test pins the two entries together so they cannot drift again.
The docstring claimed azure pricing mirrors the openai family, which stopped
being true when gpt-5.6-sol took its promotional cut and azure did not. Azure
publishes no sol rate of its own today, so the entries stay where they are.
@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 71400e1. Configure here.

…test

The new test set LITELLM_LOCAL_MODEL_COST_MAP and reassigned litellm.model_cost
by hand, leaking both into every test that ran after it and skipping the
get_model_info cache clear. The conftest fixture already does this properly and
restores the original map on the way out.
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 1a55418. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@mateo-berri
mateo-berri merged commit 76d4627 into litellm_internal_staging Aug 22, 2026
71 checks passed
@mateo-berri
mateo-berri deleted the litellm_gpt56_sol_promo_pricing branch August 22, 2026 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants