Skip to content

fix(pricing): map Azure GPT-5.6 Luna snapshot - #41433

Closed
devin-ai-integration[bot] wants to merge 6 commits into
mainfrom
litellm_azure_gpt56_luna_snapshot_alias
Closed

devin-ai-integration[bot] wants to merge 6 commits into
mainfrom
litellm_azure_gpt56_luna_snapshot_alias

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Azure returns the dated gpt-5.6-luna-2026-07-09 model id
  • That id has no cost map entry, so spend is logged at $0

How it solves it:

User Flow

Before: a developer calling GPT-5.6 Luna through Azure sees every chat completion logged at $0

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "azure/gpt-5.6-luna"
  2. The 200 response reports "model": "gpt-5.6-luna-2026-07-09" with real token usage
  3. https://litellm-domain/ui/?page=logs shows the request at $0 spend

After: the same request is priced

  1. They send POST https://litellm-domain/v1/chat/completions with "model": "azure/gpt-5.6-luna"
  2. The 200 response reports "model": "gpt-5.6-luna-2026-07-09" with real token usage
  3. https://litellm-domain/ui/?page=logs shows the request at the Luna price

Relevant issues

Fixes #35762

Supersedes #35783, which targets litellm_internal_staging and cannot be rebased in place because its fork branch is not pushable from here. Commits are kept with the original author. The aliases approach from #35783 was swapped for a full entry, matching the other dated azure/gpt-5.x keys, because the cost-map-guard runs the default branch's schema generator against the PR JSON and rejects a key that the same PR declares. Overlaps with #41423, which strips dated suffixes generically; this PR is the narrower data-only fix

Affected release

Linear ticket

Resolves LIT-7863

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

This box has no provider egress, so the proof is the public completion_cost() path with an Azure-shaped response of 100 prompt and 50 completion tokens, using the local cost map

Before (9cd7873)

  1. LITELLM_LOCAL_MODEL_COST_MAP=True uv run python repro.py
  2. Output: cost=0.00000000

After (643620a)

  1. LITELLM_LOCAL_MODEL_COST_MAP=True uv run python repro.py
  2. Output: dated_key_present=True and cost=0.00008000

Type

🐛 Bug Fix

Caveats (if any)

Medium

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/cc74f675691a4d5e909ecdf06aff2c10
Open in Devin Desktop: https://app.devin.ai/desktop/session/cc74f675691a4d5e909ecdf06aff2c10?variant=devin

@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 16, 2026 16:03
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@codspeed

codspeed Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_azure_gpt56_luna_snapshot_alias (643620a) with main (29a712b)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds a complete Azure pricing entry for the dated GPT-5.6 Luna snapshot so response-based cost calculation no longer records zero spend. The regression test was moved into the existing mapped Azure cost test file, fully addressing the previous repository-rule finding

  • Keeps the primary and backup cost maps synchronized
  • Verifies the snapshot and canonical entries agree
  • Verifies completion_cost() prices an Azure-shaped response using the dated model ID

Confidence Score: 5/5

The PR appears safe to merge because the dated snapshot is mapped consistently and the regression test exercises the affected public cost path

No actionable new issues remain. The previous misplaced-test finding is fully fixed by moving the regression coverage into the existing Azure cost calculation test file

Important Files Changed
Filename Overview
model_prices_and_context_window.json Adds the dated Azure GPT-5.6 Luna snapshot with pricing and capabilities matching the canonical entry
litellm/model_prices_and_context_window_backup.json Mirrors the dated snapshot entry in the bundled backup cost map
tests/test_litellm/llms/azure/test_azure_cost_calculation.py Adds a focused regression test for dated-snapshot lookup and completion cost calculation in the existing mapped test file

Reviews (2): Last reviewed commit: "test(pricing): drop standalone Luna snap..." | Re-trigger Greptile

The cost-map-guard runs the default branch's schema generator against the PR's JSON, so a new aliases key cannot land in the same PR that declares it. Use a full entry like the other dated azure gpt-5.x snapshots instead

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
from litellm.types.utils import Choices, Message, ModelResponse, Usage


def test_completion_cost_supports_luna_dated_snapshot(monkeypatch):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Misplaced regression test

This bug fix creates a new test file, but repository rules require extending the existing mapped Azure cost test before merging

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ Linxiushen
❌ devin-ai-integration[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…st file

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Superseded by rolling registry PR #41112, which already contains the azure/gpt-5.6-luna-2026-07-09 snapshot with the same values

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Azure gpt-5.6-luna dated snapshot gpt-5.6-luna-2026-07-09 missing from cost map

2 participants