Skip to content

fix(pricing): add undated azure aliases for gpt-audio-mini and gpt-realtime-mini - #33291

Closed
mihidumh wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
mihidumh:fix/azure-audio-mini-price-aliases
Closed

fix(pricing): add undated azure aliases for gpt-audio-mini and gpt-realtime-mini#33291
mihidumh wants to merge 4 commits into
BerriAI:litellm_internal_stagingfrom
mihidumh:fix/azure-audio-mini-price-aliases

Conversation

@mihidumh

Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #33170

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Production observation (litellm proxy 1.92.0 on Azure, real Azure gpt-audio-mini deployment, real audio-modality completions): with no undated azure/gpt-audio-mini map key, spend logs and the x-litellm-response-cost header billed text tokens at $0 while audio tokens billed correctly. We verified by putting the correct prices in litellm_params custom pricing on the deployment row — after which the cost header matched the hand-computed text+audio math exactly ($0.60/M in, $2.40/M out — the same numbers as the dated azure/gpt-audio-mini-2025-10-06 entry this PR mirrors).

Map state before (base 10d5804b3e):

$ jq 'has("azure/gpt-audio-mini"), has("azure/gpt-realtime-mini")' model_prices_and_context_window.json
false
false
$ jq -r 'keys[] | select(test("^azure/gpt-(audio|realtime)-mini"))' model_prices_and_context_window.json
azure/gpt-audio-mini-2025-10-06
azure/gpt-realtime-mini-2025-10-06

After (2f906a3415):

$ pytest tests/test_litellm/test_azure_audio_price_aliases.py -q
2 passed
# on base: 2 failed (ValueError: This model isn't mapped yet)

The new test resolves both undated names through litellm.get_model_info (local cost map fixture, same pattern as test_cloudflare_workers_ai_model_metadata.py) and asserts every cost field matches the dated entry and is non-zero.

Note on the second commit: running the repo's own ci_cd/check_files_match.py sync tool surfaced pre-existing drift on the daily branch (9 entries in the root map missing from the packaged backup — gemini-3-pro-image, zai/glm-5.1, etc.). That sync is kept as a separate chore: commit so the alias fix itself stays a clean 63-line-per-file addition. Happy to drop that commit if you'd rather take the drift separately.

Type

🐛 Bug Fix

Changes

  • Add azure/gpt-audio-mini and azure/gpt-realtime-mini to model_prices_and_context_window.json (and the packaged backup), each an exact mirror of its -2025-10-06 dated entry — same relationship as the undated openai gpt-audio-mini / gpt-realtime-mini entries.
  • Parametrized test asserting the undated aliases resolve and match the dated entries with non-zero costs.

@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a billing bug (issue #33170) where Azure deployments using the undated gpt-audio-mini or gpt-realtime-mini model names had text tokens billed at $0, because those keys were absent from the model cost map. It adds the two missing aliases as exact mirrors of their -2025-10-06 dated entries and syncs pre-existing drift between the root JSON and the packaged backup.

  • Adds azure/gpt-audio-mini and azure/gpt-realtime-mini to both model_prices_and_context_window.json and its backup, each field-for-field identical to the corresponding dated entry — verified by direct inspection of both files.
  • Adds two parametrized test functions: one checking the four core COST_FIELDS are equal and non-zero, and a second (test_undated_azure_audio_alias_is_exact_mirror) doing full dictionary equality against litellm.model_cost, covering every field including realtime-specific cache_creation_input_audio_token_cost, cache_read_input_token_cost, and input_cost_per_image.

Confidence Score: 5/5

Safe to merge — the change is additive data only (two new JSON keys) with no modifications to existing entries or runtime logic, and the new test suite guards the critical invariant going forward.

Both undated alias entries are confirmed to be byte-for-byte copies of their dated counterparts in both the root and backup JSON files. The test file is mock-only (no network calls), uses the local cost-map fixture correctly, and the new full-equality test closes the gap identified in the prior review round by covering every field including realtime-specific cache and audio cost keys.

No files require special attention. The backup drift-sync entries are straightforward catch-up additions that were already present in the root map.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds azure/gpt-audio-mini and azure/gpt-realtime-mini as exact mirrors of their -2025-10-06 dated counterparts; no other changes to the root map.
litellm/model_prices_and_context_window_backup.json Adds the two azure aliases (identical to root), then syncs nine previously missing entries (Gemini image models, vertex_ai aliases, openrouter/zai models, token-limit corrections, snowflake adaptive-thinking flag) from the root map into the backup. Both undated azure entries verified to be field-for-field identical to their dated siblings.
tests/test_litellm/test_azure_audio_price_aliases.py New mock-only test file (uses local cost-map fixture, no network calls). Provides two parametrized tests: COST_FIELDS equality + non-zero guards, and a full dictionary equality check (test_undated_azure_audio_alias_is_exact_mirror) covering every field including realtime-specific cache/audio cost keys.

Reviews (2): Last reviewed commit: "test(pricing): assert undated azure audi..." | Re-trigger Greptile

Comment thread tests/test_litellm/test_azure_audio_price_aliases.py
@mihidumh

Copy link
Copy Markdown
Contributor Author

@greptileai — added a full-entry equality test asserting each undated alias exactly mirrors its dated sibling, covering the realtime-specific cost keys COST_FIELDS missed.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mihidumh
mihidumh force-pushed the fix/azure-audio-mini-price-aliases branch from 8966298 to dc0dbc3 Compare July 22, 2026 22:43
@mihidumh
mihidumh requested a review from a team July 22, 2026 22:43
@mihidumh
mihidumh changed the base branch from litellm_oss_daily_2026_07_14 to litellm_oss_daily_2026_07_20 July 22, 2026 22:43
@mihidumh
mihidumh changed the base branch from litellm_oss_daily_2026_07_20 to litellm_internal_staging July 28, 2026 23:13
@mihidumh
mihidumh force-pushed the fix/azure-audio-mini-price-aliases branch from dc0dbc3 to 483966a Compare July 28, 2026 23:22
@codspeed-hq

codspeed-hq Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing mihidumh:fix/azure-audio-mini-price-aliases (e189a58) with litellm_internal_staging (47a7e17)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (5290150) during the generation of this report, so 47a7e17 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

mihidumh and others added 3 commits August 20, 2026 15:38
…altime-mini

Azure deployments are commonly created against the undated model name,
and the cost-tracking docs say to set base_model to azure/<model> — but
only the dated -2025-10-06 entries existed for these two models (the
openai provider has undated aliases for both). base_model:
azure/gpt-audio-mini therefore resolved to nothing and, depending on the
fallback path, text tokens billed at $0 while audio tokens billed fine.

Mirror the -2025-10-06 entries as undated aliases, exactly like the
undated openai entries mirror their newest dated variant.

Fixes BerriAI#33170

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r dated entries

Review follow-up: COST_FIELDS missed realtime-specific cost keys
(cache_creation_input_audio_token_cost, cache_read_input_token_cost,
input_cost_per_image). Full-entry equality catches drift on every field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e-mini alias

Upstream changed the dated entry's mode from chat to realtime after this
branch was cut; the undated alias must stay a byte-for-byte mirror.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mihidumh
mihidumh force-pushed the fix/azure-audio-mini-price-aliases branch from 483966a to d1853ff Compare August 20, 2026 05:50
@tin-berri

Copy link
Copy Markdown
Contributor

Thanks for the fix @mihidumh! I copied this to a litellm_-prefixed branch so the full CI pipeline runs: #37867 (your commits and authorship preserved, rebased onto latest litellm_internal_staging). All checks are green there with a Greptile 5/5 — a maintainer can review/merge from that PR.

@mateo-berri

Copy link
Copy Markdown
Contributor

Superseded by #37867, which landed your commits with added tests for the root cost map. Thanks for the fix and the report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: price map missing undated azure aliases azure/gpt-audio-mini and azure/gpt-realtime-mini

3 participants