[Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) - #26361
Conversation
Low: No security issuesThis PR adds Azure GPT-5.5 model entries to the static model cost/pricing JSON files and adds corresponding tests. The changes are purely data additions to JSON configuration files with no impact on authentication, authorization, input validation, or any runtime code paths. Status: 0 open Posted by Veria AI · 2026-04-24T22:41:32.572Z |
Greptile SummaryThis PR adds four Azure GPT-5.5 model entries (
Confidence Score: 4/5Safe to merge after adding the missing reasoning-effort flags to the two dated Azure variants. One P1 finding: the dated variants are missing reasoning-effort capability flags that their base entries declare, causing silent None returns from get_model_info. No security issues, no pricing errors, no test coverage regressions. The fix is mechanical (copy the three/four boolean fields into the dated variant objects). model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json — both need reasoning-effort flags added to the dated variant entries.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds azure/gpt-5.5, azure/gpt-5.5-2026-04-23, azure/gpt-5.5-pro, azure/gpt-5.5-pro-2026-04-23 entries; dated variants are missing reasoning-effort capability flags present in the base entries and their OpenAI counterparts |
| litellm/model_prices_and_context_window_backup.json | Mirror of model_prices_and_context_window.json — same four Azure GPT-5.5 entries with the same gap in reasoning-effort flags on dated variants |
| tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py | Adds pricing and reasoning-effort flag tests for azure/gpt-5.5*; reasoning-effort test only covers base models, not dated variants, so the missing flags go undetected |
| tests/test_litellm/test_utils.py | Extends JSON schema validator to allow supports_low_reasoning_effort field — straightforward schema update |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[PR adds 4 Azure model entries] --> B["azure/gpt-5.5 (base)"]
A --> C["azure/gpt-5.5-2026-04-23 (dated)"]
A --> D["azure/gpt-5.5-pro (base)"]
A --> E["azure/gpt-5.5-pro-2026-04-23 (dated)"]
B --> B1["✅ supports_none_reasoning_effort: true"]
B --> B2["✅ supports_xhigh_reasoning_effort: true"]
B --> B3["✅ supports_minimal_reasoning_effort: false"]
C --> C1["❌ supports_none_reasoning_effort: MISSING"]
C --> C2["❌ supports_xhigh_reasoning_effort: MISSING"]
C --> C3["❌ supports_minimal_reasoning_effort: MISSING"]
D --> D1["✅ supports_none_reasoning_effort: false"]
D --> D2["✅ supports_xhigh_reasoning_effort: true"]
D --> D3["✅ supports_minimal_reasoning_effort: false"]
D --> D4["✅ supports_low_reasoning_effort: false"]
E --> E1["❌ supports_none_reasoning_effort: MISSING"]
E --> E2["❌ supports_xhigh_reasoning_effort: MISSING"]
E --> E3["❌ supports_minimal_reasoning_effort: MISSING"]
E --> E4["❌ supports_low_reasoning_effort: MISSING"]
Reviews (2): Last reviewed commit: "test: register supports_low_reasoning_ef..." | Re-trigger Greptile
|
|
||
| openai_entry = litellm.model_cost["gpt-5.5"] | ||
| azure_entry = litellm.model_cost["azure/gpt-5.5"] | ||
|
|
||
| # Provider differs by design; everything else should match. | ||
| assert openai_entry["litellm_provider"] == "openai" | ||
| assert azure_entry["litellm_provider"] == "azure" | ||
|
|
||
| parity_fields = [ | ||
| "input_cost_per_token", | ||
| "output_cost_per_token", | ||
| "cache_read_input_token_cost", | ||
| "max_input_tokens", | ||
| "max_output_tokens", | ||
| "max_tokens", | ||
| "mode", | ||
| "supported_endpoints", | ||
| "supported_modalities", | ||
| "supported_output_modalities", | ||
| "supports_function_calling", | ||
| "supports_native_streaming", | ||
| "supports_parallel_function_calling", | ||
| "supports_pdf_input", |
There was a problem hiding this comment.
supports_web_search missing from parity fields
supports_web_search is not included in parity_fields, so if it is ever added to gpt-5.5 (OpenAI) but not backported to azure/gpt-5.5, this test will silently pass while the entries diverge. All comparable azure/gpt-4.1* entries carry "supports_web_search": false, suggesting it is expected to be declared explicitly.
| openai_entry = litellm.model_cost["gpt-5.5"] | |
| azure_entry = litellm.model_cost["azure/gpt-5.5"] | |
| # Provider differs by design; everything else should match. | |
| assert openai_entry["litellm_provider"] == "openai" | |
| assert azure_entry["litellm_provider"] == "azure" | |
| parity_fields = [ | |
| "input_cost_per_token", | |
| "output_cost_per_token", | |
| "cache_read_input_token_cost", | |
| "max_input_tokens", | |
| "max_output_tokens", | |
| "max_tokens", | |
| "mode", | |
| "supported_endpoints", | |
| "supported_modalities", | |
| "supported_output_modalities", | |
| "supports_function_calling", | |
| "supports_native_streaming", | |
| "supports_parallel_function_calling", | |
| "supports_pdf_input", | |
| parity_fields = [ | |
| "input_cost_per_token", | |
| "output_cost_per_token", | |
| "cache_read_input_token_cost", | |
| "max_input_tokens", | |
| "max_output_tokens", | |
| "max_tokens", | |
| "mode", | |
| "supported_endpoints", | |
| "supported_modalities", | |
| "supported_output_modalities", | |
| "supports_function_calling", | |
| "supports_native_streaming", | |
| "supports_parallel_function_calling", | |
| "supports_pdf_input", | |
| "supports_prompt_caching", | |
| "supports_reasoning", | |
| "supports_response_schema", | |
| "supports_system_messages", | |
| "supports_tool_choice", | |
| "supports_service_tier", | |
| "supports_vision", | |
| "supports_web_search", | |
| "supports_none_reasoning_effort", | |
| "supports_xhigh_reasoning_effort", | |
| "supports_minimal_reasoning_effort", | |
| ] |
Michael-RZ-Berri
left a comment
There was a problem hiding this comment.
Looks good probably add the web search thing + hit endpoint once
…ariants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once #26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags.
azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in #26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order.
0b64bc6 to
72649c7
Compare
| "max_input_tokens": 1050000, | ||
| "max_output_tokens": 128000, | ||
| "max_tokens": 128000, | ||
| "mode": "chat", | ||
| "output_cost_per_token": 3e-05, | ||
| "output_cost_per_token_above_272k_tokens": 4.5e-05, | ||
| "output_cost_per_token_priority": 6e-05, | ||
| "output_cost_per_token_above_272k_tokens_priority": 9e-05, | ||
| "supported_endpoints": [ | ||
| "/v1/chat/completions", | ||
| "/v1/batch", | ||
| "/v1/responses" | ||
| ], | ||
| "supported_modalities": [ | ||
| "text", |
There was a problem hiding this comment.
Dated variant missing reasoning-effort flags
azure/gpt-5.5-2026-04-23 is missing supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort that its base entry (azure/gpt-5.5) declares. The corresponding OpenAI dated snapshot gpt-5.5-2026-04-23 correctly includes all three flags. Any code that calls get_model_info("azure/gpt-5.5-2026-04-23") and reads these fields will silently receive None instead of True/False, misreporting the model's capabilities.
The same gap exists for azure/gpt-5.5-pro-2026-04-23 (missing supports_none_reasoning_effort, supports_xhigh_reasoning_effort, supports_minimal_reasoning_effort, and supports_low_reasoning_effort compared to azure/gpt-5.5-pro).
The flags should be added to both dated variants to match the base entry, mirroring how gpt-5.5-2026-04-23 mirrors gpt-5.5 on the OpenAI side.
…Usage (#26506) * [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (#26449) * feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro Add pricing + capability entries for the new GPT-5.5 family launched by OpenAI on 2026-04-24: - gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M input/output/cached input - gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6 per 1M input/output/cached input Other fees (long-context >272k, flex, batches, priority, cache discounts) follow the same ratios as GPT-5.4, with context window retained at 1.05M input / 128K output. No transformation / classifier code changes are required: OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via numeric version parsing, and model registration is driven from the JSON. The existing responses-API bridge for tools + reasoning_effort (litellm/main.py:970) already covers gpt-5.5-pro. Tests: - GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants - New test_generic_cost_per_token_gpt55_pro cost-calc test - Updated test_generic_cost_per_token_gpt55 for long-context fields * fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort flags that their non-dated counterparts define. Reasoning-effort routing in OpenAIGPT5Config is fully capability-driven from these JSON flags — since an absent flag is treated as False for opt-in levels (xhigh), users pinning to a dated snapshot would silently lose xhigh support and diverge from the base alias on logprobs + flexible temperature handling. Copy the flags onto both dated variants so every dated snapshot inherits the base model's reasoning-effort capability profile. Adds a parametrized regression test that asserts supports_{none,minimal,xhigh}_reasoning_effort parity between each dated variant and its non-dated counterpart, preventing future drift when new snapshots are added. * [Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) (#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once #26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in #26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order. * fix(arize/langfuse_otel): handle Pydantic usage objects without `.get` `_set_usage_outputs` called `usage.get(...)` and `usage.get('output_tokens_details', {}).get('reasoning_tokens')`. These crash with `AttributeError: 'CompletionUsage' object has no attribute 'get'` when `usage` (or the nested token-details object) is a raw OpenAI Pydantic model rather than a dict / litellm `Usage` wrapper. Reproduces on the langfuse_otel + arize Responses API logging paths. Fixes #13672. Changes: - Add `_safe_get(obj, key, default)` that prefers dict-style `.get` when available and otherwise falls back to `getattr`. Works uniformly for dicts, litellm's `Usage`, and plain Pydantic models like `openai.types.completion_usage.CompletionUsage` / `CompletionTokensDetails` / `OutputTokensDetails`. - Use `_safe_get` for total / completion / prompt / output tokens. - Look for reasoning tokens in `completion_tokens_details` (Chat Completions API) before falling back to `output_tokens_details` (Responses API). Previously reasoning tokens from the Chat Completions API were silently dropped. Tests: - `test_set_usage_outputs_pydantic_completion_usage` — covers the chat completions path with raw `CompletionUsage` + `CompletionTokensDetails`. - `test_set_usage_outputs_pydantic_response_api_usage` — covers the Responses API path with a Pydantic usage object lacking `.get`. Both tests fail on main before this commit and pass after. --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: alvinttang <alvin@pm.me> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
…s) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in BerriAI#26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order.
* Use auth key name if there are no app id in in headers or in extra_data * use key alias instead of key name * Fix * last priority key alias * Fix * Add tests * [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (#26449) * feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro Add pricing + capability entries for the new GPT-5.5 family launched by OpenAI on 2026-04-24: - gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M input/output/cached input - gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6 per 1M input/output/cached input Other fees (long-context >272k, flex, batches, priority, cache discounts) follow the same ratios as GPT-5.4, with context window retained at 1.05M input / 128K output. No transformation / classifier code changes are required: OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via numeric version parsing, and model registration is driven from the JSON. The existing responses-API bridge for tools + reasoning_effort (litellm/main.py:970) already covers gpt-5.5-pro. Tests: - GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants - New test_generic_cost_per_token_gpt55_pro cost-calc test - Updated test_generic_cost_per_token_gpt55 for long-context fields * fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort flags that their non-dated counterparts define. Reasoning-effort routing in OpenAIGPT5Config is fully capability-driven from these JSON flags — since an absent flag is treated as False for opt-in levels (xhigh), users pinning to a dated snapshot would silently lose xhigh support and diverge from the base alias on logprobs + flexible temperature handling. Copy the flags onto both dated variants so every dated snapshot inherits the base model's reasoning-effort capability profile. Adds a parametrized regression test that asserts supports_{none,minimal,xhigh}_reasoning_effort parity between each dated variant and its non-dated counterpart, preventing future drift when new snapshots are added. * [Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) (#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once #26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in #26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order. * Use sanitize deep copy style to replace deepcopy usage * Added test checking error is not happening anymore * Added warning log when json copy failed * Reduce to one change * Fix spaces --------- Co-authored-by: Ido Lavi <ido@noma.security> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: TomAlon <tom@noma.security>
…#26605) * Use auth key name if there are no app id in in headers or in extra_data * use key alias instead of key name * Fix * last priority key alias * Fix * Add tests * [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (BerriAI#26449) * feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro Add pricing + capability entries for the new GPT-5.5 family launched by OpenAI on 2026-04-24: - gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M input/output/cached input - gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6 per 1M input/output/cached input Other fees (long-context >272k, flex, batches, priority, cache discounts) follow the same ratios as GPT-5.4, with context window retained at 1.05M input / 128K output. No transformation / classifier code changes are required: OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via numeric version parsing, and model registration is driven from the JSON. The existing responses-API bridge for tools + reasoning_effort (litellm/main.py:970) already covers gpt-5.5-pro. Tests: - GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants - New test_generic_cost_per_token_gpt55_pro cost-calc test - Updated test_generic_cost_per_token_gpt55 for long-context fields * fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort flags that their non-dated counterparts define. Reasoning-effort routing in OpenAIGPT5Config is fully capability-driven from these JSON flags — since an absent flag is treated as False for opt-in levels (xhigh), users pinning to a dated snapshot would silently lose xhigh support and diverge from the base alias on logprobs + flexible temperature handling. Copy the flags onto both dated variants so every dated snapshot inherits the base model's reasoning-effort capability profile. Adds a parametrized regression test that asserts supports_{none,minimal,xhigh}_reasoning_effort parity between each dated variant and its non-dated counterpart, preventing future drift when new snapshots are added. * [Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in BerriAI#26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order. * Use sanitize deep copy style to replace deepcopy usage * Added test checking error is not happening anymore * Added warning log when json copy failed * Reduce to one change * Fix spaces --------- Co-authored-by: Ido Lavi <ido@noma.security> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: TomAlon <tom@noma.security>
…Usage (BerriAI#26506) * [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (BerriAI#26449) * feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro Add pricing + capability entries for the new GPT-5.5 family launched by OpenAI on 2026-04-24: - gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M input/output/cached input - gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6 per 1M input/output/cached input Other fees (long-context >272k, flex, batches, priority, cache discounts) follow the same ratios as GPT-5.4, with context window retained at 1.05M input / 128K output. No transformation / classifier code changes are required: OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via numeric version parsing, and model registration is driven from the JSON. The existing responses-API bridge for tools + reasoning_effort (litellm/main.py:970) already covers gpt-5.5-pro. Tests: - GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants - New test_generic_cost_per_token_gpt55_pro cost-calc test - Updated test_generic_cost_per_token_gpt55 for long-context fields * fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort flags that their non-dated counterparts define. Reasoning-effort routing in OpenAIGPT5Config is fully capability-driven from these JSON flags — since an absent flag is treated as False for opt-in levels (xhigh), users pinning to a dated snapshot would silently lose xhigh support and diverge from the base alias on logprobs + flexible temperature handling. Copy the flags onto both dated variants so every dated snapshot inherits the base model's reasoning-effort capability profile. Adds a parametrized regression test that asserts supports_{none,minimal,xhigh}_reasoning_effort parity between each dated variant and its non-dated counterpart, preventing future drift when new snapshots are added. * [Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order. * fix(arize/langfuse_otel): handle Pydantic usage objects without `.get` `_set_usage_outputs` called `usage.get(...)` and `usage.get('output_tokens_details', {}).get('reasoning_tokens')`. These crash with `AttributeError: 'CompletionUsage' object has no attribute 'get'` when `usage` (or the nested token-details object) is a raw OpenAI Pydantic model rather than a dict / litellm `Usage` wrapper. Reproduces on the langfuse_otel + arize Responses API logging paths. Fixes BerriAI#13672. Changes: - Add `_safe_get(obj, key, default)` that prefers dict-style `.get` when available and otherwise falls back to `getattr`. Works uniformly for dicts, litellm's `Usage`, and plain Pydantic models like `openai.types.completion_usage.CompletionUsage` / `CompletionTokensDetails` / `OutputTokensDetails`. - Use `_safe_get` for total / completion / prompt / output tokens. - Look for reasoning tokens in `completion_tokens_details` (Chat Completions API) before falling back to `output_tokens_details` (Responses API). Previously reasoning tokens from the Chat Completions API were silently dropped. Tests: - `test_set_usage_outputs_pydantic_completion_usage` — covers the chat completions path with raw `CompletionUsage` + `CompletionTokensDetails`. - `test_set_usage_outputs_pydantic_response_api_usage` — covers the Responses API path with a Pydantic usage object lacking `.get`. Both tests fail on main before this commit and pass after. --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: alvinttang <alvin@pm.me> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
…s) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in BerriAI#26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order.
…#26605) * Use auth key name if there are no app id in in headers or in extra_data * use key alias instead of key name * Fix * last priority key alias * Fix * Add tests * [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (BerriAI#26449) * feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro Add pricing + capability entries for the new GPT-5.5 family launched by OpenAI on 2026-04-24: - gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M input/output/cached input - gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6 per 1M input/output/cached input Other fees (long-context >272k, flex, batches, priority, cache discounts) follow the same ratios as GPT-5.4, with context window retained at 1.05M input / 128K output. No transformation / classifier code changes are required: OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via numeric version parsing, and model registration is driven from the JSON. The existing responses-API bridge for tools + reasoning_effort (litellm/main.py:970) already covers gpt-5.5-pro. Tests: - GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants - New test_generic_cost_per_token_gpt55_pro cost-calc test - Updated test_generic_cost_per_token_gpt55 for long-context fields * fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort flags that their non-dated counterparts define. Reasoning-effort routing in OpenAIGPT5Config is fully capability-driven from these JSON flags — since an absent flag is treated as False for opt-in levels (xhigh), users pinning to a dated snapshot would silently lose xhigh support and diverge from the base alias on logprobs + flexible temperature handling. Copy the flags onto both dated variants so every dated snapshot inherits the base model's reasoning-effort capability profile. Adds a parametrized regression test that asserts supports_{none,minimal,xhigh}_reasoning_effort parity between each dated variant and its non-dated counterpart, preventing future drift when new snapshots are added. * [Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in BerriAI#26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order. * Use sanitize deep copy style to replace deepcopy usage * Added test checking error is not happening anymore * Added warning log when json copy failed * Reduce to one change * Fix spaces --------- Co-authored-by: Ido Lavi <ido@noma.security> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: TomAlon <tom@noma.security>
…s) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in BerriAI#26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order.
…Usage (BerriAI#26506) * [Feat] Day-0 support for GPT-5.5 and GPT-5.5 Pro (BerriAI#26449) * feat(openai): day-0 support for GPT-5.5 and GPT-5.5 Pro Add pricing + capability entries for the new GPT-5.5 family launched by OpenAI on 2026-04-24: - gpt-5.5 / gpt-5.5-2026-04-23 (chat): $5/$30/$0.50 per 1M input/output/cached input - gpt-5.5-pro / gpt-5.5-pro-2026-04-23 (responses-only): $60/$360/$6 per 1M input/output/cached input Other fees (long-context >272k, flex, batches, priority, cache discounts) follow the same ratios as GPT-5.4, with context window retained at 1.05M input / 128K output. No transformation / classifier code changes are required: OpenAIGPT5Config.is_model_gpt_5_4_plus_model() already matches 5.5+ via numeric version parsing, and model registration is driven from the JSON. The existing responses-API bridge for tools + reasoning_effort (litellm/main.py:970) already covers gpt-5.5-pro. Tests: - GPT5_MODELS regression list now covers gpt-5.5-pro and dated variants - New test_generic_cost_per_token_gpt55_pro cost-calc test - Updated test_generic_cost_per_token_gpt55 for long-context fields * fix(openai): mirror reasoning_effort flags onto gpt-5.5 dated variants gpt-5.5-2026-04-23 and gpt-5.5-pro-2026-04-23 were missing the supports_none_reasoning_effort, supports_xhigh_reasoning_effort, and supports_minimal_reasoning_effort flags that their non-dated counterparts define. Reasoning-effort routing in OpenAIGPT5Config is fully capability-driven from these JSON flags — since an absent flag is treated as False for opt-in levels (xhigh), users pinning to a dated snapshot would silently lose xhigh support and diverge from the base alias on logprobs + flexible temperature handling. Copy the flags onto both dated variants so every dated snapshot inherits the base model's reasoning-effort capability profile. Adds a parametrized regression test that asserts supports_{none,minimal,xhigh}_reasoning_effort parity between each dated variant and its non-dated counterpart, preventing future drift when new snapshots are added. * [Feat] Add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) (BerriAI#26361) * feat(azure): add azure/gpt-5.5 + azure/gpt-5.5-pro entries (+ dated variants) Azure variants of OpenAI's GPT-5.5 family. Microsoft has not yet shipped GPT-5.5 on Azure OpenAI (latest GA on the Foundry models page is GPT-5.4 as of 2026-04-24), but adding the entries day-0 mirrors the established precedent for azure/gpt-5.4* (which were in the cost map before the Azure rollout) so cost tracking and capability flags work the moment customers deploy. Schema follows the existing azure/gpt-5.4* shape: - Same base/long-context pricing as openai/gpt-5.5*: $5/$30 chat, $60/$360 pro per 1M, with priority tier 2x base - Azure variants drop the flex/batches keys (Azure has no flex tier) but keep priority pricing, matching gpt-5.4* precedent - mode=chat for the thinking model, mode=responses for pro reasoning_effort capability flags mirror the OpenAI variants exactly since Azure proxies the same API contract: minimal rejection on both chat and pro, low/none rejection on pro. Once BerriAI#26456 (which sets supports_low_reasoning_effort + minimal=false on openai/gpt-5.5*) lands, OpenAI and Azure flag profiles align. Tests pin entry presence + pricing for all four Azure variants and verify the live-API-derived reasoning_effort flags. * test: register supports_low_reasoning_effort in cost-map JSON schema azure/gpt-5.5-pro and azure/gpt-5.5-pro-2026-04-23 added in this branch carry supports_low_reasoning_effort=false. The strict 'additionalProperties: false' schema in test_aaamodel_prices_and_context_window_json_is_valid rejected the new key. Register it alongside the other supports_*_reasoning_effort entries. Note: the runtime side of this flag (code that reads it) lands in BerriAI#26456. Until that PR merges the flag is inert for both Azure and OpenAI pro entries, but having the schema accept it lets cost-map tests pass on either merge order. * fix(arize/langfuse_otel): handle Pydantic usage objects without `.get` `_set_usage_outputs` called `usage.get(...)` and `usage.get('output_tokens_details', {}).get('reasoning_tokens')`. These crash with `AttributeError: 'CompletionUsage' object has no attribute 'get'` when `usage` (or the nested token-details object) is a raw OpenAI Pydantic model rather than a dict / litellm `Usage` wrapper. Reproduces on the langfuse_otel + arize Responses API logging paths. Fixes BerriAI#13672. Changes: - Add `_safe_get(obj, key, default)` that prefers dict-style `.get` when available and otherwise falls back to `getattr`. Works uniformly for dicts, litellm's `Usage`, and plain Pydantic models like `openai.types.completion_usage.CompletionUsage` / `CompletionTokensDetails` / `OutputTokensDetails`. - Use `_safe_get` for total / completion / prompt / output tokens. - Look for reasoning tokens in `completion_tokens_details` (Chat Completions API) before falling back to `output_tokens_details` (Responses API). Previously reasoning tokens from the Chat Completions API were silently dropped. Tests: - `test_set_usage_outputs_pydantic_completion_usage` — covers the chat completions path with raw `CompletionUsage` + `CompletionTokensDetails`. - `test_set_usage_outputs_pydantic_response_api_usage` — covers the Responses API path with a Pydantic usage object lacking `.get`. Both tests fail on main before this commit and pass after. --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: alvinttang <alvin@pm.me> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
Relevant issues
Day-0 Azure entries for OpenAI's GPT-5.5 family. Microsoft hasn't shipped GPT-5.5 on Azure OpenAI yet (latest GA on the Foundry models page is the GPT-5.4 series as of 2026-04-24), but the existing precedent —
azure/gpt-5.4*was in the cost map before its Azure rollout — means we should land these entries now so cost tracking + capability flags Just Work the moment customers deploy.This PR was force-pushed to expand scope from a single
azure/gpt-5.5entry to the full set of four entries (chat + pro, base + dated variants), match the long-context pricing structure ofgpt-5.4, and add tests. The previous narrower revision (#26458) is now superseded and closed.Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🆕 New Feature
Changes
Adds four entries to both
model_prices_and_context_window.jsonandlitellm/model_prices_and_context_window_backup.json:azure/gpt-5.5azure/gpt-5.5-2026-04-23azure/gpt-5.5-proazure/gpt-5.5-pro-2026-04-23Pricing per-1M-tokens. Schema follows the existing
azure/gpt-5.4*shape:openai/gpt-5.5*counterpartsmode: chatfor thinking,mode: responsesfor proreasoning_effortcapability flags mirror the OpenAI variants since Azure proxies the same API contract —minimalrejected on both chat and pro,lowandnonerejected on pro.Dependency note
supports_low_reasoning_effortis set onazure/gpt-5.5-pro*here, but the schema entry + transformation logic that reads the flag is introduced in #26456. Until #26456 lands, the flag on the Azure entries is inert (no effect —lowcontinues to pass through). After #26456 merges, both OpenAI and Azure pro variants will rejectlowconsistently.The schema-test fix (
supports_low_reasoning_effortregistered intests/test_litellm/test_utils.py) is included here sotest_aaamodel_prices_and_context_window_json_is_validpasses regardless of merge order.Tests
tests/test_litellm/litellm_core_utils/llm_cost_calc/test_llm_cost_calc_utils.py:test_azure_gpt55_entries_present_with_correct_pricing— parametrized over all four entries; verifies provider, mode, base + long-context pricing, context windowtest_azure_gpt55_reasoning_effort_flags_match_live_openai_api— pins the live-API-derivedsupports_{none, minimal, xhigh}_reasoning_effortprofiletests/test_litellm/test_utils.py:supports_low_reasoning_effortalongside the existing reasoning-effort flags