fix(azure): rename max_tokens to max_completion_tokens for gpt-5-chat deployments

[yassin-berriai]

TLDR

Problem this solves:

• Azure gpt-5-chat deployments 400 on every max_tokens request
• /health reports those deployments permanently unhealthy
• The rename that fixes it skipped the whole gpt-5-chat family

How it solves it:

• New requires_max_completion_tokens predicate covers every gpt-5 name
• Only the max_tokens rename keys off it, nothing else
• Reasoning params keep keying off is_model_gpt_5_model, so [Bug]: OpenAI GPT-5 Chat model does not support "temperature" parameter #13781 stays fixed

User Flow

Before: a developer whose Azure deployment is named gpt-5-chat gets a 400 on every request that sets max_tokens, and the deployment sits red in /health forever

1. They send POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5-chat", "messages": [...], "max_tokens": 5}
2. HTTP 400 comes back: Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
3. Their admin opens GET https://litellm-domain/health?model=gpt-5-chat and sees the deployment under unhealthy_endpoints with that same message, healthy_count 0
4. The synthetic monitor watching that endpoint pages on-call, and keeps paging

After: the same request succeeds and the same deployment reports healthy

1. They send the same POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5-chat", "messages": [...], "max_tokens": 5}
2. HTTP 200 comes back with a normal chat completion
3. Their admin opens GET https://litellm-domain/health?model=gpt-5-chat and sees the deployment under healthy_endpoints, unhealthy_count 0
4. The monitor goes quiet

Relevant issues

No upstream issue covers this defect. #13781 is the one this change is careful not to reopen: it reported gpt-5-chat-latest wrongly rejecting temperature, which is why the family was pulled off the GPT-5 reasoning path in the first place. #24779, closed as not planned, claims the same Azure error string for azure/gpt-4o

Linear ticket

Resolves LIT-5549

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

There are no Azure credentials on the machine this was captured on, so the run below points a live proxy at a local HTTP server standing in for Azure. That server records the exact JSON body litellm puts on the wire and returns Azure's own 400 whenever max_tokens is present. The wire body is measured, the rejection is simulated, and the claim that Azure really rejects max_tokens for this family rests on a customer's production error plus Azure/azure-sdk-for-net#51844, which names gpt-5-chat among the deployments returning that exact string

Both runs use the same config, four Azure deployments (gpt-5-chat, gpt-5, o3-mini, gpt-4o) all pointed at the stand-in on 127.0.0.1:45550, proxy on 127.0.0.1:45549

Before, at e0f388c (this branch's base):

$ curl -s http://127.0.0.1:45549/health/liveliness
"I'm alive!"

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:45549/v1/chat/completions \
    -H 'Authorization: Bearer sk-lit5549' -H 'Content-Type: application/json' \
    -d '{"model":"gpt-5-chat","messages":[{"role":"user","content":"hi"}],"max_tokens":5}'
{"error":{"message":"litellm.BadRequestError: AzureException BadRequestError - Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.. Received Model Group=gpt-5-chat\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":"max_tokens","code":"400"}}
HTTP 400

body litellm put on the wire to Azure:
{"path": "/openai/deployments/gpt-5-chat/chat/completions?api-version=2024-12-01-preview", "body": {"messages": [{"role": "user", "content": "hi"}], "model": "gpt-5-chat", "max_tokens": 5, "stream": false}}

$ curl -s "http://127.0.0.1:45549/health?model=gpt-5-chat" -H 'Authorization: Bearer sk-lit5549'
{"healthy_endpoints":[],"unhealthy_endpoints":[{"api_base":"http://127.0.0.1:45550","model":"azure/gpt-5-chat","max_tokens":5,"error":"litellm.BadRequestError: AzureException BadRequestError - Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead. ...","exception_status":400}],"healthy_count":0,"unhealthy_count":1}

body litellm put on the wire to Azure:
{"path": "/openai/deployments/gpt-5-chat/chat/completions?api-version=2024-12-01-preview", "body": {"messages": [{"role": "user", "content": "What's 1 + 1?"}], "model": "gpt-5-chat", "max_tokens": 5}}

controls, same request shape:
  gpt-5   -> HTTP 200  wire: {"model": "gpt-5", "max_completion_tokens": 5, ...}
  o3-mini -> HTTP 200  wire: {"model": "o3-mini", "max_completion_tokens": 5, ...}
  gpt-4o  -> HTTP 400  wire: {"model": "gpt-4o", "max_tokens": 5, ...}

After, at 1e877f2, since rebased onto e1f3d6e, then 29d8bed, and now 5b0ea66 (this PR). The capture is still the code at the current head: litellm/llms/azure/chat/gpt_transformation.py carries a byte-identical contribution at every one of those commits, and the latest rebase only merged a neighbouring test from #34462 into the test file:

$ curl -s http://127.0.0.1:45549/health/liveliness
"I'm alive!"

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:45549/v1/chat/completions \
    -H 'Authorization: Bearer sk-lit5549' -H 'Content-Type: application/json' \
    -d '{"model":"gpt-5-chat","messages":[{"role":"user","content":"hi"}],"max_tokens":5}'
{"id":"chatcmpl-lit5549","created":1,"model":"gpt-5-chat","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"ok","role":"assistant"}}],"usage":{"completion_tokens":1,"prompt_tokens":1,"total_tokens":2}}
HTTP 200

body litellm put on the wire to Azure:
{"path": "/openai/deployments/gpt-5-chat/chat/completions?api-version=2024-12-01-preview", "body": {"messages": [{"role": "user", "content": "hi"}], "model": "gpt-5-chat", "max_completion_tokens": 5, "stream": false}}

$ curl -s "http://127.0.0.1:45549/health?model=gpt-5-chat" -H 'Authorization: Bearer sk-lit5549'
{"healthy_endpoints":[{"api_base":"http://127.0.0.1:45550","model":"azure/gpt-5-chat","max_tokens":5,"model_id":"2f92a5a8f85a34351a0b0e03034c389e3cab56c3101e47e6cbf1cd0f2c3ac4f5"}],"unhealthy_endpoints":[],"healthy_count":1,"unhealthy_count":0}

body litellm put on the wire to Azure:
{"path": "/openai/deployments/gpt-5-chat/chat/completions?api-version=2024-12-01-preview", "body": {"messages": [{"role": "user", "content": "What's 1 + 1?"}], "model": "gpt-5-chat", "max_completion_tokens": 5}}

controls, same request shape:
  gpt-5   -> HTTP 200  wire: {"model": "gpt-5", "max_completion_tokens": 5, ...}
  o3-mini -> HTTP 200  wire: {"model": "o3-mini", "max_completion_tokens": 5, ...}
  gpt-4o  -> HTTP 400  wire: {"model": "gpt-4o", "max_tokens": 5, ...}

The gpt-5 and o3-mini rows are the load-bearing part of the before run: the same harness prints max_completion_tokens whenever litellm emits it, so max_tokens on the gpt-5-chat row means the code picked that key rather than the fixture never carrying the alternative. gpt-4o is unchanged on purpose and its 400 only shows that the stand-in rejects max_tokens from everyone

Type

🐛 Bug Fix

Caveats (if any)

• azure/gpt-4o still sends max_tokens; [Bug]: Azure gpt-4o rejects max_tokens — needs translation to max_completion_tokens like o-series/gpt-5 #24779 is out of scope
• Widening the predicate there needs checking against current Azure behaviour
• The Azure 400 is simulated locally; the request body is real

Review notes

Greptile scored this 4/5 on 1e877f2 with two findings it framed as non-blocking follow-ups. I looked into both and am deliberately not changing the code for either

On the predicate being hardcoded rather than read from the model cost map: no such flag exists. I enumerated every support* key present anywhere in model_prices_and_context_window.json, 40 of them across roughly three thousand entries, and none expresses "this deployment rejects the legacy max_tokens key". The nearest neighbours are supports_reasoning, supports_sampling_params and the supports_*_reasoning_effort family, which all answer the reasoning question instead. supports_reasoning is the worst of them to borrow here: azure/gpt-5-chat carries supports_reasoning: true today, so keying the rename off it would rebuild the exact conflation of two independent questions that caused this bug. The map also already spends the name max_tokens on the output-token ceiling, so a capability flag named around it would read as a collision. Model naming is the only source of truth for this capability at present, and all three pre-existing renames in the tree agree: llms/openai/chat/o_series_transformation.py:99 keys off is_o_series_model, and llms/openai/chat/gpt_5_transformation.py:200 and :256 key off is_model_gpt_5_search_model and is_model_gpt_5_model. Introducing a flag would mean populating it correctly across the whole gpt-5 and o-series families before it could be trusted, which is a far larger and riskier change than this defect justifies

On the new branch introducing another parameter mutation: optional_params is a local accumulator here, not caller-owned state. It is created as a fresh {} inside pre_process_optional_params (utils.py:3855), returned into get_optional_params, and threaded through the provider map_openai_params chain with the return value reassigned at every hop, which is the contract BaseConfig.map_openai_params declares. Filling it is what the function is for, and every other branch of the same loop fills it the same way. The new branch is actually the least mutating of the four renames in the tree, since the three cited above each call non_default_params.pop("max_tokens") and reach into the caller's dict, while this one writes only into the accumulator and pops nothing

The automated security review raised a Medium, that a request carrying both max_tokens=1 and max_completion_tokens=10000 reserves one output token while the provider receives ten thousand, and asked for the conflicting fields to be rejected or normalised before the pre-call rate-limit hooks run. The under-reservation is real and worth fixing. It is being fixed in #37001, in the limiter, and deliberately not here. Three measurements, each of which independently rules out doing it in this file

Ordering. The reservation is taken by _estimate_tokens_for_request at proxy/hooks/parallel_request_limiter_v3.py:607, reading data.get("max_tokens") or data.get("max_completion_tokens") straight off the raw request body inside async_pre_call_hook, which fires at proxy/common_request_processing.py:1471 before routing and long before any provider transformation. Driven directly, that body reserves 2 tokens where the same body carrying only max_completion_tokens reserves 10001. Nothing in map_openai_params can move those numbers, because the reservation is already written by the time the transformation runs. Rejecting there would convert a bypass into a 400 after the fact rather than reserve correctly, which is not the same thing

Reach. This is not introduced here. At this branch's merge-base, the same request already drops max_tokens: 1 and emits max_completion_tokens: 10000 for azure/gpt-5, azure/o3-mini, openai/gpt-5 and openai/o3-mini. On a live proxy at base, azure/gpt-5 with both fields returns HTTP 200 with ten thousand on the wire, which is exactly what azure/gpt-5-chat does after this change. Closing it inside AzureOpenAIConfig would shut one of five doors while implying the corridor was secured

Blast radius. Rejecting the combination would break a working configuration. LiteLLM_Params sets extra="allow" and the router merges deployment params underneath client kwargs, so an operator's deployment-level max_tokens default and a client's max_completion_tokens arrive together by design. Run end to end through the router, a deployment carrying max_tokens: 4096 with a client sending max_completion_tokens: 100 returns HTTP 200 and puts {"max_completion_tokens": 100} on the wire, the client value correctly winning. A conflict error would fail every operator in that shape

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR updates Azure GPT-5 chat parameter translation so legacy max_tokens inputs are emitted as max_completion_tokens without moving chat variants onto the reasoning-model path.

• Adds the Azure GPT-5-family token-key translation.
• Adds regression coverage for GPT-5 chat variants, reasoning parameter behavior, and unaffected control models.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/azure/chat/gpt_transformation.py	Adds an isolated Azure request-parameter translation for GPT-5 chat deployments while preserving existing reasoning-model classification.
tests/test_litellm/llms/azure/chat/test_azure_chat_gpt_transformation.py	Adds focused regression tests covering token-key emission and GPT-5 chat versus reasoning behavior.

_{Reviews (3): Last reviewed commit: "fix(azure): rename max_tokens to max_com..." | Re-trigger Greptile}

[greptile-apps]

Hardcoded model capability predicate

The new predicate infers token-parameter support exclusively from gpt-5 and gpt5_series substrings instead of model metadata. This requires LiteLLM releases for future capability changes and lets naming aliases select the wrong outbound parameter; represent the capability in model metadata and access it through get_model_info.

Rule Used: What: Do not hardcode model-specific flags in the ... (source)

Knowledge Base Used: LLM Provider Adapters

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Parameter mapping mutates input

Calling setdefault directly mutates the caller-provided optional_params mapping, adding another in-place state change that makes ownership and parameter precedence harder to reason about. Construct and return the updated mapping without mutating the function parameter.

Context Used: CLAUDE.md (source)

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai adjudicated both findings in the PR description under Review notes: no such model-map flag exists, and optional_params is a local accumulator. Head 1e877f2

[greptile-apps]

Both adjudications hold up.

Model-map flag: Correct that no suitable flag exists. supports_reasoning would be the wrong key to borrow — azure/gpt-5-chat carries it today, so keying the rename off it would recreate the conflation this PR is fixing. Name-based predicates are the existing convention across all three prior renames in the tree, and the new predicate ("gpt-5" in model or "gpt5_series" in model) is consistent with that pattern. No objection.

optional_params mutation: setdefault is actually the most conservative of the four renames — it only writes to the accumulator and leaves non_default_params untouched, while the three existing renames all call non_default_params.pop("max_tokens"). The concern doesn't apply.

The test suite is thorough: it covers the new family (gpt-5-chat, gpt-5-chat-latest, gpt-5-chat-2025-08-07), the positive control for the reasoning path (gpt-5), the regression guard for #13781 (gpt-5-chat* stays off the reasoning path), and an unchanged negative (gpt-4o). Both original findings are resolved by the evidence in the PR description and the diff. The fix looks correct.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit5549_azure_gpt5_chat_max_completion_tokens (5b0ea66) with litellm_internal_staging (973329e)}

[yassin-berriai]

@greptileai rebased onto current staging, no code change since the last review. Please re-review head 29d8bed. Rebuttal stands in the PR description under Review notes

[yassin-berriai]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 29d8bed. Configure here.}

[yassin-berriai]

@veria-ai answered in the PR description under Review notes: the reservation predates this change and is fixed in #37001. Head 29d8bed

[yassin-berriai]

@greptileai please review the current head 5b0ea66. Only change since your 5/5: rebase onto staging, resolving a test-file conflict with #34462

[yassin-berriai]

Rebased onto staging to clear the conflict. The approval at 29d8bed still holds: identical production diff, test file only gained #34462 neighbour