fix(proxy): use max_completion_tokens for gpt-chat-latest health

[AleksandrLiadov]

What

Fix LiteLLM proxy health checks for Azure gpt-chat-latest deployments by sending max_completion_tokens instead of max_tokens.

Azure gpt-chat-latest rejects health probes containing max_tokens with:

Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

Deployment names can include suffixes such as azure/gpt-chat-latest-gs, so the health check now detects gpt-chat-latest in either:

• litellm_params.model
• model_info.base_model

and sends the bounded health token value as max_completion_tokens for those probes.

Why

The model supports /chat/completions, but the health check currently marks it unhealthy because it uses the wrong token-limit parameter. This is similar to the existing Azure GPT-5/o-series max token handling issue, but gpt-chat-latest is an alias and does not necessarily trigger GPT-5-specific model-name routing.

Related context:

• [Bug]: Azure gpt-4o rejects max_tokens — needs translation to max_completion_tokens like o-series/gpt-5 #24779 - Azure models rejecting max_tokens and requiring max_completion_tokens
• Add Azure gpt-chat-latest model metadata #27420 - Azure gpt-chat-latest model metadata

Tests

• uv run pytest tests/test_litellm/proxy/test_health_check_max_tokens.py::test_azure_gpt_chat_latest_health_check_uses_max_completion_tokens -q
• uv run pytest tests/test_litellm/proxy/test_health_check_max_tokens.py -q
• uv run ruff check litellm/proxy/health_check.py tests/test_litellm/proxy/test_health_check_max_tokens.py

Note: local uv emitted a warning parsing exclude-newer = "3 days", but the tests/lint above passed.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Congrats! CodSpeed is installed 🎉

🆕 16 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_cost_per_token_openai: 557.9 µs
• test_get_model_info_with_provider: 79.8 µs
• test_token_counter_long_content: 1.7 ms
• test_get_llm_provider_anthropic: 143.9 µs
• test_get_model_info_openai: 81 µs
• test_cost_per_token_anthropic: 557.6 µs
• test_token_counter_with_tools: 414.7 µs
• test_token_counter_multi_turn: 568.7 µs
• test_get_llm_provider_azure: 144 µs
• test_token_counter_simple_message: 234.7 µs
• test_token_counter_raw_text: 182.9 µs
• test_get_llm_provider_openai: 138.7 µs
• test_get_model_cost_key_exact_match: 79.6 µs
• test_get_llm_provider_with_prefix: 136.5 µs
• test_get_model_info_anthropic: 79.8 µs
• test_get_model_cost_key_case_insensitive: 81.7 µs

---

[codecov]

Codecov Report

❌ Patch coverage is 89.47368% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/health_check.py	89.18%	4 Missing ⚠️
litellm/llms/azure/chat/gpt_transformation.py	90.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes Azure gpt-chat-latest health checks by adding the model to the cost map with a map_max_tokens_to_max_completion_tokens: true flag, then reading that flag generically in both the health-check path (via a new _resolve_health_check_model_info resolver that handles suffixed deployment names like azure/gpt-chat-latest-gs through base_model lookup) and the normal param-mapping path in AzureOpenAIConfig.map_openai_params.

• Model cost map: azure/gpt-chat-latest added to both JSON files with the new map_max_tokens_to_max_completion_tokens: true field.
• Health check path: _resolve_health_check_model_info merges cost-map metadata with deployment model_info for suffixed deployments; _update_litellm_params_for_health_check emits max_completion_tokens when the flag is set.
• Regular call path: _should_map_max_tokens_to_max_completion_tokens translates max_tokens for the canonical azure/gpt-chat-latest name; suffixed deployment names are not covered here.

Confidence Score: 5/5

The change is narrowly scoped to health-check parameter selection and Azure param mapping; the model-map flag is opt-in and can be overridden at the deployment level.

The core fix is correct: health checks for both canonical and suffixed gpt-chat-latest deployments now emit max_completion_tokens, and regular calls for the canonical name benefit too. The reviewer-noted lint CI failure is worth resolving before merge, but the logic is sound and well-tested with mock-only unit tests.

The lint CI check is still failing per the reviewer — worth verifying the full pipeline, as black formatting and ruff on gpt_transformation.py were not included in the author's local checks.

Important Files Changed

Filename	Overview
litellm/proxy/health_check.py	Adds _resolve_health_check_model_info to merge model-cost metadata with deployment-level model_info; minor issue: GetModelCostMap.load_local_model_cost_map() called inside loop, reading disk once per missed candidate.
litellm/llms/azure/chat/gpt_transformation.py	Adds _should_map_max_tokens_to_max_completion_tokens; works for canonical azure/gpt-chat-latest but suffixed deployment names are not resolved here.
model_prices_and_context_window.json	Adds azure/gpt-chat-latest metadata with map_max_tokens_to_max_completion_tokens: true.
litellm/model_prices_and_context_window_backup.json	Mirrors the azure/gpt-chat-latest entry in the in-package backup JSON.
tests/test_litellm/proxy/test_health_check_max_tokens.py	Adds two mock-only tests covering the metadata path and deployment-level override.
tests/test_litellm/llms/azure/chat/test_azure_chat_gpt_transformation.py	Adds two unit tests for map_openai_params translation for flagged and unflagged models.
tests/test_litellm/test_utils.py	Adds map_max_tokens_to_max_completion_tokens to the model-price JSON schema validator.

_{Reviews (3): Last reviewed commit: "fix(azure): health check + chat use max_..." | Re-trigger Greptile}

[Sameerlite]

Thanks for the contribution! Kicking off a Greptile code review on this one.

@greptileai

[Sameerlite]

Thanks , this correctly fixes health checks for gpt-chat-latest suffix deployments via model-map metadata.

One gap: this only affects the health probe. Normal /chat/completions calls with max_tokens will still 400 on Azure, because gpt-chat-latest doesn't route through GPT-5/o-series config and AzureOpenAIConfig doesn't translate the param.

Could we add a general model-map flag (e.g. map_max_tokens_to_max_completion_tokens) and use it in AzureOpenAIConfig.map_openai_params as well as health checks, or

Also: unrelated test_openapi_compliance.py change - intentional or accidental?

Also: Please change the base branch to litellm_internal_staging

[AleksandrLiadov]

Thanks @Sameerlite, addressed all three points:

1. General param translation (not just health checks): Replaced the health-check-only flag with a general model-map flag map_max_tokens_to_max_completion_tokens. It's now consumed in AzureOpenAIConfig.map_openai_params, so normal /chat/completions calls with max_tokens are translated to max_completion_tokens for flagged models (e.g. azure/gpt-chat-latest) instead of 400ing. The health-check probe uses the same flag. The flag is read from the raw model-cost map (not get_model_info, which strips unknown metadata keys), with a local-map fallback so it works before the remote map ships the entry. Added map_openai_params tests covering both flagged and unflagged models.

2. test_openapi_compliance.py: Accidental — reverted.

3. Base branch: Retargeted to litellm_internal_staging.

[Sameerlite]

Thanks for the contribution! The lint CI check is still failing — could you take a look? Also triggering a fresh Greptile review on the latest commit.

@greptileai

[Sameerlite]

Good fix direction (metadata flag + map_openai_params), but the cost-map lookup doesn’t match repo patterns.

Don’t call GetModelCostMap.load_local_model_cost_map() directly - use litellm.model_cost + _get_bundled_model_cost_map() (see _get_model_cost_entry_for_provider_config in utils.py).

Don’t fall back to get_model_info() for this flag - it won’t return map_max_tokens_to_max_completion_tokens. Either read the raw map entry or add the flag to ProviderSpecificModelInfo and use _supports_factory.

Health check is overbuilt - _resolve_health_check_model_info duplicates lookup logic and changes more than token params. Simpler path: pass model_info/base_model into litellm_params, keep injecting max_tokens, let AzureOpenAIConfig translate.

Tests and the cost-map entry look good. Please simplify the lookup + health-check path and I’ll re-review.

[Sameerlite]

Thanks so much for this fix, @AleksandrLiadov — the map_max_tokens_to_max_completion_tokens flag approach looks like a clean, idiomatic solution! Triggering a fresh Greptile review on the latest commit.\n\n@greptileai

[shivamrawat1]

@AleksandrLiadov can your resolve the merge conflicts? i can get it to merge then.