Repository navigation
fix(azure): rename max_tokens to max_completion_tokens for gpt-5-chat deployments #36857
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -112,6 +112,16 @@ def get_supported_openai_params(self, model: str) -> list[str]: | |
| "store", | ||
| ] | ||
|
|
||
| @classmethod | ||
| def requires_max_completion_tokens(cls, model: str) -> bool: | ||
| """Whether Azure rejects the legacy ``max_tokens`` key for this deployment. | ||
|
|
||
| Deliberately wider than ``AzureOpenAIGPT5Config.is_model_gpt_5_model``: the whole gpt-5 | ||
| name family needs the rename, including the ``gpt-5-chat*`` models that are excluded from | ||
| the reasoning path by https://github.com/BerriAI/litellm/issues/13781. | ||
| """ | ||
| return "gpt-5" in model or "gpt5_series" in model | ||
|
|
||
| def _is_response_format_supported_model(self, model: str) -> bool: | ||
| """ | ||
| Determines if the model supports response_format. | ||
|
|
@@ -160,6 +170,7 @@ def map_openai_params( | |
| api_version: str = "", | ||
| ) -> dict: | ||
| supported_openai_params: Final = self.get_supported_openai_params(model) | ||
| renames_max_tokens: Final = self.requires_max_completion_tokens(model) | ||
| api_version_times: Final = api_version.split("-") | ||
|
|
||
| if len(api_version_times) >= 3: | ||
|
|
@@ -172,7 +183,9 @@ def map_openai_params( | |
| api_version_day = None | ||
|
|
||
| for param, value in non_default_params.items(): | ||
| if param == "tool_choice": | ||
| if param == "max_tokens" and renames_max_tokens: | ||
| optional_params.setdefault("max_completion_tokens", value) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Calling Context Used: CLAUDE.md (source)
veria-ai[bot] marked this conversation as resolved.
|
||
| elif param == "tool_choice": | ||
| """ | ||
| This parameter requires API version 2023-12-01-preview or later | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new predicate infers token-parameter support exclusively from
gpt-5andgpt5_seriessubstrings instead of model metadata. This requires LiteLLM releases for future capability changes and lets naming aliases select the wrong outbound parameter; represent the capability in model metadata and access it throughget_model_info.Rule Used: What: Do not hardcode model-specific flags in the ... (source)
Knowledge Base Used: LLM Provider Adapters
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!