Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds day-0 support for Key blockers:
Not flagged (speculative in prior reviews):
Confidence Score: 1/5
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["Model String"] --> B{"is_model_gpt_5_1_model?"}
B -->|startswith gpt-5.1| C["True — full sampling params allowed"]
B -->|startswith gpt-5.2, no pro, no -chat| C
B -->|startswith gpt-5.4, no pro, no -chat| C
B -->|other gpt-5 models| D["False — logprobs/top_p/top_logprobs removed"]
A --> E{"is_model_gpt_5_2_model?"}
E -->|startswith gpt-5.2| F["True — xhigh effort + Azure logprobs"]
E -->|startswith gpt-5.4| F
E -->|other| G["False"]
F --> H{"Azure path?"}
H -->|Yes| I["logprobs added back to supported params"]
H -->|No — OpenAI| J["xhigh reasoning_effort allowed"]
K["⚠️ Issue: Both methods use hardcoded startswith checks<br/>violating no-hardcoding rule 2605a1b1"]
L["⚠️ Issue: is_model_gpt_5_2_model returns true for gpt-5.4<br/>despite its name — semantically incorrect"]
Last reviewed commit: 3b457b5 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| is_gpt_5_4 = ( | ||
| model_name.startswith("gpt-5.4") | ||
| and "pro" not in model_name | ||
| and not model_name.startswith("gpt-5.4-chat") | ||
| ) | ||
| return is_gpt_5_1 or is_gpt_5_2 or is_gpt_5_4 |
There was a problem hiding this comment.
Hardcoded model name violates no-hardcoding rule
The custom rule for this repository explicitly prohibits hardcoding model-specific flags directly in transformation code. Instead, capabilities should be read from model_prices_and_context_window.json via helper functions like get_model_info or supports_reasoning().
gpt-5.4 is already added to the JSON with "supports_reasoning": true, "supports_tool_choice": true, etc. The code should leverage those fields rather than adding another startswith("gpt-5.4") guard. This means every future model revision (e.g. gpt-5.5, gpt-5.4-turbo) will again require a code change instead of a JSON update.
The same hardcoded pattern also exists in is_model_gpt_5_2_model (line 82) and propagates to the Azure config at litellm/llms/azure/chat/gpt_5_transformation.py:49,51.
Context Used: Rule from dashboard - What: Do not hardcode model-specific flags in the codebase. Instead, put them in model_prices_and_co... (source)
| def is_model_gpt_5_2_model(cls, model: str) -> bool: | ||
| """Check if the model is a gpt-5.2 variant (including pro).""" | ||
| model_name = model.split("/")[-1] | ||
| return model_name.startswith("gpt-5.2") | ||
| return model_name.startswith("gpt-5.2") or model_name.startswith("gpt-5.4") |
There was a problem hiding this comment.
is_model_gpt_5_2_model now incorrectly returns True for gpt-5.4
The method is named is_model_gpt_5_2_model but now returns True for gpt-5.4 models too:
return model_name.startswith("gpt-5.2") or model_name.startswith("gpt-5.4")This is semantically wrong and will be a source of confusion for future contributors who call is_model_gpt_5_2_model(model) and expect it to only match gpt-5.2 family models. All downstream callers in azure/chat/gpt_5_transformation.py (lines 49 and 51) now silently treat gpt-5.4 as gpt-5.2 without any indication that this is intentional.
If gpt-5.4 needs the same logprobs/xhigh treatment as gpt-5.2, a more maintainable approach would be to introduce a separate is_model_gpt_5_4_model method and update the call-sites explicitly, or better, to consolidate these checks into a single capability-based helper that reads from model info.
| "supports_web_search": true | ||
| }, | ||
| "gpt-5.4": { | ||
| "cache_read_input_token_cost": 2.5e-07, | ||
| "cache_read_input_token_cost_priority": 5e-07, | ||
| "input_cost_per_token": 2.5e-06, | ||
| "input_cost_per_token_priority": 5e-06, | ||
| "litellm_provider": "openai", | ||
| "max_input_tokens": 272000, | ||
| "max_output_tokens": 128000, | ||
| "max_tokens": 128000, | ||
| "mode": "responses", | ||
| "output_cost_per_token": 1.5e-05, | ||
| "output_cost_per_token_priority": 3e-05, | ||
| "supported_endpoints": [ | ||
| "/v1/chat/completions", | ||
| "/v1/batch", | ||
| "/v1/responses" | ||
| ], | ||
| "supported_modalities": [ | ||
| "text", | ||
| "image" | ||
| ], | ||
| "supported_output_modalities": [ | ||
| "text", | ||
| "image" | ||
| ], | ||
| "supports_function_calling": true, | ||
| "supports_native_streaming": true, | ||
| "supports_parallel_function_calling": true, | ||
| "supports_pdf_input": true, | ||
| "supports_prompt_caching": true, | ||
| "supports_reasoning": true, | ||
| "supports_response_schema": true, | ||
| "supports_system_messages": true, | ||
| "supports_tool_choice": true, | ||
| "supports_service_tier": true, | ||
| "supports_vision": true | ||
| }, | ||
| "gpt-5-pro": { | ||
| "input_cost_per_token": 1.5e-05, | ||
| "input_cost_per_token_batches": 7.5e-06, |
There was a problem hiding this comment.
mode field differs from gpt-5.2
gpt-5.4 is registered with "mode": "responses", while gpt-5.2 (line 20618) uses "mode": "chat". If the mode field drives routing or feature detection elsewhere in LiteLLM, this inconsistency could cause gpt-5.4 to behave differently from gpt-5.2 in ways that aren't related to their actual API differences.
If gpt-5.4 genuinely operates only via the /v1/responses endpoint, this is correct — but it's worth double-checking because both models list /v1/chat/completions under supported_endpoints.
| docker pull ghcr.io/berriai/litellm:v1.81.14-stable.gpt-4o | ||
| ``` | ||
|
|
There was a problem hiding this comment.
Docker image tag references wrong model
The Docker image pulled and run in the usage example is tagged v1.81.14-stable.gpt-4o, which refers to GPT-4o, not GPT-5.4. This should either be a generic/latest tag or updated to a tag that actually contains GPT-5.4 support.
| docker pull ghcr.io/berriai/litellm:v1.81.14-stable.gpt-4o | |
| ``` | |
| docker pull ghcr.io/berriai/litellm:main-latest |
| is_gpt_5_4 = ( | ||
| model_name.startswith("gpt-5.4") | ||
| and "pro" not in model_name | ||
| and not model_name.startswith("gpt-5.4-chat") | ||
| ) | ||
| return is_gpt_5_1 or is_gpt_5_2 or is_gpt_5_4 |
There was a problem hiding this comment.
Hardcoded model-specific flag violates the no-hardcoding rule. Custom rule [2605a1b1] explicitly requires putting model capabilities in model_prices_and_context_window.json and reading them via helper functions like supports_reasoning(), not via startswith() checks in the transformation code. This pattern propagates further to is_model_gpt_5_2_model() (line 82) and the Azure config. Future model revisions (e.g., gpt-5.5, gpt-5.4-turbo) will again require code changes instead of just updating the JSON.
If gpt-5.4 needs the same parameter handling as gpt-5.2 and gpt-5.1, add the necessary capability fields to its JSON entry and use a helper function to check them.
Context Used: Rule from dashboard - What: Do not hardcode model-specific flags in the codebase. Instead, put them in model_prices_and_co... (source)
| def is_model_gpt_5_2_model(cls, model: str) -> bool: | ||
| """Check if the model is a gpt-5.2 variant (including pro).""" | ||
| model_name = model.split("/")[-1] | ||
| return model_name.startswith("gpt-5.2") | ||
| return model_name.startswith("gpt-5.2") or model_name.startswith("gpt-5.4") |
There was a problem hiding this comment.
Method name is_model_gpt_5_2_model() is semantically incorrect — it now returns True for both gpt-5.2 and gpt-5.4 models:
return model_name.startswith("gpt-5.2") or model_name.startswith("gpt-5.4")This misleads future contributors who expect the method to only match gpt-5.2 variants. Downstream callers in azure/chat/gpt_5_transformation.py (lines 49, 51) silently apply gpt-5.2 parameter handling (logprobs support) to gpt-5.4 without explicit indication.
Consider either introducing a separate is_model_gpt_5_4_model() method with explicit call-site updates, or consolidating into a single capability-based helper that reads from model info.
| ## Docker Image | ||
|
|
||
| ```bash | ||
| docker pull ghcr.io/berriai/litellm:v1.81.14-stable.gpt-4o |
There was a problem hiding this comment.
Docker image tag references GPT-4o instead of GPT-5.4. The tag v1.81.14-stable.gpt-4o explicitly names gpt-4o, which conflicts with this blog post documenting gpt-5.4 support. Update to a release tag that actually contains gpt-5.4 support, or use a generic/latest tag if the appropriate release doesn't exist yet.
|
Related follow-up: #22734 adds the remaining ChatGPT OAuth aliases ( |
2 participants
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes