fix(openai): preserve reasoning_effort summary field for Responses API - #23151
Conversation
When reasoning_effort is passed as a dict with additional fields like 'summary' or 'generate_summary', preserve the full dict format instead of normalizing it to a string. This ensures that when requests are routed to the OpenAI Responses API, all reasoning parameters are correctly included. The normalization to string format now only happens for simple dicts with just the 'effort' key, which is appropriate for the Chat Completions API. Fixes issue where summary field was being dropped when routing gpt-5.4+ requests with tools + reasoning to Responses API. Made-with: Cursor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes a bug where Key changes:
Issues found:
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/llms/openai/chat/gpt_5_transformation.py | Core change: preserves reasoning_effort dict (with summary/generate_summary fields) for Responses API routing. Adds _get_effort_level (duplicate of existing _normalize_reasoning_effort_for_chat_completion) and is_model_gpt_5_4_plus_model (hardcoded model-version parsing violating the no-hardcoded-model-flags rule). Tool+reasoning guard (lines 228-240) is dead code — the inner condition is never True. |
| litellm/llms/azure/chat/gpt_5_transformation.py | Azure-specific handling correctly uses _get_effort_level for all effort-level guards and adds a new block to drop reasoning_effort when tools are present (Azure doesn't route to Responses API). The is_model_gpt_5_4_plus_model call here inherits the same hardcoded-model-flag issue from the parent class. |
| tests/test_litellm/llms/openai/chat/test_openai_gpt_transformation.py | New TestGPT5ReasoningEffortPreservation class provides good coverage of dict preservation, xhigh validation, none-as-effort for tools/sampling/temperature guards. Tests are mock-only and well-structured. |
| tests/test_litellm/llms/openai/test_gpt5_transformation.py | Renamed and expanded existing tests to match the new preservation behavior; adds regression tests for xhigh-dict, none-dict with sampling, etc. All mock-only. |
| tests/test_litellm/llms/azure/chat/test_azure_gpt5_transformation.py | Adds test_azure_gpt5_4_drops_reasoning_effort_when_tools_present validating the new Azure-specific drop path. Mock-only, clear and correct. |
| tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py | Adds test_map_optional_params_preserves_reasoning_summary verifying the end-to-end Responses API transformation preserves the dict. No real network calls. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["map_openai_params called\nwith reasoning_effort"] --> B{Is reasoning_effort a dict?}
B -->|"Yes: {effort, summary, ...}"| C{Keys == just 'effort'?}
B -->|"No: string"| D["Use as-is\neffective_effort = string"]
C -->|"Yes (only 'effort' key)"| E["Normalize to string\nfor Chat Completions API"]
C -->|"No (has 'summary', etc.)"| F["Preserve full dict\nfor Responses API routing"]
D --> G["effective_effort = _get_effort_level(value)"]
E --> G
F --> G
G --> H{effective_effort == 'xhigh'?}
H -->|"Yes + model unsupported"| I["drop or raise UnsupportedParamsError"]
H -->|"No"| J{is gpt-5.4 model + tools present?}
I --> J
J -->|"Yes (OpenAI)"| K{"is_model_gpt_5_4_plus_model?\n(always True for gpt-5.4)"}
J -->|"No"| L["Continue to sampling/temperature guards"]
K -->|"True → inner condition False"| M["Dead code: reasoning_effort NOT dropped\n(Responses API will handle it)"]
M --> L
L --> N{Azure path?}
N -->|"Yes (gpt-5.4+)"| O["Drop reasoning_effort\n(Azure has no Responses API routing)"]
N -->|"No (OpenAI)"| P["Pass through to Responses API transformation"]
Comments Outside Diff (1)
-
litellm/llms/openai/chat/gpt_5_transformation.py, line 228-240 (link)Dead code block — inner condition is always
Falsewhen outer condition isTrueis_model_gpt_5_4_modelreturnsTrueonly for model names starting withgpt-5.4.is_model_gpt_5_4_plus_modelalso returnsTruefor any model starting withgpt-5.4(sinceint("4") >= 4). Therefore the inner guardnot self.is_model_gpt_5_4_plus_model(model)is alwaysFalsewhen the outerif self.is_model_gpt_5_4_model(model)isTrue.Consequence: the three-line drop path (lines 238–240) is never executed —
reasoning_effortis never removed here. While this may be intentional for OpenAI (Responses API routing), the structure is misleading and the comment ("otherwise drop it for chat completions API") implies there are cases where the drop path runs, which is untrue.Additionally,
reasoning_effort = Noneon line 240 sets a variable that is no longer read after this point (all subsequent guards useeffective_effort), making it doubly dead.
Last reviewed commit: ee3ecb5
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| reasoning_effort = normalized or raw_reasoning_effort | ||
| if reasoning_effort is not None and reasoning_effort == "xhigh": |
There was a problem hiding this comment.
xhigh validation silently bypassed when reasoning_effort is a dict
After this PR, reasoning_effort can be left as a dict (e.g. {"effort": "xhigh", "summary": "detailed"}). On line 184 the guard:
if reasoning_effort is not None and reasoning_effort == "xhigh":will always be False when reasoning_effort is a dict, regardless of whether effort is "xhigh". This means the model-support validation for xhigh is completely skipped for dict-format inputs, so unsupported models will silently receive "xhigh" effort instead of raising UnsupportedParamsError or dropping the param.
The fix needs to extract the effort string before doing these comparisons, for example:
effort_level = (
reasoning_effort.get("effort")
if isinstance(reasoning_effort, dict)
else reasoning_effort
)
if effort_level is not None and effort_level == "xhigh":And use effort_level for all subsequent string comparisons on lines 211 and 221.
…s for dict inputs - Add _get_effort_level() to extract effective effort from string or dict - Use effective_effort for xhigh validation, tool-drop, sampling, temperature guards - Preserve dict format when it has summary/generate_summary for Responses API - Add tests: xhigh-dict validation, none-dict for tools/sampling/temperature - Update tests: dict-with-summary now preserved (not normalized) Made-with: Cursor
| def _get_effort_level(value: Union[str, dict, None]) -> Optional[str]: | ||
| """Extract the effective effort level from reasoning_effort (string or dict). | ||
|
|
||
| Use this for guards that compare effort level (e.g. xhigh validation, "none" checks). | ||
| Ensures dict inputs like {"effort": "none", "summary": "detailed"} are correctly | ||
| treated as effort="none" for validation purposes. | ||
| """ | ||
| if value is None: | ||
| return None | ||
| if isinstance(value, str): | ||
| return value | ||
| if isinstance(value, dict) and "effort" in value: | ||
| return value["effort"] | ||
| return None |
There was a problem hiding this comment.
Duplicate implementation of _normalize_reasoning_effort_for_chat_completion
_get_effort_level (lines 28–41) and _normalize_reasoning_effort_for_chat_completion (lines 11–25) have byte-for-byte identical implementations. Both functions:
- return
NoneforNoneinput - return the string directly for string input
- return
value["effort"]for a dict that contains"effort" - return
Noneas a fallback
Having two functions with the same logic but different names makes it easy for them to diverge in the future. Consider removing _normalize_reasoning_effort_for_chat_completion and calling _get_effort_level everywhere, or at least making one delegate to the other (e.g. _normalize_reasoning_effort_for_chat_completion = _get_effort_level).
| @classmethod | ||
| def is_model_gpt_5_4_plus_model(cls, model: str) -> bool: | ||
| """Check if the model is gpt-5.4 or newer (5.4, 5.5, 5.6, etc., including pro).""" | ||
| model_name = model.split("/")[-1] | ||
| if not model_name.startswith("gpt-5."): | ||
| return False | ||
| try: | ||
| version_str = model_name.replace("gpt-5.", "").split("-")[0] | ||
| major = version_str.split(".")[0] | ||
| return int(major) >= 4 | ||
| except (ValueError, IndexError): | ||
| return False |
There was a problem hiding this comment.
Hardcoded model-version parsing violates the "no model flags in code" rule
is_model_gpt_5_4_plus_model determines model capability by string-parsing the version number directly from the model name (int(major) >= 4). This is the pattern the codebase rule explicitly forbids: every time a new model crosses the 5.4+ threshold (e.g. gpt-5.5, gpt-5.6), users must upgrade LiteLLM to get the correct behavior rather than just getting an update via model_prices_and_context_window.json.
Per the custom rule, the capability flag (routes_to_responses_api_with_tools, or similar) should be stored in model_prices_and_context_window.json and read via get_model_info/_supports_factory, exactly as _supports_reasoning_effort_level already does for effort-level capabilities.
Rule Used: What: Do not hardcode model-specific flags in the ... (source)
…3151) Independent fix (base: main) collaterally removed by PR #23276. Restores: - _get_effort_level() for extracting effort from string or dict - is_model_gpt_5_4_plus_model() classmethod - effective_effort usage in xhigh/tool-drop/sampling/temperature guards - Azure: _get_effort_level import and usage for dict reasoning_effort - Azure: gpt-5.4+ tool+reasoning drop logic
…ing-summary-for-responses-api fix(openai): preserve reasoning_effort summary field for Responses API
…rriAI#23151) Independent fix (base: main) collaterally removed by PR BerriAI#23276. Restores: - _get_effort_level() for extracting effort from string or dict - is_model_gpt_5_4_plus_model() classmethod - effective_effort usage in xhigh/tool-drop/sampling/temperature guards - Azure: _get_effort_level import and usage for dict reasoning_effort - Azure: gpt-5.4+ tool+reasoning drop logic
1 participant
When reasoning_effort is passed as a dict with additional fields like 'summary' or 'generate_summary', preserve the full dict format instead of normalizing it to a string. This ensures that when requests are routed to the OpenAI Responses API, all reasoning parameters are correctly included.
The normalization to string format now only happens for simple dicts with just the 'effort' key, which is appropriate for the Chat Completions API.
Fixes issue where summary field was being dropped when routing gpt-5.4+ requests with tools + reasoning to Responses API.
Made-with: Cursor
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes