fix: convert Pydantic models from TOOL_CALLS_CACHE to dict - #20808
fix: convert Pydantic models from TOOL_CALLS_CACHE to dict#20808themavik wants to merge 1 commit into
Conversation
TOOL_CALLS_CACHE stores ChatCompletionMessageToolCall objects (Pydantic models), but _ensure_tool_results_have_corresponding_ tool_calls expects plain dicts. The isinstance(_, dict) check fails for Pydantic objects, replacing them with empty dicts. This causes malformed tool calls with empty function names and arguments, breaking multi-turn conversations. Now uses model_dump()/dict() to properly convert Pydantic models to dicts, preserving all tool call data. Fixes #20699
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile OverviewGreptile SummaryThis PR adjusts tool-call caching normalization in Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/responses/litellm_completion_transformation/transformation.py | Changes tool-call cache normalization to convert cached Pydantic tool call models to dicts (via model_dump()/dict()) instead of replacing with {}; main concern is the dumped shape may not match the downstream {id,type,function:{name,arguments}} expectations without additional normalization. |
Sequence Diagram
sequenceDiagram
participant Caller as Responses session handler
participant Xform as LiteLLMCompletionResponsesConfig
participant Cache as TOOL_CALLS_CACHE
Caller->>Xform: _ensure_tool_results_have_corresponding_tool_calls(messages, tools)
loop each message
Xform->>Xform: if role != "tool" skip
Xform->>Xform: find previous assistant message
Xform->>Xform: if missing tool_call_id try recover from assistant.tool_calls
alt tool_call_id empty and removable
Xform->>Xform: mark tool message for removal
else tool_call_id present
Xform->>Xform: check assistant.tool_calls contains tool_call_id
alt missing tool_call
Xform->>Cache: get_cache(tool_call_id)
alt cache miss and tools provided
Xform->>Xform: reconstruct minimal tool_call from tools
end
alt cached tool_call is Pydantic model
Xform->>Xform: model_dump()/dict() to convert to dict
end
Xform->>Xform: _create_tool_call_chunk(dict, tool_call_id, idx)
Xform->>Xform: _add_tool_call_to_assistant(prev_assistant, tool_call_chunk)
end
end
end
Xform-->>Caller: fixed_messages (tool_calls added / some tool msgs removed)
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if _tool_use_definition: | ||
| if not isinstance(_tool_use_definition, dict): | ||
| _tool_use_definition = {} | ||
| # Convert Pydantic models (e.g. ChatCompletionMessageToolCall | ||
| # from TOOL_CALLS_CACHE) to dict to preserve tool call data | ||
| if hasattr(_tool_use_definition, 'model_dump'): | ||
| _tool_use_definition = _tool_use_definition.model_dump() | ||
| elif hasattr(_tool_use_definition, 'dict'): | ||
| _tool_use_definition = _tool_use_definition.dict() | ||
| else: | ||
| _tool_use_definition = {} |
There was a problem hiding this comment.
Pydantic dump may break expected shape
_create_tool_call_chunk expects a tool_use_definition dict with top-level keys like id, type, and a nested function dict containing name/arguments (see _create_tool_call_chunk at transformation.py:606-628). Converting cached ChatCompletionMessageToolCall via model_dump()/dict() may produce a different shape depending on model config (e.g., function may be a Pydantic object that becomes a dict with extra fields, or arguments may not be a string). This will still lead to empty name or malformed arguments in the reconstructed tool call.
Consider normalizing the dumped object into the exact {id,type,function:{name,arguments}} structure (and coercing arguments to a JSON string) before calling _create_tool_call_chunk, rather than passing the raw dump through.
|
Closing in favor of #20700 |
ghost
closed this
2 participants
Summary
Fixes #20699
Root cause:
TOOL_CALLS_CACHEstoresChatCompletionMessageToolCallPydantic objects, but_ensure_tool_results_have_corresponding_tool_callschecksisinstance(_, dict)which fails for Pydantic models. The code replaces the cached value with{}, creating malformed tool calls with empty function names and arguments.Changes
litellm/responses/litellm_completion_transformation/transformation.py: Instead of replacing non-dict cached values with{}, usemodel_dump()(Pydantic v2) ordict()(Pydantic v1) to properly convert the Pydantic model to a dict while preserving all tool call data.Risk Assessment
Low - The fix only changes the fallback behavior when a cached value isn't already a dict. It preserves data that was previously being discarded.