fix(ollama): set finish_reason to tool_calls and remove broken capability check - #18924
Merged
5 commits merged intoJan 13, 2026
Merged
Conversation
When qwen3 models return tool_calls through Ollama, the finish_reason was incorrectly left as "stop" instead of being set to "tool_calls". This caused clients to miss the tool_calls in the response. Added _get_finish_reason helper method following OpenAI provider's pattern, and fixed both streaming and non-streaming response paths. Fixes: BerriAI#18922
The previous code tried to check model capability via get_model_info() which made network calls to localhost:11434. When Ollama is remote, this fails and falls back to JSON format, breaking tool calling. Ollama 0.4+ supports native tool calling - let Ollama handle model capability detection instead of LiteLLM. Fixes BerriAI#18922
Ollama returns tool_calls with arguments as dict, but OpenAI format requires arguments to be a JSON string. Also ensures 'type': 'function' field is present. Completes the fix for BerriAI#18922
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Fixes BerriAI#18922 Two issues addressed: 1. Remove broken model capability check - get_model_info() fails when Ollama runs on remote server - Broken fallback triggered JSON prompt injection - Now passes tools directly - Ollama 0.4+ handles detection 2. Set finish_reason correctly - Was hardcoded to "stop" even with tool_calls present - Clients use this to know how to process the response - Now returns "tool_calls" when tool_calls are in response Both streaming and non-streaming responses are fixed. Tests: - All 14 existing Ollama tests pass - Added 3 focused tests for the fixes
ghost
merged commit Jan 13, 2026
f76938a
into
BerriAI:litellm_staging_01_13_2026
6 of 7 checks passed
Jordi-Jaspers
added a commit
to Jordi-Jaspers/litellm
that referenced
this pull request
Feb 6, 2026
…ct finish_reason Ollama sends tool_calls in one chunk and done:true in a separate chunk. The existing fix from BerriAI#18924 only checked for tool_calls in the current chunk when determining finish_reason, causing it to miss tool_calls from previous chunks and incorrectly return 'stop' instead of 'tool_calls'. This commit adds a seen_tool_calls state variable to OllamaChatCompletionResponseIterator that tracks whether tool_calls were seen in any chunk during streaming. When the final chunk arrives with done:true, it checks both the current chunk and the historical state to set the correct finish_reason. Fixes: BerriAI#18922 (streaming case)
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…lity check (BerriAI#18924) * Update CLAUDE.md with qwen3 tool_calls bug fix instructions (BerriAI#18922) * fix(ollama): set finish_reason to "tool_calls" when tool_calls present When qwen3 models return tool_calls through Ollama, the finish_reason was incorrectly left as "stop" instead of being set to "tool_calls". This caused clients to miss the tool_calls in the response. Added _get_finish_reason helper method following OpenAI provider's pattern, and fixed both streaming and non-streaming response paths. Fixes: BerriAI#18922 * fix(ollama): pass tools directly without model capability check The previous code tried to check model capability via get_model_info() which made network calls to localhost:11434. When Ollama is remote, this fails and falls back to JSON format, breaking tool calling. Ollama 0.4+ supports native tool calling - let Ollama handle model capability detection instead of LiteLLM. Fixes BerriAI#18922 * fix(ollama): transform tool_calls response to OpenAI format Ollama returns tool_calls with arguments as dict, but OpenAI format requires arguments to be a JSON string. Also ensures 'type': 'function' field is present. Completes the fix for BerriAI#18922 * fix(ollama): set finish_reason to "tool_calls" when tool_calls present Fixes BerriAI#18922 Two issues addressed: 1. Remove broken model capability check - get_model_info() fails when Ollama runs on remote server - Broken fallback triggered JSON prompt injection - Now passes tools directly - Ollama 0.4+ handles detection 2. Set finish_reason correctly - Was hardcoded to "stop" even with tool_calls present - Clients use this to know how to process the response - Now returns "tool_calls" when tool_calls are in response Both streaming and non-streaming responses are fixed. Tests: - All 14 existing Ollama tests pass - Added 3 focused tests for the fixes
4 tasks
This pull request was closed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix Ollama tool calling issues
Fixes: #18922
The Problems
Two issues causing tool calls to be ignored:
finish_reasonhardcoded to"stop"- Clients usefinish_reasonto determine how to process responses. When tool_calls were present, it should be"tool_calls"not"stop".Broken model capability check -
get_model_info()fails when Ollama runs on a remote server, triggering a broken JSON prompt injection fallback.The Fix
~30 lines removed, ~10 lines added.
Breaking Change
Removes the old JSON prompt injection fallback for pre-0.4 Ollama. That approach was unreliable anyway, and Ollama 0.4 has been out for a while with proper native tool calling.
Testing