Skip to content

fix(ollama): set finish_reason to tool_calls and remove broken capability check - #18924

Merged
5 commits merged into
BerriAI:litellm_staging_01_13_2026from
rsp2k:fix/qwen3-tool-calls
Jan 13, 2026
Merged

fix(ollama): set finish_reason to tool_calls and remove broken capability check#18924
5 commits merged into
BerriAI:litellm_staging_01_13_2026from
rsp2k:fix/qwen3-tool-calls

Conversation

@rsp2k

@rsp2k rsp2k commented Jan 11, 2026

Copy link
Copy Markdown
Contributor

Fix Ollama tool calling issues

Fixes: #18922

The Problems

Two issues causing tool calls to be ignored:

  1. finish_reason hardcoded to "stop" - Clients use finish_reason to determine how to process responses. When tool_calls were present, it should be "tool_calls" not "stop".

  2. Broken model capability check - get_model_info() fails when Ollama runs on a remote server, triggering a broken JSON prompt injection fallback.

The Fix

~30 lines removed, ~10 lines added.

# Before: Complex try/except with network call and broken fallback
try:
    model_info = litellm.get_model_info(model=model, custom_llm_provider="ollama")
    if model_info.get("supports_function_calling") is True:
        optional_params["tools"] = value
    else:
        raise Exception
except Exception:
    # Broken fallback with JSON prompt injection...

# After: Just pass tools directly - Ollama 0.4+ handles capability detection
optional_params["tools"] = value
# Before: finish_reason always "stop"
_message = litellm.Message(**response_json_message)
model_response.choices[0].message = _message

# After: Set "tool_calls" when tool_calls present
_message = litellm.Message(**response_json_message)
model_response.choices[0].message = _message
if _message.tool_calls:
    model_response.choices[0].finish_reason = "tool_calls"

Breaking Change

Removes the old JSON prompt injection fallback for pre-0.4 Ollama. That approach was unreliable anyway, and Ollama 0.4 has been out for a while with proper native tool calling.

Testing

  • All 14 existing Ollama tests pass
  • Added 3 focused tests for the fixes
  • Live tested with qwen3 + Ollama - tool_calls work correctly

rsp2k and others added 4 commits January 11, 2026 00:05
When qwen3 models return tool_calls through Ollama, the finish_reason
was incorrectly left as "stop" instead of being set to "tool_calls".
This caused clients to miss the tool_calls in the response.

Added _get_finish_reason helper method following OpenAI provider's
pattern, and fixed both streaming and non-streaming response paths.

Fixes: BerriAI#18922
The previous code tried to check model capability via get_model_info()
which made network calls to localhost:11434. When Ollama is remote,
this fails and falls back to JSON format, breaking tool calling.

Ollama 0.4+ supports native tool calling - let Ollama handle
model capability detection instead of LiteLLM.

Fixes BerriAI#18922
Ollama returns tool_calls with arguments as dict, but OpenAI format
requires arguments to be a JSON string. Also ensures 'type': 'function'
field is present.

Completes the fix for BerriAI#18922
@vercel

vercel Bot commented Jan 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
litellm Ready Ready Preview, Comment Jan 11, 2026 5:43pm

@CLAassistant

CLAassistant commented Jan 11, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Fixes BerriAI#18922

Two issues addressed:

1. Remove broken model capability check
   - get_model_info() fails when Ollama runs on remote server
   - Broken fallback triggered JSON prompt injection
   - Now passes tools directly - Ollama 0.4+ handles detection

2. Set finish_reason correctly
   - Was hardcoded to "stop" even with tool_calls present
   - Clients use this to know how to process the response
   - Now returns "tool_calls" when tool_calls are in response

Both streaming and non-streaming responses are fixed.

Tests:
- All 14 existing Ollama tests pass
- Added 3 focused tests for the fixes
@rsp2k rsp2k changed the title Fix qwen3 tool_calls being dropped in Ollama provider fix(ollama): set finish_reason to tool_calls and remove broken capability check Jan 11, 2026
@ghost
ghost changed the base branch from main to litellm_staging_01_13_2026 January 13, 2026 22:20
@ghost
ghost merged commit f76938a into BerriAI:litellm_staging_01_13_2026 Jan 13, 2026
6 of 7 checks passed
Jordi-Jaspers added a commit to Jordi-Jaspers/litellm that referenced this pull request Feb 6, 2026
…ct finish_reason

Ollama sends tool_calls in one chunk and done:true in a separate chunk.
The existing fix from BerriAI#18924 only checked for tool_calls in the current chunk
when determining finish_reason, causing it to miss tool_calls from previous
chunks and incorrectly return 'stop' instead of 'tool_calls'.

This commit adds a seen_tool_calls state variable to OllamaChatCompletionResponseIterator
that tracks whether tool_calls were seen in any chunk during streaming. When the
final chunk arrives with done:true, it checks both the current chunk and the
historical state to set the correct finish_reason.

Fixes: BerriAI#18922 (streaming case)
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…lity check (BerriAI#18924)

* Update CLAUDE.md with qwen3 tool_calls bug fix instructions (BerriAI#18922)

* fix(ollama): set finish_reason to "tool_calls" when tool_calls present

When qwen3 models return tool_calls through Ollama, the finish_reason
was incorrectly left as "stop" instead of being set to "tool_calls".
This caused clients to miss the tool_calls in the response.

Added _get_finish_reason helper method following OpenAI provider's
pattern, and fixed both streaming and non-streaming response paths.

Fixes: BerriAI#18922

* fix(ollama): pass tools directly without model capability check

The previous code tried to check model capability via get_model_info()
which made network calls to localhost:11434. When Ollama is remote,
this fails and falls back to JSON format, breaking tool calling.

Ollama 0.4+ supports native tool calling - let Ollama handle
model capability detection instead of LiteLLM.

Fixes BerriAI#18922

* fix(ollama): transform tool_calls response to OpenAI format

Ollama returns tool_calls with arguments as dict, but OpenAI format
requires arguments to be a JSON string. Also ensures 'type': 'function'
field is present.

Completes the fix for BerriAI#18922

* fix(ollama): set finish_reason to "tool_calls" when tool_calls present

Fixes BerriAI#18922

Two issues addressed:

1. Remove broken model capability check
   - get_model_info() fails when Ollama runs on remote server
   - Broken fallback triggered JSON prompt injection
   - Now passes tools directly - Ollama 0.4+ handles detection

2. Set finish_reason correctly
   - Was hardcoded to "stop" even with tool_calls present
   - Clients use this to know how to process the response
   - Now returns "tool_calls" when tool_calls are in response

Both streaming and non-streaming responses are fixed.

Tests:
- All 14 existing Ollama tests pass
- Added 3 focused tests for the fixes
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants