fix(ollama): one stream id, 400 on non-text tool content, separated tool results, no empty message item

[mateo-berri]

TLDR

Problem this solves:

• ollama/ streams gave every chunk its own chatcmpl- id
• A tool message with a number as content answered 500 instead of 400
• /v1/responses streams on a tool call ended with an empty message item beside the function_call, and the OpenAI SDK's responses.stream() crashed on the item's index
• Consecutive tool results and text parts reached the model run together with no separator

How it solves it:

• The ollama/ stream mints one id and stamps it on every chunk
• Non-text tool content, non-list user content, a part with no value, and a part that is not an object are a 400 naming the message and what it carried. On a proxy with a database, the enterprise managed-files pre-call hook rejects non-list user content first with its own error, so this PR's wording reaches user content only on a proxy without a database (Caveats, first row)
• The Responses bridge numbers output items in the order they stream, from 0, and response.completed lists the items the stream announced in that order
• Merged user, tool, and function turns and their text parts are joined with a newline

Devin opened this PR under my own account (the org opens PRs as the session's user now), so it needs a reviewer's approval the usual way

User Flow

Before: a developer whose agent app calls tools on an ollama/ model through the proxy gets a tool round-trip that no OpenAI client handles cleanly

1. They send POST https://litellm-domain/v1/chat/completions with "model": "ollama/qwen3:0.6b", tools, and "stream": true; every SSE chunk carries a different chatcmpl-... id
2. They send the tool result back as one tool message with two text parts, or as two tool messages; the model reads them as one word, Paris: 22 degreesclear skies
3. Their tool returned a number, so they send "content": 22 on the tool message; the proxy answers 500 can only concatenate str (not "int") to str
4. The same first turn over POST https://litellm-domain/v1/responses with "stream": true ends in response.completed listing an empty message item next to the function_call, while the non-stream call lists the function_call alone
5. They consume that same /v1/responses stream with the OpenAI SDK's client.responses.stream() against an Anthropic deployment on the same proxy, whose stream carries no reasoning item; the SDK raises IndexError: list index out of range, because the only item is announced at output_index 1 and nothing was announced at 0

After: the same round-trip behaves like one against an OpenAI model

1. They send the same POST https://litellm-domain/v1/chat/completions stream; every chunk carries the one chatcmpl-... id of that response
2. They send the same two-part or two-message tool result; the model reads Paris: 22 degrees and clear skies on separate lines
3. They send the same "content": 22 tool message; the proxy answers 400 the tool message at index 2 has int content; content must be a string or a list of content parts
4. The same POST https://litellm-domain/v1/responses stream ends in response.completed listing the function_call alone, as the non-stream call does
5. The same client.responses.stream() call completes; the function_call is announced at output_index 0 and get_final_response().output lists it alone

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Last updated: c7ac238, a tests-only commit adding the audit's integration cells. QA and /live-pr-risk ran on 169c0c1, the last commit with a product diff; /audit ran on c7ac238, whose product code is identical

/audit round 1 of 3: head c7ac238, merge base 127278f

Three rigs on one host, each a two-worker proxy from its commit with its own Postgres, Redis and scripted upstream, booted and run through the integration harness (tests/integration/run.py providers <node ids>, serial as CircleCI runs the group, INTEGRATION_PROXY_READY_SECONDS=240). The 40 cells below ran on the merge base, then twice on the head with --seed 20261007 --order-seed 20261007 (40 collected and 40 passed both times, no skips, no retries); the 30 controls ran once on a third rig from the head. Head product code is unchanged since 169c0c1: c7ac238 adds the audit's cells. No litellm component is mocked; the doubles are the scripted Ollama (/api/generate) and a scripted OpenAI-compatible SSE wire for the bridge cells

Row	Endpoint	Mode	Client	Scenario	Cell	Base	Head x2
1a	/v1/chat/completions	stream	async OpenAI SDK	text answer after a tool result carries one id	test_async_openai_sdk_stream_answers_the_tool_result_in_plain_text in test_ollama_prompt_tools_wire.py	fail	pass, pass
1c	/v1/completions	stream	sync OpenAI SDK	one id across the text-completion chunks, spend row by it	test_openai_sdk_completions_stream_keeps_one_id in test_ollama_prompt_tools_wire.py	fail	pass, pass
1d	/v1/chat/completions	stream	httpx	every data: frame shares the id, [DONE] last	test_raw_chat_stream_frames_share_one_id_and_end_with_done in test_ollama_prompt_tools_wire.py	fail	pass, pass
1g	/v1/chat/completions	stream, cache hit	async OpenAI SDK	second identical stream served from cache, one id each, Ollama reached once	test_cached_stream_replay_keeps_one_id_and_reaches_ollama_once in test_ollama_prompt_tools_wire.py	fail	pass, pass
2a	/v1/chat/completions	non-stream	httpx	int tool content: 400 naming the tool message, deployment serves next	test_int_tool_result_content_is_a_400_naming_the_field_and_leaves_the_deployment_serving in test_ollama_prompt_tools_wire.py	fail	pass, pass
2a	/v1/chat/completions	non-stream	httpx	int tool content names 'the tool message at index 2 has int content'	test_malformed_content_is_a_400_naming_the_message_and_field[tool-int-content] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2b	/v1/chat/completions	non-stream	httpx	int user content rejected by the managed-files pre-call hook, never forwarded (pre-existing, tracked separately)	test_non_list_user_content_is_rejected_before_it_reaches_ollama[user-int-content] in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
2c	/v1/chat/completions	non-stream	httpx	dict user content rejected before translation (pre-existing, tracked separately)	test_non_list_user_content_is_rejected_before_it_reaches_ollama[user-dict-content] in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
2d	/v1/chat/completions	non-stream	httpx	tool text part with int text	test_malformed_content_is_a_400_naming_the_message_and_field[tool-int-text-part] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2e	/v1/chat/completions	non-stream	httpx	tool text part with no text	test_malformed_content_is_a_400_naming_the_message_and_field[tool-text-part-without-text] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2f	/v1/chat/completions	non-stream	httpx	user image_url int (base 200 with the part dropped)	test_malformed_content_is_a_400_naming_the_message_and_field[user-int-image-url] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2g	/v1/chat/completions	non-stream	httpx	user image_url object without url	test_malformed_content_is_a_400_naming_the_message_and_field[user-image-url-object-without-url] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2h	/v1/chat/completions	non-stream	httpx	user image_url part with no image_url	test_malformed_content_is_a_400_naming_the_message_and_field[user-image-url-part-without-image-url] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2i	/v1/chat/completions	non-stream	httpx	bare-string user part rejected before translation (pre-existing, tracked separately)	test_non_list_user_content_is_rejected_before_it_reaches_ollama[user-string-part] in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
2j	/v1/chat/completions	non-stream	httpx	the 400 writes one failure spend row, the next call succeeds	test_malformed_content_400_writes_a_failure_spend_row_and_leaves_the_deployment_serving in test_ollama_prompt_tools_wire.py	fail	pass, pass
2j	/v1/chat/completions	non-stream	httpx	bad key on the malformed shape: 401, Ollama untouched	test_malformed_content_on_an_unauthenticated_request_is_a_401_before_translation in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
2k	/v1/messages	non-stream	sync Anthropic SDK	int tool_result content normalized away by the adapter, section dropped	test_anthropic_sdk_int_tool_result_content_is_dropped_before_the_prompt in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
2l	/v1/responses	non-stream	sync OpenAI SDK	int function_call_output reaches the prompt as text	test_openai_sdk_responses_int_function_output_reaches_the_prompt_as_text in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
2m	/v1/chat/completions	non-stream burst of 12	async OpenAI SDK	8 answers, 4 BadRequestError naming the tool message, 8 prompts, 8 success + 4 failure rows	test_concurrent_valid_and_malformed_turns_are_each_answered_in_their_own_shape in test_ollama_prompt_tools_wire.py	fail	pass, pass
2n	/v1/chat/completions	non-stream	httpx	dict tool content	test_malformed_content_is_a_400_naming_the_message_and_field[tool-dict-content] in test_ollama_prompt_tools_wire.py	fail	pass, pass
2o	/v1/chat/completions	non-stream	httpx	tool content list holding a bare string	test_malformed_content_is_a_400_naming_the_message_and_field[tool-string-part] in test_ollama_prompt_tools_wire.py	fail	pass, pass
3a	/v1/chat/completions	non-stream	httpx	two text parts in one tool result joined by a newline	test_tool_result_content_shapes_reach_the_prompt[text-parts-joined-by-newline] in test_ollama_prompt_tools_wire.py	fail	pass, pass
3b	/v1/chat/completions	non-stream	httpx	the same tool result twice, separated	test_the_same_tool_result_twice_is_forwarded_twice_under_one_instruction in test_ollama_prompt_tools_wire.py	fail	pass, pass
3c	/v1/chat/completions	non-stream	httpx	two consecutive user messages	test_consecutive_user_messages_are_joined_by_a_newline in test_ollama_prompt_tools_wire.py	fail	pass, pass
3d	/v1/chat/completions	non-stream	httpx	tool result then user text	test_a_tool_result_followed_by_user_text_is_joined_by_a_newline in test_ollama_prompt_tools_wire.py	fail	pass, pass
3e	/v1/messages	non-stream	sync Anthropic SDK	two tool_result blocks in one user turn	test_anthropic_sdk_two_tool_results_in_one_turn_are_separated in test_ollama_prompt_tools_wire.py	fail	pass, pass
3f	/v1/messages	stream	async Anthropic SDK	tool_result plus text block in one user turn	test_async_anthropic_sdk_stream_separates_a_tool_result_from_user_text_in_one_turn in test_ollama_prompt_tools_wire.py	fail	pass, pass
3g	/v1/responses	non-stream	sync OpenAI SDK	two function_call_output items	test_openai_sdk_responses_two_function_outputs_are_separated in test_ollama_prompt_tools_wire.py	fail	pass, pass
3h	/v1/chat/completions	non-stream	httpx	empty-string user content skipped	test_empty_and_null_user_content_are_dropped_from_the_user_section[empty-string-skipped] in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
3h	/v1/chat/completions	non-stream	httpx	empty text parts add nothing	test_empty_and_null_user_content_are_dropped_from_the_user_section[empty-text-parts-dropped] in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
3h	/v1/chat/completions	non-stream	httpx	null user content skipped	test_empty_and_null_user_content_are_dropped_from_the_user_section[null-content-skipped] in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
3i	/v1/chat/completions	non-stream	httpx	input_audio part dropped, text kept	test_a_content_part_of_an_unknown_type_is_dropped_from_the_prompt in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
4a	/v1/responses	stream	async OpenAI SDK	tool-call stream completes with [function_call] only	test_async_openai_sdk_responses_stream_emits_the_function_call_item in test_ollama_prompt_tools_wire.py	fail	pass, pass
4b	/v1/responses	stream	sync OpenAI SDK responses.stream()	one output_item.added (function_call, 0); base raised IndexError	test_openai_sdk_responses_stream_context_manager_gets_the_function_call_without_an_empty_message in test_ollama_prompt_tools_wire.py	fail	pass, pass
4c	/v1/responses	stream	async OpenAI SDK	thinking-only stream: [reasoning] at 0, no empty message item	test_async_openai_sdk_responses_stream_of_thinking_only_emits_one_reasoning_item in test_ollama_prompt_tools_wire.py	fail	pass, pass
4d	/v1/responses	stream	async OpenAI SDK	empty answer keeps one message item at 0	test_async_openai_sdk_responses_stream_with_an_empty_answer_keeps_one_message_item in test_ollama_prompt_tools_wire.py	pass (same on both sides by design)	pass, pass
4e	/v1/responses (bridged hosted_vllm)	stream	sync OpenAI SDK responses.stream()	role-only chunk then tool call: added [(function_call, 0)]	test_openai_sdk_responses_stream_with_a_tool_call_only_reply_announces_no_message_item in test_responses_bridge_stream_output_items_wire.py	fail	pass, pass
4f	/v1/responses (bridged hosted_vllm)	stream	async OpenAI SDK	text reply: one message item at 0	test_async_openai_sdk_responses_stream_text_reply_has_one_message_item_at_index_zero in test_responses_bridge_stream_output_items_wire.py	pass (same on both sides by design)	pass, pass
4g	/v1/responses (bridged hosted_vllm)	stream	async OpenAI SDK	reasoning_content then text: [(reasoning,0),(message,1)]	test_async_openai_sdk_responses_stream_reasoning_then_text_gets_contiguous_output_indexes in test_responses_bridge_stream_output_items_wire.py	fail	pass, pass
4h	/v1/responses (bridged hosted_vllm)	stream	httpx	text then tool call: [(message,0),(function_call,1)]	test_raw_responses_stream_text_then_tool_call_keeps_the_message_first in test_responses_bridge_stream_output_items_wire.py	pass (same on both sides by design)	pass, pass

Controls, once on the head rig (29 pass): test_async_openai_sdk_stream_flushes_the_held_tool_call_once, test_async_anthropic_sdk_stream_answers_the_tool_result_in_text, test_async_openai_sdk_responses_stream_answers_the_function_output_in_text, test_tool_result_content_shapes_reach_the_prompt[5kb-string-forwarded-intact], test_tool_result_content_shapes_reach_the_prompt[empty-string-drops-the-section], test_responses_stream_completes_through_empty_choices_metadata_and_usage_chunks, test_codex_namespace_tool_is_flattened_for_chat_upstream_and_restored_in_responses_output, test_legacy_functions_param_is_instructed_the_same_way, test_bedrock_passthrough_converse_stream_response_carries_event_stream_content_type, test_concurrent_tool_result_turns_across_endpoints_each_get_their_own_instructed_prompt, test_dropped_ollama_connections_fail_their_callers_and_the_rest_keep_their_prompts, test_a_non_function_json_answer_is_returned_as_text, test_a_second_function_call_after_the_result_is_surfaced_as_tool_calls, test_anthropic_sdk_tool_request_comes_back_as_tool_use, test_anthropic_sdk_tool_result_turn_ends_with_text, test_async_anthropic_sdk_stream_emits_the_tool_use_block, test_identical_uncached_tool_result_turns_are_each_forwarded_and_billed_once, test_list_system_message_keeps_one_system_section_with_the_instruction, test_ollama_chat_keeps_native_tools_and_gets_no_instruction, test_ollama_model_not_found_reaches_the_caller_after_one_attempt, test_ollama_server_error_on_the_tool_result_turn_does_not_take_the_deployment_down, test_openai_sdk_responses_function_output_turn_gets_a_message, test_openai_sdk_responses_tool_request_comes_back_as_a_function_call, test_openai_sdk_tool_request_reaches_ollama_as_an_instructed_prompt, test_openai_sdk_tool_result_turn_gets_a_plain_text_answer, test_string_system_message_keeps_one_system_section_with_the_instruction, test_the_cell_deployment_is_gone_after_its_scenario, test_two_tools_are_both_listed_under_one_instruction, test_unauthenticated_tool_request_never_reaches_ollama

One control went red once: test_worker_sigkill_mid_burst_leaves_the_sibling_instructing_ollama (its own two-worker proxy, 20 held calls) had one call answered 400 with 'CredentialLiteLLMParams' object has no attribute 'use_litellm_proxy' on the just-booted second worker, a pre-existing intermittent pydantic-field error on the provider-lookup path that this PR does not touch (the same cell was green on the same product code in the pre-validation lane, and the sibling GenericLiteLLMParams.provider_affinity_header shape is already tracked internally). Recorded as a Low caveat, not rerun

QA against a real Ollama (169c0c1)

Cases 1 to 4 run against the same real Ollama (ollama version is 0.40.0, model qwen3:0.6b). Each leg is one proxy process booted with --num_workers 2, no database, one deployment ollama-qwen -> ollama/qwen3:0.6b, drop_params: true. The before proxy runs the merge base 127278f951 on port 44170 and the after proxy runs 169c0c181b on port 22768. Both point at Ollama through a byte relay (127.0.0.1:21174 -> 127.0.0.1:11434) that appends every request body Ollama receives to a log, which is where the "prompt Ollama received" lines come from. Case 5 runs against the /live-pr-risk rig instead: the same two trees as two-worker proxies with Postgres (base on port 16754, after on 16755), with a second deployment haiku -> anthropic/claude-haiku-5-5 behind a request recorder in front of the real Anthropic API. Every call sends cache: {"no-cache": true}. Clients: the OpenAI Python SDK for /v1/chat/completions streams and /v1/responses, the Anthropic Python SDK for /v1/messages, and httpx.post for the raw chat posts. The 0.6b model sometimes answers a tool prompt with plain text instead of a tool call, so the /v1/responses first-turn calls were repeated until the model chose the tool on both legs; every other block is the leg's single run. A merge-ref check ran the rig cells once more on d6762d8c46, this branch merged into main at c38505d265, on port 31296: every cell matched the after side, and the OpenAI SDK responses.stream() call completed there too

Shared payloads:

TOOL = {"type": "function", "function": {"name": "get_weather", "description": "Weather for a city",
        "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}
RESPONSES_TOOL = {"type": "function", "name": "get_weather", "description": "Weather for a city", "parameters": TOOL["function"]["parameters"]}
ANTHROPIC_TOOL = {"name": "get_weather", "description": "Weather for a city", "input_schema": TOOL["function"]["parameters"]}
QUESTION = "What is the weather in Paris?"
def second_turn(result):
    return [{"role": "user", "content": QUESTION},
            {"role": "assistant", "content": None, "tool_calls": [{"id": "call_probe_1", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Paris\"}"}}]},
            {"role": "tool", "tool_call_id": "call_probe_1", "content": result}]
client = openai.OpenAI(base_url=BASE + "/v1", api_key=KEY, max_retries=0)
aclient = anthropic.Anthropic(base_url=BASE, api_key=KEY, max_retries=0)

Before (127278f)

One id per stream (/v1/chat/completions)

1. Run

   chunks = list(client.chat.completions.create(model="ollama-qwen", messages=[{"role": "user", "content": "Reply with the single word hello and nothing else."}], stream=True, extra_body={"cache": {"no-cache": True}}))
   print("chunks:", len(chunks), "distinct ids:", len({c.id for c in chunks}))
   for c in chunks: print(c.id, repr(c.choices[0].delta.content), c.choices[0].finish_reason)

2. Observed

   chunks: 77 distinct ids: 76
   chatcmpl-963f0429-b977-421a-ae4c-320499e53ffd None None
   chatcmpl-3b0c00db-79dc-48e5-815b-53c1e38c63ec None None
   ... (one new id per chunk)
   chatcmpl-b12043ec-4260-4197-818c-c575a1876190 None None
   chatcmpl-f34a7806-ba71-456f-8826-9a13c852f7dc 'hello' None
   chatcmpl-f34a7806-ba71-456f-8826-9a13c852f7dc None stop
   

Int tool content (/v1/chat/completions)

1. Run

   r = httpx.post(BASE + "/v1/chat/completions", headers={"Authorization": f"Bearer {KEY}"}, json={"model": "ollama-qwen", "messages": second_turn(22), "tools": [TOOL], "cache": {"no-cache": True}})
   print("status:", r.status_code); print(r.text)

2. Observed

   status: 500
   {"error":{"message":"litellm.APIConnectionError: can only concatenate str (not \"int\") to str","type":"internal_server_error","param":null,"code":"500"}}
   

Responses stream on a tool call (/v1/responses)

1. Run

   events = list(client.responses.create(model="ollama-qwen", input=QUESTION, tools=[RESPONSES_TOOL], store=False, stream=True, extra_body={"cache": {"no-cache": True}}))
   print("output_item.done items:", [(e.item.type, getattr(e.item, "name", None)) for e in events if e.type == "response.output_item.done"])
   completed = [e for e in events if e.type == "response.completed"][0]
   print("response.completed output:", [(item.type, getattr(item, "name", None)) for item in completed.response.output]); print("output_text:", repr(completed.response.output_text))

2. Observed

   output_item.done items: [('function_call', 'get_weather'), ('message', None)]
   response.completed output: [('reasoning', None), ('message', None), ('function_call', 'get_weather')]
   the message item carries ResponseOutputText(text='')
   output_text: ''
   

3. Control, the same call without stream=True

   resp = client.responses.create(model="ollama-qwen", input=QUESTION, tools=[RESPONSES_TOOL], store=False, extra_body={"cache": {"no-cache": True}})
   print("output:", [(item.type, getattr(item, "name", None)) for item in resp.output])

4. Observed

   output: [('reasoning', None), ('function_call', 'get_weather')]
   

Separated tool results, all three endpoints

1. Two tool messages over /v1/chat/completions

   messages = [*second_turn("Paris: 22 degrees Celsius"), {"role": "tool", "tool_call_id": "call_probe_1", "content": "Sky: clear"}]
   r = httpx.post(BASE + "/v1/chat/completions", headers={"Authorization": f"Bearer {KEY}"}, json={"model": "ollama-qwen", "messages": messages, "tools": [TOOL], "cache": {"no-cache": True}})

2. Observed

   status: 200
   prompt Ollama received (tail): Paris: 22 degrees CelsiusSky: clear\n\n###
   

3. One tool message with two text parts over /v1/chat/completions

   messages = second_turn([{"type": "text", "text": "Paris: 22 degrees"}, {"type": "text", "text": "clear skies"}])
   r = httpx.post(BASE + "/v1/chat/completions", headers={"Authorization": f"Bearer {KEY}"}, json={"model": "ollama-qwen", "messages": messages, "tools": [TOOL], "cache": {"no-cache": True}})

4. Observed

   status: 200
   prompt Ollama received (tail): Paris: 22 degreesclear skies\n\n###
   

5. Two tool_result blocks over /v1/messages

   resp = aclient.messages.create(model="ollama-qwen", max_tokens=256, tools=[ANTHROPIC_TOOL], extra_body={"cache": {"no-cache": True}}, messages=[
       {"role": "user", "content": QUESTION},
       {"role": "assistant", "content": [{"type": "tool_use", "id": "toolu_1", "name": "get_weather", "input": {"city": "Paris"}}, {"type": "tool_use", "id": "toolu_2", "name": "get_weather", "input": {"city": "Paris"}}]},
       {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_1", "content": "Paris: 22 degrees Celsius"}, {"type": "tool_result", "tool_use_id": "toolu_2", "content": "Sky: clear"}]}])
   print("stop_reason:", resp.stop_reason)

6. Observed

   stop_reason: tool_use
   prompt Ollama received (tail): Paris: 22 degrees CelsiusSky: clear\n\n###
   

7. Two function_call_output items over /v1/responses

   resp = client.responses.create(model="ollama-qwen", tools=[RESPONSES_TOOL], store=False, extra_body={"cache": {"no-cache": True}}, input=[
       {"role": "user", "content": QUESTION},
       {"type": "function_call", "call_id": "call_a", "name": "get_weather", "arguments": "{\"city\": \"Paris\"}"},
       {"type": "function_call", "call_id": "call_b", "name": "get_weather", "arguments": "{\"city\": \"Paris\"}"},
       {"type": "function_call_output", "call_id": "call_a", "output": "Paris: 22 degrees Celsius"},
       {"type": "function_call_output", "call_id": "call_b", "output": "Sky: clear"}])
   print("output:", [(item.type, getattr(item, "name", None)) for item in resp.output])

8. Observed

   output: [('reasoning', None), ('function_call', 'get_weather')]
   prompt Ollama received (tail): Paris: 22 degrees CelsiusSky: clear\n\n###
   

OpenAI SDK responses.stream() on a bridged tool call (/v1/responses)

1. Run, against the rig proxy with the haiku deployment

   with client.responses.stream(model="haiku", input="What is the weather in Paris? Use the tool.", tools=[RESPONSES_TOOL], tool_choice="required") as s:
       items = [(e.type.split(".")[-1], e.item.type, e.output_index) for e in s if e.type in ("response.output_item.added", "response.output_item.done")]
       final = s.get_final_response()
   print("item events:", items); print("final output:", [(o.type, getattr(o, "name", None)) for o in final.output])

2. Observed

   IndexError: list index out of range
             ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
   IndexError: list index out of range
   

After (169c0c1)

One id per stream (/v1/chat/completions)

1. Same call

2. Observed

   chunks: 123 distinct ids: 1
   chatcmpl-aa3c71a7-e72f-4545-a33f-ebd19c8c5fda None None
   chatcmpl-aa3c71a7-e72f-4545-a33f-ebd19c8c5fda None None
   ... (the same id on every chunk)
   chatcmpl-aa3c71a7-e72f-4545-a33f-ebd19c8c5fda None None
   chatcmpl-aa3c71a7-e72f-4545-a33f-ebd19c8c5fda 'hello' None
   chatcmpl-aa3c71a7-e72f-4545-a33f-ebd19c8c5fda None stop
   

Int tool content (/v1/chat/completions)

1. Same post

2. Observed

   status: 400
   {"error":{"message":"litellm.BadRequestError: Invalid Message the tool message at index 2 has int content; content must be a string or a list of content parts\n\nLiteLLM: model group 'ollama-qwen' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
   

Responses stream on a tool call (/v1/responses)

1. Same stream

2. Observed

   output_item.done items: [('reasoning', None), ('function_call', 'get_weather')]
   response.completed output: [('reasoning', None), ('function_call', 'get_weather')]
   output_text: ''
   

3. Same control without stream=True

4. Observed

   output: [('reasoning', None), ('function_call', 'get_weather')]
   

Separated tool results, all three endpoints

1. Same two tool messages over /v1/chat/completions

2. Observed

   status: 200
   prompt Ollama received (tail): Paris: 22 degrees Celsius\nSky: clear\n\n###
   

3. Same two text parts over /v1/chat/completions

4. Observed

   status: 200
   prompt Ollama received (tail): Paris: 22 degrees\nclear skies\n\n###
   

5. Same two tool_result blocks over /v1/messages

6. Observed

   stop_reason: tool_use
   prompt Ollama received (tail): Paris: 22 degrees Celsius\nSky: clear\n\n###
   

7. Same two function_call_output items over /v1/responses

8. Observed

   output: [('reasoning', None), ('function_call', 'get_weather')]
   prompt Ollama received (tail): Paris: 22 degrees Celsius\nSky: clear\n\n###
   

OpenAI SDK responses.stream() on a bridged tool call (/v1/responses)

1. Same call

2. Observed

   item events: [('added', 'function_call', 0), ('done', 'function_call', 0)]
   final output: [('function_call', 'get_weather')]
   

Type

🐛 Bug Fix

Caveats (if any)

Low

• A user or tool message whose content is neither a string nor a list (false, 0, an object), a content part that is not an object, a text or image_url part with no value, and an image_url object without a url string answer 400 now, naming the message index and what the message carried. Before, the int, missing-value, and non-object shapes answered 500, and false, 0, {} content and a null or int image_url answered 200 from a prompt that silently left that message out, so that 200 was a request that wrongly succeeded and now fails as designed. The index counts litellm's message list, so a system message litellm injects shifts it by one. On a proxy with a database, the enterprise managed-files pre-call hook reads every user message's content as a list before translation, so a non-list user content is rejected there first on the merge base and on this branch alike (500 for an int, 400 with a traceback for an object or a bare-string part); that hook is pre-existing and tracked separately, and this PR's user-content wording is reached only on a proxy without a database, while the tool-message and content-part shapes reach it on both. The audit's cells pin what both sides share (rejected, never forwarded to Ollama, the deployment keeps serving) and tighten to this PR's wording once the hook is fixed
• A tool-only /v1/responses stream through the bridge now announces its function_call at output_index 0 where it used to sit at 1 with nothing at 0; a client that hardcoded 1 reads the wrong slot
• A bridged stream whose answer is empty text after reasoning (max_output_tokens spent inside the reasoning) lists only the reasoning item in response.completed, since no message item was ever announced; the non-stream call still lists an empty message item after the reasoning
• An empty text part adds no blank line to the prompt, matching the old concatenation
• A content part of a type the ollama/ prompt path cannot render (input_audio, file, a custom type) still drops silently, as it did on main; rejecting it is a wider contract change than this ticket's malformed-value 400s, so it stays out of this PR
• A legacy functions request still answers tool_calls, as every non-OpenAI provider does; left as is on purpose
• Consecutive assistant turns are still joined with no separator; the ticket's scope was the user and tool side
• A reasoning delta that arrives after a tool call in the same bridged stream still gets no output_item.added for its reasoning item, on main and on this branch alike; filed as its own ticket, not fixed here
• The /v1/completions stream's finish chunk carries text: null where OpenAI sends an empty string, on main and on this branch alike, in the shared text-completion stream wrapper; tracked separately, not fixed here
• A /v1/responses function_call_output whose output is a list of input_text parts is joined with no separator by the bridge one layer above the Ollama prompt builder, on main and on this branch alike; this PR separates string outputs and tracked the list shape separately
• One audit control that boots its own two-worker proxy saw one call of twenty answered 400 with 'CredentialLiteLLMParams' object has no attribute 'use_litellm_proxy' on the just-booted second worker, a pre-existing intermittent error on the provider-lookup path this PR does not touch (the same cell was green at the same product code in the pre-validation lane); tracked separately
• Three unit-test shards that are not required checks went red at c7ac238 (a tests-only commit): misc and proxy-infra-root on the new /moyai/connect/* routes and moyai_endpoints.py that main's feat(proxy,ui): add Moyai to the view switcher with quick connect #45196 added without their allowlist and key-family entries, red on main's own run at 0e07600 and green at this PR's previous tip, and Vertex AI on test_path_source_peak_stays_below_list_pipeline, a tracemalloc ratio assertion whose own docstring says the ratio drifts with GC timing and the worker's live set, unrelated to the diff, green at the previous tip, green on main's last eight runs, and green when that one job was rerun at the same merge commit; none is caused by this PR, and the required checks are green at the tip

/live-pr-risk

Verdict: no dependent path that worked at the merge base fails at the tip. The contract changes below are the fixes themselves plus three collateral shifts, each decided here. The one rig cell that differed outside those, the deleted-key probe, answered 200 on one merge-base run and 401 on the next with no code change on that side (two workers, one in-memory key cache each), so it is rig noise, not a finding

Breaking

• None observed. Every cell that answered 200 at the merge base and 400 at the tip (false, 0, {} content; null or int image_url) was served at the merge base from a prompt missing that message, and the ticket asks for a 400 naming the field for non-text content; the falsy and placeholder shapes ride the same rule, decided in this PR by its author (Caveats, first row)

Backward incompatible

• ollama/ streams on /v1/chat/completions and /v1/completions carry one id per stream instead of one per chunk; the spend row's request_id and the logs page row key off that one id now (the fix itself)
• Malformed ollama/ prompts answer 400 instead of 500, so the router no longer retries or falls back on them (the fix itself)
• Bridged /v1/responses streams number output items from 0 in emission order and response.completed lists the announced items in that order; a tool-only stream moves its function_call from index 1 to 0 and loses the empty message item (the fix itself, Caveats rows two and three)
• Merged user, tool, and function text in ollama/ prompts is newline-separated (the fix itself)

Regression risk

• Providers whose streams open with a content: "" role chunk other than Ollama and Anthropic (Bedrock Converse, Gemini, Groq) go through the same deferred message item; not driven here, covered by the unit tests' synthetic chunks only
• The guardrail text rewrite on /v1/responses streams keys on (output_index, content_index); it was read, not driven, and the indexes it sees now start at 0 for every stream
• /v1/responses with previous_response_id replaying a tool-only turn from the spend log; not driven
• /v1/messages with use_chat_completions_api: true on a tool call stream; not driven

Dependency graph

Format: dependent path, entrypoint, how it reaches the change, status

Ollama prompt builder, reached by every ollama/ deployment on /v1/chat/completions, /v1/completions through the chat adapter, /v1/messages, and /v1/responses when the tool request falls back to the prompt path:

• user, tool, and function turn merging and text-part joining: verified live on all three unified endpoints (two tool messages, one tool message with two text parts, two tool_result blocks, two function_call_output items), tested in test_litellm_core_utils_prompt_templates_factory.py and tests/integration/providers/test_ollama_prompt_tools_wire.py
• malformed content and content parts (non-string tool content, non-list user content, a part with no value, a non-object part, an image_url object without a url): verified live on /v1/chat/completions and /v1/responses through the hostile cells on the no-database QA proxies; on the audit's database-backed rigs the enterprise managed-files pre-call hook rejected non-list user content first on both legs (Caveats, first row) while the tool-content and content-part shapes reached this PR's 400; tested in the factory unit tests and the wire test's malformed-content cells
• custom_prompt_dict path for ollama/ deployments with a registered template: untouched by the diff, not driven

Streamed chat chunk id, minted once per ollama/ stream by the text completion response iterator and adopted by the stream wrapper:

• /v1/chat/completions and /v1/completions streams: verified live (one id per stream on both)
• the spend row's request_id and the logs page row keyed by that id: verified live through the spend row of every streamed cell

Responses bridge streaming iterator (every /v1/responses stream on a model without a native Responses API, and /v1/messages with use_chat_completions_api: true):

• deferred message item until a delta carries output: verified live on Ollama and Anthropic tool streams; tested in test_streaming_iterator_transformation.py
• contiguous output_index in emission order and response.completed listing the announced items in streamed order: verified live with the OpenAI Python SDK's client.responses.stream() on an Anthropic tool_choice: required stream (base raises IndexError in the SDK accumulator, tip completes) and on Ollama text and tool streams; tested in test_streaming_iterator_transformation.py
• the completed snapshot dropping a message item the stream never announced: verified live on the Ollama tool stream and on an Ollama reasoning-only stream; tested in test_streaming_iterator_transformation.py
• reasoning-first streams (reasoning item at 0, tool at 1, message at 2): tested; the live Ollama tool stream carries a reasoning item first and completed in order
• consumers of output_index: the OpenAI SDK accumulator (verified live), the guardrail text rewrite keyed by (output_index, content_index) on /v1/responses streams, background response polling, the Anthropic adapter's content_block_start ordering: read, not driven
• a reasoning delta arriving after a tool call in one stream still streams with no output_item.added for the reasoning item, on base and tip alike; pre-existing, filed as its own ticket, not fixed here

Spend logging, logging integrations, caching, and guardrail post-call hooks: the rebuilt chat message is unchanged by this PR (an earlier revision changed it and was reverted at 40571a5), so none of them see a difference; spend rows compared equal on every non-hostile cell

Not verified

• Bedrock Converse, Gemini, Groq, and other providers' own stream frames: the deferred message item was driven on Ollama, Anthropic, and the audit's scripted OpenAI-compatible stream that opens with a content: "" role chunk, not on those providers' frames
• /v1/responses with previous_response_id replaying a tool-only turn from the spend log
• A refusal-only stream, which carries no text delta and whose message item opens on the finish chunk
• /v1/messages with use_chat_completions_api: true on a tool call stream
• The guardrail text rewrite keyed by output_index on a /v1/responses stream
• A reasoning delta after a tool call in one stream (pre-existing, see the graph)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
• 169c0c1 passes /live-pr-risk
• c7ac238 passes /audit

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Fixes Ollama streaming response handling and tool result formatting.

The PR appears safe to merge, with no new blocking issue found

What we checked:

• Tests stay on local providers: The tests override each model's api_base with a scripted server bound to 127.0.0.1. The provider keys are synthetic

Summary

The PR gives Ollama streams one response ID, rejects malformed prompt content with a 400, separates merged text with newlines, and keeps Responses output items in streamed order

• Changes since the last review only add integration tests. They cover SDK streams, raw frames, cache replay, malformed inputs, concurrent requests, and spend logs
• The earlier type and Final findings are fixed. Missing text and image_url values now raise BadRequestError
• mateo-berri disputed moving the prompt helpers because they serve the shared template and moving them would create a cycle. Greptile withdrew that finding
• mateo-berri accepted newline joining as intentional because the old behavior fused words. Greptile withdrew the compatibility finding
• mateo-berri explicitly documents the malformed-input 400s and reasoning-only output change as intentional. Unsupported content types, joined assistant turns, legacy functions behavior, and late reasoning remain outside this fix
• No new actionable issue found. Tests were not run during this review

_{Reviews (6) · Last reviewed commit: "test(integration): add ollama prompt-too..." · Reviewed by Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codecov]

Codecov Report

❌ Patch coverage is 96.55172% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...llm/litellm_core_utils/prompt_templates/factory.py	93.47%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_ollama_stream_id_tool_result_fixes (c7ac238) with main (0e07600)}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Finish reason opens empty message
  • Removed the finish_reason exception in _ensure_output_item_for_chunk so a payload-less terminal chunk no longer opens an empty message beside assembled function_call items.

Or push these changes by commenting:

@cursor push a27fc1e91a

Preview (a27fc1e91a)

diff --git a/litellm/responses/litellm_completion_transformation/streaming_iterator.py b/litellm/responses/litellm_completion_transformation/streaming_iterator.py
--- a/litellm/responses/litellm_completion_transformation/streaming_iterator.py
+++ b/litellm/responses/litellm_completion_transformation/streaming_iterator.py
@@ -976,7 +976,7 @@
         if not chunk.choices:
             return
         delta: Final = chunk.choices[0].delta
-        if chunk.choices[0].finish_reason is None and not _delta_carries_output(delta):
+        if not _delta_carries_output(delta):
             return
 
         self._sequence_number += 1

diff --git a/tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.py b/tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.py
--- a/tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.py
+++ b/tests/unit/responses/litellm_completion_transformation/test_streaming_iterator_transformation.py
@@ -1082,6 +1082,55 @@
 
 @pytest.mark.parametrize("sync_mode", [True, False])
 @pytest.mark.asyncio
+async def test_finish_reason_without_payload_does_not_open_a_message_beside_assembled_tool_calls(
+    sync_mode: bool,
+):
+    iterator: Final = _build_iterator([_chunk(""), _chunk("", finish_reason="tool_calls")])
+    iterator.litellm_model_response = ModelResponse(
+        id="resp-assembled",
+        created=1748575031,
+        model="claude-haiku-4-5",
+        object="chat.completion",
+        choices=[
+            {
+                "index": 0,
+                "finish_reason": "tool_calls",
+                "message": {
+                    "role": "assistant",
+                    "content": None,
+                    "tool_calls": [
+                        {
+                            "id": "call_assembled",
+                            "type": "function",
+                            "function": {"name": "run_command", "arguments": '{"command":"pwd"}'},
+                            "index": 0,
+                        }
+                    ],
+                },
+            }
+        ],
+    )
+
+    events: Final = await _collect_events(iterator, sync_mode)
+
+    item_events: Final = [
+        (event.type, event.item.type)
+        for event in events
+        if getattr(event, "type", None)
+        in (ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED, ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE)
+    ]
+    assert item_events == [
+        (ResponsesAPIStreamEvents.OUTPUT_ITEM_ADDED, "function_call"),
+        (ResponsesAPIStreamEvents.OUTPUT_ITEM_DONE, "function_call"),
+    ]
+    completed: Final = [
+        event for event in events if getattr(event, "type", None) == ResponsesAPIStreamEvents.RESPONSE_COMPLETED
+    ]
+    assert [item.type for item in completed[0].response.output] == ["function_call"]
+
+
+@pytest.mark.parametrize("sync_mode", [True, False])
+@pytest.mark.asyncio
 async def test_empty_leading_delta_still_opens_the_message_item_for_the_first_text_delta(sync_mode: bool):
     iterator: Final = _build_iterator([_chunk(""), _chunk("Hi"), _chunk("", finish_reason="stop")])

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c7ac238. Configure here.}