fix(responses): preserve tool call argument deltas when streaming id is omitted

[emerzon]

Relevant issues

Fixes #20711

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🐛 Bug Fix
✅ Test

Changes

• Add tool-call index resolution in LiteLLMCompletionStreamingIterator so chunks with id=None can reuse a previously seen call_id from the same index.
• Preserve existing behavior when id is present, while populating index -> call_id mapping for later chunks.
• Keep skip behavior for chunks that have neither a resolvable id nor a mapped index.
• Add regression test for OpenAI-style streaming where only the first chunk has id and subsequent chunks only carry index.
• Add regression test for parallel tool calls to ensure per-index routing remains correct with id=None deltas.

Validation

• Ran:
  • poetry run pytest tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py
• Result: 5 passed

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 8, 2026 2:54pm

[greptile-apps]

Greptile Overview

Greptile Summary

This PR updates LiteLLMCompletionStreamingIterator to preserve tool-call argument deltas when streaming chunks omit the tool-call id by caching an index -> call_id mapping and using it to route later id=None deltas. It also adds regression tests covering OpenAI-style streaming where only the first delta includes an id, and a parallel tool-call scenario to ensure per-index routing stays correct.

Confidence Score: 4/5

• This PR is close to safe to merge, but the new index→call_id cache can misroute deltas if a provider reuses indices across distinct tool calls.
• Core change is small and tests cover the reported regression (missing id with stable index), but correctness depends on provider behavior: if indices are reused or tool calls overlap, the mapping overwrite can attribute id-less deltas to the wrong call_id.
• litellm/responses/litellm_completion_transformation/streaming_iterator.py

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Adds index -> call_id memoization so tool-call argument deltas with missing id can still be routed; main behavior preserved but mapping can be overwritten if providers reuse an index for multiple tool calls.
tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py	Adds regression tests covering streaming tool-call deltas where only the first chunk has id and later chunks rely on index, including parallel tool call routing.

Sequence Diagram

sequenceDiagram
    participant Provider as LLM Provider (chat.completion.chunk)
    participant It as LiteLLMCompletionStreamingIterator
    participant Q as _pending_tool_events
    participant Client as Responses API stream consumer

    Provider->>It: chunk delta.tool_calls[{index=0,id=call_abc,...args='{"lo'}]
    It->>It: _normalize_tool_call_index() => 0
    It->>It: _tool_call_id_by_index[0] = 'call_abc'
    It->>It: _get_or_assign_tool_output_index('call_abc')
    It->>Q: enqueue output_item.added(item.id='call_abc')
    It->>Q: enqueue function_call_arguments.delta(delta='{"lo'...)

    Provider->>It: chunk delta.tool_calls[{index=0,id=null,args='cation":'}]
    It->>It: resolve call_id via _tool_call_id_by_index[0]
    It->>Q: enqueue function_call_arguments.delta(delta='cation":'...)

    Client->>It: next() / drain events
    It-->>Client: output_item.added
    It-->>Client: function_call_arguments.delta (multiple)

    Note over It: If index is reused for a different call_id later,
    Note over It: _tool_call_id_by_index can be overwritten and misroute id=null deltas.

Loading

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

litellm/responses/litellm_completion_transformation/streaming_iterator.py
Index mapping can misroute
If a provider reuses the same index for a later, distinct tool call (i.e., new id at the same index), the mapping will be overwritten and subsequent id=None deltas would be attributed to the new call_id even if they belong to the older call. This breaks argument reconstruction for interleaved/overlapping calls that share an index; consider resetting/guarding the mapping when a call is marked done or when a new call starts at the same index.

[emerzon]

Follow-up update for the Greptile edge case is now pushed.

What changed:

• Added an ambiguity guard in LiteLLMCompletionStreamingIterator.
• If the same tool-call index is later seen with a different explicit call_id, that index is marked ambiguous for id-less fallback.
• For ambiguous indexes, id=None deltas are skipped instead of being potentially misattributed.
• Explicit id chunks still stream normally.

Regression coverage:

• Added test_reused_index_with_new_call_id_marks_fallback_ambiguous in tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py.

Validation:

• poetry run pytest tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py
• Result: 6 passed

Commit: cf17a440cd

[Copilot]

Pull request overview

This PR fixes a Responses API streaming bug where tool-call argument deltas were dropped after the first chunk when subsequent chunks omit id and only provide index (OpenAI-style tool streaming). It adds an index -> call_id resolution mechanism so later id=None deltas are correctly attributed and streamed incrementally.

Changes:

• Track tool_call.index -> tool_call.id during streaming and use it to route later id=None tool-call deltas.
• Keep existing skip behavior when neither id nor a previously mapped index is available.
• Add regression tests for single and parallel tool-call streaming where only the first chunk includes id.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Adds index normalization + per-index call-id mapping to preserve tool argument delta streaming when id is omitted.
tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py	Adds regression coverage for id=None tool-call deltas (single + parallel tool calls) using index-based routing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[emerzon]

@greptile

[greptile-apps]

Greptile Overview

Greptile Summary

Preserves tool call argument deltas when streaming chunks omit the id field by introducing an index-to-call_id mapping mechanism. When a chunk includes an id, it's stored with its index for future reference. Subsequent chunks with the same index but no id can now resolve the correct call_id from the mapping, ensuring argument deltas are properly routed to the correct tool call.

Key improvements:

• Added _tool_call_id_by_index dict to map index → call_id for id-less chunks
• Added _ambiguous_tool_call_indexes set to track when an index is reused with different call_ids, preventing silent misrouting
• Created _normalize_tool_call_index() helper to safely extract and parse index values
• Modified _queue_tool_call_delta_events() to populate and consult the mapping when id is absent
• Added comprehensive test coverage for single tool calls, parallel tool calls, and ambiguous index reuse scenarios

This fix enables LiteLLM to handle OpenAI-style streaming where only the first chunk contains the tool call id, while subsequent deltas rely solely on index for routing.

Confidence Score: 5/5

• Safe to merge - well-tested bug fix with defensive programming against edge cases
• The implementation is clean, focused, and includes excellent test coverage. The code adds defensive logic to detect and guard against ambiguous index reuse, preventing silent misrouting. The changes are isolated to tool call streaming logic with no impact on the critical request path. All three test cases validate the core scenarios (single tool, parallel tools, ambiguous reuse), and the PR description confirms tests pass.
• No files require special attention

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Adds index-to-call_id mapping for streaming tool calls, enabling proper routing when id is omitted from subsequent chunks while guarding against ambiguous index reuse
tests/test_litellm/responses/litellm_completion_transformation/test_tool_call_streaming_transformation.py	Comprehensive tests for id-less tool call deltas: single tool with index fallback, parallel tool calls routing, and ambiguous index reuse detection

Sequence Diagram

sequenceDiagram
    participant Provider as LLM Provider
    participant Iterator as LiteLLMCompletionStreamingIterator
    participant Mapping as _tool_call_id_by_index
    participant Queue as _pending_tool_events
    participant Client as API Client

    Note over Provider,Client: Streaming Tool Call with id in first chunk

    Provider->>Iterator: chunk[0] {index:0, id:"call_abc", args:'{"lo'}
    Iterator->>Iterator: _normalize_tool_call_index(tc) → 0
    Iterator->>Iterator: Extract call_id = "call_abc"
    Iterator->>Mapping: Store mapping[0] = "call_abc"
    Iterator->>Queue: Queue OUTPUT_ITEM_ADDED event
    Iterator->>Queue: Queue FUNCTION_CALL_ARGUMENTS_DELTA
    Iterator->>Client: Emit queued events

    Note over Provider,Client: Subsequent chunks without id

    Provider->>Iterator: chunk[1] {index:0, args:'cation":'}
    Iterator->>Iterator: _normalize_tool_call_index(tc) → 0
    Iterator->>Iterator: call_id_raw is None/empty
    Iterator->>Mapping: Lookup mapping[0] → "call_abc"
    Iterator->>Iterator: Use mapped call_id = "call_abc"
    Iterator->>Queue: Queue FUNCTION_CALL_ARGUMENTS_DELTA for call_abc
    Iterator->>Client: Emit delta event

    Provider->>Iterator: chunk[2] {index:0, args:' "New'}
    Iterator->>Mapping: Lookup mapping[0] → "call_abc"
    Iterator->>Queue: Queue FUNCTION_CALL_ARGUMENTS_DELTA for call_abc
    Iterator->>Client: Emit delta event

    Note over Provider,Client: Parallel tool calls scenario

    Provider->>Iterator: chunk[0] {index:0, id:"call_a", args:'{"x":'}
    Provider->>Iterator: chunk[0] {index:1, id:"call_b", args:'{"y":'}
    Iterator->>Mapping: Store mapping[0] = "call_a"
    Iterator->>Mapping: Store mapping[1] = "call_b"
    Iterator->>Client: Emit OUTPUT_ITEM_ADDED for both

    Provider->>Iterator: chunk[1] {index:0, args:'1}'}, {index:1, args:'2}'}
    Iterator->>Mapping: Lookup mapping[0] → "call_a"
    Iterator->>Mapping: Lookup mapping[1] → "call_b"
    Iterator->>Queue: Queue deltas for call_a and call_b
    Iterator->>Client: Emit deltas correctly routed

    Note over Provider,Client: Ambiguous index reuse detection

    Provider->>Iterator: chunk[0] {index:0, id:"call_a"}
    Iterator->>Mapping: Store mapping[0] = "call_a"
    
    Provider->>Iterator: chunk[1] {index:0, id:"call_b"}
    Iterator->>Mapping: Existing mapping[0] = "call_a" != "call_b"
    Iterator->>Iterator: Mark index 0 as ambiguous
    Iterator->>Mapping: Update mapping[0] = "call_b"
    
    Provider->>Iterator: chunk[2] {index:0, NO id}
    Iterator->>Iterator: Check if index 0 is ambiguous
    Iterator->>Iterator: Skip fallback - prevent misrouting
    Note over Iterator: Delta is dropped to avoid<br/>routing to wrong call_id

Loading

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[Sameerlite]

LGTM