fix: route streamed tool-call args by tool_calls index by tonycoder-hub · Pull Request #1305 · mozilla-ai/any-llm

tonycoder-hub · 2026-08-17T11:01:30Z

Description

chat_completion_chunk_to_message_stream_events attached every streamed tool-call argument fragment to state.current_block_index instead of the block for that tool_calls[].index. Parallel interleaved calls mixed arguments (one tool got unparseable JSON, the other got none).

StreamingState now maps each tool_calls index to its content block. tool_use blocks stay open through the tool section and stop together. Repeated ids reuse the same block.

Tests: uv run pytest tests/unit — 2160 passed, 69 skipped. Distinct from #1296-#1304.

PR Type

Bug Fix

Relevant issues

No open issue.

Checklist

I understand the code I am submitting.
I have added unit tests that prove my fix/feature works
I have run this code locally and verified it fixes the issue.
New and existing tests pass locally
Documentation was updated where necessary
I have read and followed the contribution guidelines
AI Usage:
- This is fully AI-generated.

AI Usage Information

AI Model used: Claude Opus 5
AI Developer Tool used: Cursor cloud agent
I am an AI Agent filling out this form (check box if true)

The Anthropic Messages bridge attached every streamed tool-call argument fragment to state.current_block_index, so a provider that announces all parallel tool calls before streaming their arguments filed each fragment under whichever tool_use block opened last. One call ended up with the other's arguments concatenated onto its input and the rest received none. Track the content block opened for each OpenAI tool_calls[].index and route fragments through that map. Because parallel calls can interleave, tool_use blocks now stay open for the whole tool section and are stopped together when the section ends, instead of being closed as soon as the next call starts (which would put deltas after their block's content_block_stop). Reusing the map also stops providers that resend the id on every fragment from opening a duplicate block per fragment. Co-authored-by: Tony Coder <407243179@qq.com>

coderabbitai · 2026-08-17T11:01:51Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ab5b1f82-3ba4-4820-a422-3cfd192f6cf3

📥 Commits

Reviewing files that changed from the base of the PR and between 56d1e0c and 95a0987.

📒 Files selected for processing (3)

src/any_llm/any_llm.py
src/any_llm/utils/messages_compat.py
tests/unit/test_messages_compat.py

Included review availability: Your plan includes up to 8 reviews per rolling hour; 2 remain after this review.

Walkthrough

Changes

Parallel tool-call streaming

Layer / File(s)	Summary
Index-based tool-call routing `src/any_llm/utils/messages_compat.py`, `tests/unit/test_messages_compat.py`	Streaming state maps OpenAI tool-call indexes to content blocks. Interleaved fragments, repeated fragments, and unknown indexes use the defined routing behaviour.
Open-block closure integration `src/any_llm/utils/messages_compat.py`, `src/any_llm/any_llm.py`, `tests/unit/test_messages_compat.py`	`close_open_blocks` stops all active tool-use blocks. `AnyLLM._amessages` emits each stop event after streaming, with tests for completion and idempotent closure.

Possibly related PRs

mozilla-ai/any-llm#1151: Both changes update AnyLLM._amessages and streaming block-closure behaviour.
mozilla-ai/any-llm#1224: Both changes update messages_compat.py and streaming message handling.

Suggested labels: 1.21.0

Suggested reviewers: njbrake

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarises the main fix: routing streamed tool-call arguments by their tool_calls index.
Description check	✅ Passed	The description explains the bug, the implementation, the tests, the PR type, checklist status, and AI usage information.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

codecov · 2026-08-18T09:47:59Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
src/any_llm/any_llm.py	`80.38% <100.00%> (-1.15%)`	⬇️
src/any_llm/utils/messages_compat.py	`100.00% <100.00%> (ø)`

... and 34 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Brings in upstream's merges of our mozilla-ai#1291/mozilla-ai#1292/mozilla-ai#1310 plus mozilla-ai#1297, mozilla-ai#1299, mozilla-ai#1301, mozilla-ai#1302, mozilla-ai#1303, mozilla-ai#1305. Carried-until-merged fork work stays: mozilla-ai#1294 (gemini reasoning_effort=none), mozilla-ai#1308 (aresponses timeout), mozilla-ai#1309 (gemini native tool dicts), and the mozilla-ai#1300 carry. One conflict in tests/unit/test_responses.py: kept our mozilla-ai#1308 timeout test next to upstream's flatten test. Unit suite: 2234 passed. Claude-Session: https://claude.ai/code/session_018D3FGNvb1hRZQmsXFoA44J

github-actions Bot added the missing-template PR is missing required template checklist label Aug 17, 2026

tonycoder-hub mentioned this pull request Aug 17, 2026

fix: map Anthropic refusal and context-window stop reasons #1306

Closed

9 tasks

github-actions Bot removed the missing-template PR is missing required template checklist label Aug 17, 2026

tonycoder-hub temporarily deployed to integration-tests August 18, 2026 09:46 — with GitHub Actions Inactive

tbille approved these changes Aug 18, 2026

View reviewed changes

tbille merged commit d97a155 into mozilla-ai:main Aug 18, 2026
19 of 20 checks passed

github-actions Bot added the 1.27.0 Included in release 1.27.0 label Sep 3, 2026

Navigation Menu

fix: route streamed tool-call args by tool_calls index - #1305

fix: route streamed tool-call args by tool_calls index#1305
tbille merged 1 commit into
mozilla-ai:mainfrom
tonycoder-hub:cursor/fix-messages-bridge-parallel-tool-call-index-8611

tonycoder-hub commented Aug 17, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 17, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Aug 18, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tonycoder-hub commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

PR Type

Relevant issues

Checklist

AI Usage Information

Uh oh!

coderabbitai Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

codecov Bot commented Aug 18, 2026

Codecov Report

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tonycoder-hub commented Aug 17, 2026 •

edited

Loading

coderabbitai Bot commented Aug 17, 2026 •

edited

Loading