fix(providers): keep the assistant text when the turn also calls a tool - #1325
Merged
javiermtorres merged 3 commits intoSep 1, 2026
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. WalkthroughAnthropic conversion now preserves assistant text before tool-use blocks. Gemini conversion preserves text, function calls, and thought signatures in messages and non-streaming responses. Tests cover ordering, empty content, signature validation, finish reasons, and round-tripping. ChangesAssistant content preservation
Suggested reviewers: Merge Risk: ⚪ Minimal · up to The change preserves assistant text when a turn also includes tool calls without changing tool authority or deployment behavior. No actionable merge-blocking risk remains after normal checks and review. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Description checkExplanation The description follows the repository template. It explains the bug and implementation, identifies the bug-fix type, references the related issue, records completed checklist items, and documents AI usage. Optional AI information fields are left blank, which does not prevent approval. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
11 tasks
JamMaster1999
temporarily deployed
to
integration-tests
August 24, 2026 14:05 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
javiermtorres
self-requested a review
August 24, 2026 14:05
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
javiermtorres
force-pushed
the
fix/replay-assistant-text-with-tool-calls
branch
from
August 25, 2026 13:52
238a40a to
dcec609
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Preserves assistant text when Gemini or Anthropic responses also invoke tools.
Changes:
- Retains Gemini text alongside function calls and thought signatures.
- Replays Anthropic text before tool-use blocks.
- Adds regression tests for mixed text and tool-call turns.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
src/any_llm/providers/gemini/utils.py |
Preserves and replays mixed text and function calls. |
src/any_llm/providers/anthropic/utils.py |
Adds assistant text to tool-call turns. |
tests/unit/providers/test_gemini_provider.py |
Tests mixed turns, signatures, and round trips. |
tests/unit/providers/test_anthropic_provider.py |
Tests text ordering and empty content handling. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+158
to
+159
| if isinstance(content, str) and content: | ||
| content_blocks.append({"type": "text", "text": content}) |
Contributor
There was a problem hiding this comment.
@JamMaster1999 please take a look at this suggestion.
Contributor
Author
There was a problem hiding this comment.
@javiermtorres added test_convert_messages_keeps_text_when_tool_calls_is_empty in c223097. An assistant turn with text and tool_calls: [] now converts to a single text block; against main the same input produces content: [], so the test fails there and passes here.
javiermtorres
temporarily deployed
to
integration-tests
August 25, 2026 15:08 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
javiermtorres
force-pushed
the
fix/replay-assistant-text-with-tool-calls
branch
from
August 26, 2026 15:17
dcec609 to
f8bd6f8
Compare
JamMaster1999
temporarily deployed
to
integration-tests
August 31, 2026 10:10 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
The non-streaming converter set content to None whenever the response carried a function call, and the message converter rebuilt a tool-call turn from tool_calls alone. Both halves now keep the text: the response reports content and tool_calls together (as the streaming path already did), and the replay emits a non-empty text as its own part ahead of the function-call parts, carrying the message-level thought_signature.
…ol_use blocks The tool-call branch of _convert_messages_for_anthropic rebuilt the turn from the thinking block and the tool_use blocks and never read content, so the text Claude wrote before calling a tool vanished from history.
javiermtorres
force-pushed
the
fix/replay-assistant-text-with-tool-calls
branch
from
September 1, 2026 07:34
c223097 to
c44bbda
Compare
javiermtorres
temporarily deployed
to
integration-tests
September 1, 2026 07:34 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
javiermtorres
approved these changes
Sep 1, 2026
11 tasks
JamMaster1999
added a commit
to JamMaster1999/any-llm
that referenced
this pull request
Sep 3, 2026
…mozilla-ai#1319 Upstream has merged mozilla-ai#1291, mozilla-ai#1292, mozilla-ai#1308, mozilla-ai#1309, mozilla-ai#1310, mozilla-ai#1317, mozilla-ai#1318, mozilla-ai#1320, mozilla-ai#1325 and mozilla-ai#1352 in their final form, so the fork's own copies are dropped in favour of upstream's. The tree is exactly upstream main plus the two fixes still open there: gemini reasoning_effort="none" (mozilla-ai#1294) and closing the provider stream when the wrapped stream closes (mozilla-ai#1319). Claude-Session: https://claude.ai/code/session_01MmJSSofg7Lk7nBKZZyKV7w
11 tasks
HareeshBahuleyan
added a commit
that referenced
this pull request
Sep 14, 2026
…#1393) ## Description Sibling gap from #1325, which fixed this exact shape in Gemini and Anthropic and left Bedrock with it. `_convert_response` collects every `text` content block into `content_parts`, then in the `stopReason == "tool_use"` branch builds the message with `content=None` and discards all of it. Converse returns the model's own text as a separate text block in the same message as the `toolUse` block, so a turn like "I will look up the weather in Paris." followed by a `get_weather` call reached the caller as a bare tool call with no text. Three things keep it quiet rather than loud: 1. The streaming converter already forwards those deltas (`_create_openai_chunk_from_aws_chunk` sets `content = delta["text"]`), so streaming and non-streaming disagreed about the same response. 2. `_convert_message_to_bedrock_format` already replays an assistant turn as a text block followed by `toolUse` blocks, so the request direction was fine. There was simply never any text to replay, because the response direction had dropped it. 3. There is no API error. The model just cannot see that it already announced its plan, so on the next turn it announces it again, which is the failure #1325 describes. The non-tool branch of the same function joins `content_parts` correctly. Only the tool branch threw them away. `"".join(content_parts) or None` preserves the existing contract for a bare tool call: joining an empty list yields `""`, and reporting an empty string where the model genuinely said nothing would be a behaviour change of its own. This matches Gemini, whose `text_content` stays `None` until a text part appears. I swept the rest of the provider family for the same pattern while I was in here. `grep -rn 'content=None' src/any_llm/providers/*/utils.py src/any_llm/providers/*/base.py` returns this one site and nothing else, so Bedrock was the last one. ### Tests 3 added. The first two fail against unpatched sources with: ``` assert None == 'I will look up the weather in Paris.' ``` The third covers a tool call carrying no text and passes both before and after, deliberately: it is the guard that the `or None` did not turn every bare tool call into an empty string. ## PR Type - 🐛 Bug Fix ## Relevant issues No open issue; found by sweeping the provider family for the pattern #1325 fixed. ## Checklist - [x] I understand the code I am submitting. - [x] I have added unit tests that prove my fix/feature works - [x] I have run this code locally and verified it fixes the issue. - [x] New and existing tests pass locally - [x] Documentation was updated where necessary - [x] I have read and followed the [contribution guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md) - [x] **AI Usage:** - [ ] No AI was used. - [x] AI was used for drafting/refactoring. - [ ] This is fully AI-generated. ## AI Usage Information - AI Model used: Opus 5 - AI Developer Tool used: Claude Code - Any other info you'd like to share: `tests/unit` 2542 passed, 66 skipped; pre-commit clean. When answering questions by the reviewer, please respond yourself, do not copy/paste the reviewer comments into an AI system and paste back its answer. We want to discuss with you, not your AI :) - [ ] I am an AI Agent filling out this form (check box if true) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Non-streaming Bedrock responses now preserve model-generated text when it appears alongside a tool call. * Multiple text sections are combined correctly, while tool calls without accompanying text continue to show no message content. * **Tests** * Added coverage for text before tool calls, multiple text sections, and tool calls without text. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Hareesh <hareeshbahuleyan@gmail.com>
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When a model writes a sentence and then calls a tool in the same turn, any-llm loses the sentence on Gemini and Anthropic. Two seams, same shape:
gemini/utils.py::_convert_response_to_response_dictsetcontenttoNonewhenever the response carried a function call, so the text part never reached the caller (the streaming converter already kept it)._convert_messagesthen rebuilt a tool-call turn fromtool_callsalone, so even a caller who had the text could not replay it.anthropic/utils.py::_convert_messages_for_anthropicbuilt the tool-call turn from the thinking block plus thetool_useblocks and never readcontent.The model cannot see that it already announced its plan, so it announces it again instead of answering. No API error, so nothing in the suite noticed. Bedrock's converter already does this right (
bedrock/utils.py:377, text beforetoolUse), so the fix follows that shape. Gemini reportscontentandtool_callstogether and replays the textPartahead of the function-call parts; a message-levelthought_signaturerides on that part, which covers the limitation noted on #1320 for the only shape Gemini 3 has been seen to produce (a non-empty text part). Anthropic inserts the text block between thinking andtool_use. Only a non-empty string adds a part or block: tool-call turns withcontentNoneor""still emit only call parts, so no empty text is ever sent, and list-shaped assistant content keeps today's behaviour on both providers. A side effect on Anthropic: an assistant turn with text andtool_calls: []used to producecontent: [], which the API rejects; it now carries its text.Live, through
acompletion, prompt "Before calling any tool, write one sentence telling me your plan. Then get the weather in Paris.", tool result fed back:content/ turn 2content/ turn 2None/ "I will use the weather tool to find the current weather conditions in Paris. The current..."None/ "OK. The weather in Paris is currently 18°C and cloudy."Streamed first turns, accumulated and replayed, behave the same on both providers.
tests/integrationfor gemini and anthropic on this branch: 50 passed, 13 skipped, both agent-loop tests included. Unit tests: the response keeps text next to a function call; Gemini replays[text, function_call]with each signature on its own part and round-trips response ->ChatCompletionMessage-> dump -> model turn; a malformed message-level signature on a tool-call turn is rejected like a text-only one; Anthropic replays[thinking, text, tool_use]; tool-call turns withNone/""content still emit only call parts. The positive tests fail on main, the negative ones pin what must not change.PR Type
Relevant issues
Follow-up to #1320 (signed text part beside a function call). None open.
Checklist
AI Usage Information
AI Model used: Claude (Fable 5)
AI Developer Tool used: Claude Code
Any other info you'd like to share:
I am an AI Agent filling out this form (check box if true)
Summary by CodeRabbit