Repository navigation
fix(anthropic): preserve MCP tool results in the non-Anthropic Messages bridge - #42783
Conversation
…es bridge The tool_result user message built by the /v1/messages MCP loop used tuple content, which the Messages to Chat Completions adapter silently dropped, so non-Anthropic models re-requested the tool until the iteration cap or the provider rejected the follow-up. Emit list content so the existing tool_result branch translates it into a role tool message keyed by tool_call_id. Resolves LIT-8474 Co-Authored-By: bot_apk <apk@cognition.ai>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 8f0e8a6. Configure here.
TLDR
Problem this solves:
/v1/messageswith an MCP tool on a non-Anthropic model never sees the tool resultHow it solves it:
tool_resultblocks intorole: toolmessagesUser Flow
Before: a developer calling the Anthropic-style endpoint with an OpenAI model and a proxy MCP tool gets an error instead of an answer
"model": "gpt-4.1-mini", a user turn asking to use the deepwikiread_wiki_structuretool, and"tools": [{"type": "mcp", "server_url": "litellm_proxy", "require_approval": "never"}]OpenAIException - No tool output found for function call call_..., with the answer nowhere in the responseAfter: the same request returns the model's final answer
"stop_reason": "end_turn"and a text block answering the questionRelevant issues
Follow-up to #42711, which added the integration test with a BUG skip for this surface
Affected release
Linear ticket
Resolves LIT-8474
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup: a local proxy started with
python -m litellm.proxy.proxy_cli --config config.yaml --port 4010 --num_workers 1 --detailed_debug, real OpenAI and the public deepwiki MCP server, no mocksRequest used on both sides:
Before (59eb943)
/v1/messages with an OpenAI model and an auto-approved proxy MCP tool
{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: OpenAIException - {\n \"error\": {\n \"message\": \"No tool output found for function call call_2xJovk7Frm7y3tJe4L95SfBV.\",\n \"type\": \"invalid_request_error\",\n \"param\": \"input\",\n \"code\": null\n }\n}\n\nLiteLLM: model group 'gpt-4.1-mini' failed with the error above. No fallback was attempted."}}After (8f0e8a6)
/v1/messages with an OpenAI model and an auto-approved proxy MCP tool
{ "model": "gpt-4.1-mini", "content": [ { "type": "text", "text": "The BerriAI/litellm repository has 9 top-level sections in its documentation structure." } ], "stop_reason": "end_turn" }The deterministic version of this run is
tests/integration/mcp/test_mcp_llm_endpoints.py::test_auto_approved_gateway_tool_is_listed_executed_once_and_fed_back[messages_bridge](real proxy with Postgres and Redis, scripted upstream and MCP peer). It used to hit the BUG skip; at the tip it asserts one tool execution, two upstream requests, and the final answer, and the follow-up upstream request now carries{"role": "tool", "tool_call_id": "call_1", "content": "5"}. Providers that accept a dangling tool call instead of rejecting it (the scripted upstream behaves like this) showed the other symptom of the same bug, the tool being executed ten times before the loop gave upType
🐛 Bug Fix
Caveats (if any)
Low
ruff format, so its diff is larger than the two asserts that matterFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/5018f8150b5544a79341d0024d90e5ee
Open in Devin Desktop: https://app.devin.ai/desktop/session/5018f8150b5544a79341d0024d90e5ee?variant=devin
Note
Low Risk
One-line shape change in MCP follow-up message construction, covered by unit and integration tests on the messages bridge path.
Overview
Fixes LIT-8474: MCP tool loops on
/v1/messageswith a non-Anthropic upstream model were failing or re-running tools because executed results never reached the model._build_tool_result_messagenow builds user messagecontentas a list instead of a tuple. The Messages→Chat Completions bridge inLiteLLMAnthropicMessagesAdapteronly walksisinstance(..., list)user content when turningtool_resultblocks intorole: "tool"messages, so tuple content was silently dropped on themessages_bridgepath.Tests add a unit regression through the adapter, remove the integration BUG skip for
messages_bridge, and expect a single tool execution with tool output on the follow-up upstream request.Reviewed by Cursor Bugbot for commit 8f0e8a6. Bugbot is set up for automated code reviews on this repo. Configure here.