Skip to content

fix(anthropic): preserve MCP tool results in the non-Anthropic Messages bridge - #42783

Merged
joshua-berri merged 1 commit into
mainfrom
litellm_lit8474_bridge_tool_result
Sep 23, 2026
Merged

joshua-berri merged 1 commit into
mainfrom
litellm_lit8474_bridge_tool_result

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • /v1/messages with an MCP tool on a non-Anthropic model never sees the tool result
  • The model re-requests the tool until the iteration cap, or the provider rejects the follow-up

How it solves it:

  • The tool result message built after executing MCP tools now uses list content
  • The Messages to Chat Completions bridge already translates list tool_result blocks into role: tool messages

User Flow

Before: a developer calling the Anthropic-style endpoint with an OpenAI model and a proxy MCP tool gets an error instead of an answer

  1. They send POST http://localhost:4000/v1/messages with "model": "gpt-4.1-mini", a user turn asking to use the deepwiki read_wiki_structure tool, and "tools": [{"type": "mcp", "server_url": "litellm_proxy", "require_approval": "never"}]
  2. The proxy runs the MCP tool on their behalf
  3. They get back HTTP 400 with OpenAIException - No tool output found for function call call_..., with the answer nowhere in the response

After: the same request returns the model's final answer

  1. They send the same POST http://localhost:4000/v1/messages
  2. The proxy runs the MCP tool on their behalf
  3. They get back HTTP 200 with "stop_reason": "end_turn" and a text block answering the question

Relevant issues

Follow-up to #42711, which added the integration test with a BUG skip for this surface

Affected release

Linear ticket

Resolves LIT-8474

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup: a local proxy started with python -m litellm.proxy.proxy_cli --config config.yaml --port 4010 --num_workers 1 --detailed_debug, real OpenAI and the public deepwiki MCP server, no mocks

model_list:
  - model_name: gpt-4.1-mini
    litellm_params:
      model: openai/gpt-4.1-mini
      api_key: os.environ/OPENAI_API_KEY

mcp_servers:
  deepwiki:
    url: https://mcp.deepwiki.com/mcp
    transport: http

general_settings:
  master_key: sk-lit8474-local

Request used on both sides:

curl -s http://127.0.0.1:4010/v1/messages \
  -H "Authorization: Bearer sk-lit8474-local" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4.1-mini",
    "max_tokens": 300,
    "messages": [{"role": "user", "content": "Use the deepwiki read_wiki_structure tool for BerriAI/litellm once, then tell me in one sentence how many top-level sections it has."}],
    "tools": [{"type": "mcp", "server_url": "litellm_proxy", "server_label": "litellm", "require_approval": "never"}]
  }'

Before (59eb943)

/v1/messages with an OpenAI model and an auto-approved proxy MCP tool

  1. Run the curl above against the proxy at the merge base
  2. Observed output (HTTP 400): the proxy ran the tool, then the follow-up call to OpenAI was rejected because the tool output was missing from it
{"type":"error","error":{"type":"invalid_request_error","message":"litellm.BadRequestError: OpenAIException - {\n  \"error\": {\n    \"message\": \"No tool output found for function call call_2xJovk7Frm7y3tJe4L95SfBV.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"input\",\n    \"code\": null\n  }\n}\n\nLiteLLM: model group 'gpt-4.1-mini' failed with the error above. No fallback was attempted."}}

After (8f0e8a6)

/v1/messages with an OpenAI model and an auto-approved proxy MCP tool

  1. Run the same curl against the proxy at the PR tip
  2. Observed output (HTTP 200): the tool result reached the model and it answered
{
 "model": "gpt-4.1-mini",
 "content": [
  {
   "type": "text",
   "text": "The BerriAI/litellm repository has 9 top-level sections in its documentation structure."
  }
 ],
 "stop_reason": "end_turn"
}

The deterministic version of this run is tests/integration/mcp/test_mcp_llm_endpoints.py::test_auto_approved_gateway_tool_is_listed_executed_once_and_fed_back[messages_bridge] (real proxy with Postgres and Redis, scripted upstream and MCP peer). It used to hit the BUG skip; at the tip it asserts one tool execution, two upstream requests, and the final answer, and the follow-up upstream request now carries {"role": "tool", "tool_call_id": "call_1", "content": "5"}. Providers that accept a dangling tool call instead of rejecting it (the scripted upstream behaves like this) showed the other symptom of the same bug, the tool being executed ten times before the loop gave up

Type

🐛 Bug Fix

Caveats (if any)

Low

  • The unit test file was reformatted by ruff format, so its diff is larger than the two asserts that matter

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/5018f8150b5544a79341d0024d90e5ee
Open in Devin Desktop: https://app.devin.ai/desktop/session/5018f8150b5544a79341d0024d90e5ee?variant=devin


Note

Low Risk
One-line shape change in MCP follow-up message construction, covered by unit and integration tests on the messages bridge path.

Overview
Fixes LIT-8474: MCP tool loops on /v1/messages with a non-Anthropic upstream model were failing or re-running tools because executed results never reached the model.

_build_tool_result_message now builds user message content as a list instead of a tuple. The Messages→Chat Completions bridge in LiteLLMAnthropicMessagesAdapter only walks isinstance(..., list) user content when turning tool_result blocks into role: "tool" messages, so tuple content was silently dropped on the messages_bridge path.

Tests add a unit regression through the adapter, remove the integration BUG skip for messages_bridge, and expect a single tool execution with tool output on the follow-up upstream request.

Reviewed by Cursor Bugbot for commit 8f0e8a6. Bugbot is set up for automated code reviews on this repo. Configure here.

…es bridge

The tool_result user message built by the /v1/messages MCP loop used tuple
content, which the Messages to Chat Completions adapter silently dropped, so
non-Anthropic models re-requested the tool until the iteration cap or the
provider rejected the follow-up. Emit list content so the existing tool_result
branch translates it into a role tool message keyed by tool_call_id.

Resolves LIT-8474

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 23, 2026 19:30
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with focused production behavior and regression coverage for the reported bridge failure

Summary

This PR changes MCP tool-result messages to use list content so the Anthropic Messages to Chat Completions bridge converts them into tool-role messages correctly

  • Preserves executed MCP results when /v1/messages targets a non-Anthropic chat-completions model
  • Adds a focused adapter regression test covering multiple tool results
  • Enables the existing integration assertions for the Messages bridge instead of skipping the failing behavior

Reviews (1) · Last reviewed commit: "fix(anthropic): preserve MCP tool result..."

@codecov

codecov Bot commented Sep 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_lit8474_bridge_tool_result (8f0e8a6) with main (e0af991)

Open in CodSpeed

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8f0e8a6. Configure here.

@joshua-berri joshua-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work, Devin!

@joshua-berri
joshua-berri merged commit 5db8543 into main Sep 23, 2026
102 of 103 checks passed
@joshua-berri
joshua-berri deleted the litellm_lit8474_bridge_tool_result branch September 23, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants