fix(mcp): keep the MCP reference intact when the semantic filter narrows tools

[tin-berri]

Relevant issues

Linear ticket

Resolves LIT-4451

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Same proxy, same config, same request throughout; the only variable is the commit. The config enables the semantic tool filter and registers deepwiki, and the model is a real anthropic/claude-sonnet-4-5 so every run costs real money

model_list:
  - model_name: claude
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: text-embedding-3-small
    litellm_params:
      model: openai/text-embedding-3-small

mcp_servers:
  deepwiki:
    transport: "http"
    url: "https://mcp.deepwiki.com/mcp"

general_settings:
  master_key: sk-1234

litellm_settings:
  mcp_semantic_tool_filter:
    enabled: true
    embedding_model: "text-embedding-3-small"
    top_k: 5
    similarity_threshold: 0.3

The request is what the Admin UI playground sends when you pick an MCP server

curl -sS http://localhost:4141/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"claude",
       "messages":[{"role":"user","content":"Use deepwiki: call read_wiki_structure for repoName BerriAI/litellm and summarize."}],
       "tools":[{"type":"mcp","server_label":"litellm","server_url":"litellm_proxy/mcp/deepwiki","require_approval":"never"}],
       "tool_choice":"auto"}'

Before, captured at 74ff8d0 (current litellm_internal_staging). The filter selected tools, and the request still died in the Anthropic transformation because the tools it wrote were the wrong shape

x-litellm-semantic-filter: 3->3

{"error":{"message":"litellm.APIConnectionError: 'function'
Traceback (most recent call last):
  File \"litellm/main.py\", line 5258, in completion
    optional_params = get_optional_params(**optional_param_args, **non_default_params)
  File \"litellm/utils.py\", line 3876, in get_optional_params
    optional_params = litellm.AnthropicConfig().map_openai_params(
  File \"litellm/llms/anthropic/chat/transformation.py\", line 1414, in map_openai_params
    anthropic_tools, mcp_servers = self._map_tools(value)
  File \"litellm/llms/anthropic/chat/transformation.py\", line 910, in _map_tools
    new_tool, mcp_server_tool = self._map_tool_helper(tool)
  File \"litellm/llms/anthropic/chat/transformation.py\", line 637, in _map_tool_helper
    _input_schema: dict = tool[\"function\"].get(
KeyError: 'function'
. Received Model Group=claude"}}

On providers that skip unknown tool entries instead of raising, the same shape mismatch is silent: the model just answers as if no MCP server were connected, which is what the playground showed

After, captured at 5421fdf. The filter still runs, and the model now calls the tool and answers from the result

x-litellm-semantic-filter: 3->3

prompt_tokens : 2035
mcp_list_tools: 3
auto-executed : True
content head  : ## Summary of BerriAI/litellm Wiki Structure  The **BerriAI/litellm** repository documentation is comprehensively organized into 9 major sections:  ### 1. **Overview** -

Streaming, which is what the playground actually uses, at 5421fdf. The tool_calls then stop pair is the gateway executing the tool and feeding the result back

curl -sS -N http://localhost:4141/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"claude","stream":true,"stream_options":{"include_usage":true},
       "messages":[{"role":"user","content":"Use deepwiki: call read_wiki_structure for repoName BerriAI/litellm and summarize."}],
       "tools":[{"type":"mcp","server_label":"litellm","server_url":"litellm_proxy/mcp/deepwiki","require_approval":"never"}],
       "tool_choice":"auto"}'

finish_reasons  : ['tool_calls', 'stop']
tool_call deltas: 7
streamed content: 2133 chars -> "## Summary of BerriAI/litellm Wiki Structure ..."

Narrowing still takes effect rather than the filter being bypassed. With top_k: 1 at 5421fdf the header reports 3->1 and the model is handed only the selected tool

x-litellm-semantic-filter: 3->1
x-litellm-semantic-filter-tools: deepwiki-read_wiki_contents

Gateway Playground UI after control at 53e5b22c60

The authenticated Gateway Playground was run on the exact current head with /v1/chat/completions, model claude, MCP server deepwiki, and the semantic filter settings shown above. The prompt explicitly requested read_wiki_structure for BerriAI/litellm

Selected Chat Completions, Claude, and DeepWiki	List tools and grounded final answer
Approved call with request and response	Completed stream with timing and token metrics

The Playground rendered List tools with all three DeepWiki tools, then rendered deepwiki-read_wiki_structure with {"repoName":"BerriAI/litellm"}, status Approved, and a non-empty response beginning Available pages for BerriAI/litellm. The final answer summarized the nine documentation sections returned by that call

Sanitized proxy evidence showed the MCP reference expanding before filtering, the filter retaining 3->3 tools, the narrowed type=mcp reference reaching Anthropic through allowed_tools, streamed tool-call argument deltas, real DeepWiki execution, finish_reason=stop, and a final usage chunk with 2,661 tokens

Annotated recording and full runtime report

This section covers the requested runtime behavior only; PR checks were not all green when inspected

Type

🐛 Bug Fix

Changes

The semantic tool filter replaced each litellm_proxy MCP reference in data["tools"] with the tools it expanded from that reference. _process_mcp_tools_to_openai_format defaults to the Responses API tool shape, so a /chat/completions request came out carrying flat {"type": "function", "name": ...} entries where the provider transformations expect {"type": "function", "function": {...}}. Anthropic raised KeyError: 'function' and Bedrock skipped every MCP tool silently, so the model answered as if no MCP server were connected

Replacing the reference also removed the marker _should_use_litellm_mcp_gateway matches on, so acompletion_with_mcp never ran. That disabled tool auto-execution for require_approval="never" on /responses too, where the tool shape happened to be correct

The filter only ever needed to decide which tools survive, so it now writes that decision into the reference's allowed_tools and leaves the reference in place. The MCP gateway keeps ownership of expansion, which is already correct per endpoint, so both the tool shape and auto-execution follow from it rather than being re-derived in the hook. Expansion already applies any caller-supplied allowed_tools before the filter selects, so the selection can only narrow a reference further, never widen one

This has been broken since #20296, which introduced the expansion and accepted completion and acompletion in the hook. It reproduces on any deployment running the filter, which the shipped proxy_config.yaml enables

Two existing tests asserted the old contract and now assert the new one. test_semantic_filter_hook_filters_expanded_litellm_proxy_tools keeps its LIT-4214 intent, that the reference is genuinely narrowed and real pre/post stats are emitted, expressed through allowed_tools. test_semantic_filter_hook_expansion_skips_filter_when_disabled now asserts the reference is left untouched when the filter is disabled, which is what lets the gateway expand it

test_semantic_filter_hook_narrows_mcp_reference_for_chat_completions is the regression test. It drives a real {"type": "mcp"} reference through the hook with call_type="acompletion" and asserts the reference survives carrying the selected tools. On litellm_internal_staging it fails with assert ['function'] == ['mcp'], showing the expanded definition that replaced the reference

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes pre-call tool shaping on completion/responses paths for MCP gateway references; behavior is well covered by regression tests but affects tool availability and auto-execution for real MCP traffic.

Overview
Fixes LIT-4451: semantic filtering for litellm_proxy MCP tool references no longer swaps the reference for expanded OpenAI function dicts.

After expansion and filtering, the hook writes selected tool names onto each gateway MCP reference’s allowed_tools and leaves type: mcp entries in data["tools"]. The MCP gateway still expands per endpoint, so chat completions keep the nested function shape, Anthropic/Bedrock mappings work, and require_approval: "never" auto-execution still runs.

When the filter is disabled, the hook returns early without mutating tools (no inlined expanded tools). Tests were updated and added for chat completions, zero-match behavior on both reference and plain-tool paths, and the disabled-filter case.

^{Reviewed by Cursor Bugbot for commit 53e5b22. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR fixes a bug (LIT-4451) where the MCP semantic filter replaced litellm_proxy MCP tool references with expanded function-tool dicts, causing KeyError: 'function' on Anthropic and silent tool drops on Bedrock, and disabling require_approval="never" auto-execution by removing the marker the MCP gateway checks.

• The hook now writes semantically selected tool names into allowed_tools on each {"type":"mcp"} reference and returns it intact, so the gateway retains ownership of expansion (and its per-endpoint tool shape) while the filter's narrowing is still applied.
• When the filter is disabled, the hook returns None immediately rather than forwarding expanded tools, restoring the gateway as the sole expansion authority.
• Two existing tests are updated to assert the new contract; three new tests cover the acompletion regression, the zero-match parity between the reference and plain-tool paths, and the disabled-filter early exit.

Confidence Score: 5/5

Safe to merge — the change is surgical, backed by a concrete reproduction, and fully tested with mocks.

The fix is targeted: one new static helper (_narrow_mcp_references) and a reordering of three lines in the expansion branch. The allowed_tools contract with the gateway is confirmed by reading _filter_mcp_tools_by_allowed_tools, which correctly treats an empty union as 'no restriction'. The two updated tests reflect the real behavioral change rather than weakening coverage, and the three new tests pin the regression, zero-match behavior, and disabled-filter path. No net-new code paths are left untested.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/hooks/mcp_semantic_filter/hook.py	Narrows MCP references via allowed_tools instead of replacing them with expanded tool dicts; fixes KeyError on Anthropic and silent tool drop on Bedrock. Logic is correct and the empty-union-as-unset contract with the gateway is verified.
tests/test_litellm/proxy/_experimental/mcp_server/test_semantic_tool_filter.py	Two updated tests correctly reflect the new contract; three new tests cover the regression, zero-match parity, and disabled-filter path. All tests use mocks — no real network calls.

_{Reviews (2): Last reviewed commit: "fix(mcp): let filter_tools own the undec..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5421fdf. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit4451_semfilter_chat_tool_format (53e5b22) with litellm_internal_staging (2162da5)}

[tin-berri]

@greptileai

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 53e5b22. Configure here.}

[devin-ai-integration]

Tested exact head 53e5b22c609fba2cd45a9dab735c07bc44e41ae3 through the authenticated Gateway Playground with semantic MCP filtering enabled, real Anthropic inference, and the DeepWiki MCP server

PR checks were not all green when inspected; this comment covers the requested runtime behavior only

Gateway Playground runtime proof

• Passed: /v1/chat/completions, claude, and deepwiki were selected in the Playground
• Passed: List tools rendered all three DeepWiki tools
• Passed: deepwiki-read_wiki_structure rendered with {"repoName":"BerriAI/litellm"}, Approved, and a non-empty response
• Passed: the final answer summarized the nine documentation sections returned by DeepWiki
• Passed: the request streamed through Chat Completions with TTFT, latency, token metrics, tool-call deltas, and finish_reason=stop

Sanitized proxy evidence

MCP Semantic Tool Filter enabled: embedding_model=text-embedding-3-small, top_k=5, similarity_threshold=0.3
Detected litellm_proxy MCP references, expanding before semantic filtering
Expanded 1 MCP reference to 3 tools
Semantic tool filter: 3->3 MCP tools (0 native preserved, 1 total MCP reference)
Preserved type=mcp reference with allowed_tools=[deepwiki-read_wiki_structure, deepwiki-read_wiki_contents, deepwiki-ask_question]
Streamed tool-call deltas assembled deepwiki-read_wiki_structure with {"repoName":"BerriAI/litellm"}
MCP client called read_wiki_structure and returned a non-empty result
Follow-up stream yielded finish_reason=stop and total_tokens=2661

Detailed-debug logging also emitted non-blocking cost-calculation warnings for the synthetic MCP: list_tools metadata event; the MCP request and final stream completed successfully

Full report and annotated recording: https://app.devin.ai/sessions/f03da2725ec94d28b3facf766871b102