fix(mcp): apply semantic filter to expanded litellm_proxy tools and show filtered-out count

[tin-berri]

Relevant issues

Linear ticket

Resolves LIT-4214

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy on localhost:4000 backed by Postgres, hitting real OpenAI APIs (gpt-4o-mini plus text-embedding-3-small for the semantic router). Config: two public MCP servers (deepwiki, microsoft_learn, 6 tools total) and litellm_settings.mcp_semantic_tool_filter: {enabled: true, top_k: 2, similarity_threshold: 0.3, embedding_model: text-embedding-3-small}

Request used throughout (identical to what the admin UI test panel and its documented curl send):

curl -sD - http://localhost:4000/v1/responses \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-4o-mini",
    "input": [{"role": "user", "content": "Look up the deepwiki docs structure for the fastapi/fastapi repo", "type": "message"}],
    "tools": [{"type": "mcp", "server_url": "litellm_proxy", "require_approval": "never"}],
    "tool_choice": "required"
  }'

Before (unfixed hook, at origin/litellm_internal_staging): HTTP 200 but no semantic filter headers at all, and the outbound OpenAI payload carried all 6 tools

HTTP/1.1 200 OK
(no x-litellm-semantic-filter header present)

litellm.log:
14:55:32 - LiteLLM Proxy:INFO: hook.py:219 - Expanded MCP references to 6 tools (0 native preserved), skipping semantic filter (OpenAI nested format)

tools in outbound request payload (6 = everything):
deepwiki-ask_question, deepwiki-read_wiki_contents, deepwiki-read_wiki_structure,
microsoft_learn-microsoft_code_sample_search, microsoft_learn-microsoft_docs_fetch, microsoft_learn-microsoft_docs_search

After (this PR): same curl returns real counts and only the selected tools reach the model

HTTP/1.1 200 OK
x-litellm-semantic-filter: 6->2
x-litellm-semantic-filter-tools: deepwiki-read_wiki_structure,deepwiki-read_wiki_contents

litellm.log:
14:56:26 - LiteLLM Proxy:INFO: hook.py:247 - Expanded MCP references to 6 tools (0 native preserved), semantic filter selected 2

Selection is query dependent; an Azure flavored prompt ("Find official Azure docs on configuring Entra ID app registrations") over the same 6 tools selects the other server's tools

HTTP/1.1 200 OK
x-litellm-semantic-filter: 6->2
x-litellm-semantic-filter-tools: microsoft_learn-microsoft_docs_search,microsoft_learn-microsoft_code_sample_search

UI verification: the Results alert now reads "X of Z tools selected" with "Y tools filtered out" underneath, and turns into a yellow warning when nothing was filtered out so a no-op filter is obvious at a glance. The screenshots below drive the real MCPSemanticFilterTestPanel with the header-derived counts from the runs above (in the proxy this is at http://localhost:4000/ui -> Settings -> Admin Settings -> MCP Semantic Filter, enter a query, select gpt-4o-mini, click Test Filter)

Filtered case (matches the 6->2 curl above)

Before: a bare green "2 tools selected" over "Filtered from 6 available tools" that never states how many were dropped

After: green "2 of 6 tools selected" over "4 tools filtered out"

No-op case (filter selects everything, the same shape as the customer's 207/207 report)

Before: still a green success "6 tools selected" over "Filtered from 6 available tools", so a filter that pruned nothing reads as a win

After: a yellow warning "6 of 6 tools selected" over "0 tools filtered out", so the no-op is called out

Type

🐛 Bug Fix

Changes

The MCP semantic filter test panel showed results like "207 tools selected / Filtered from 207 available tools", which never states how many tools were filtered out and makes a filter no-op read as success. A customer hit exactly that: the panel reported 207/207 while their real traffic forwarded every MCP tool to the model unfiltered, blowing past provider tool count limits

Backend: SemanticToolFilterHook.async_pre_call_hook has a dedicated branch for requests whose tools are {"type": "mcp", "server_url": "litellm_proxy"} references, which is the exact request the test panel and its documented curl send. That branch expanded the references and returned early, skipping semantic filtering and emitting no x-litellm-semantic-filter header at all. Its stated reason ("expanded tools are in OpenAI nested format which cannot name-match") does not hold on this path: expansion goes through transform_mcp_tool_to_openai_responses_api_tool, which produces flat function dicts whose top-level name _extract_tool_info reads fine. The branch now runs the expanded tools through filter_tools and emits the same stats metadata as the generic path, so the header carries real pre/post counts and the filter actually prunes the tool list before it reaches the model. A plain string input (valid on /v1/responses) is wrapped into a message list before query extraction so it cannot break the branch. The branch mirrors the generic path's other guards too: when the filter is disabled at runtime the expanded tools pass through with no stats, and metadata emission goes through a shared _emit_filter_metadata_safe helper so an emission failure drops the header instead of aborting the expansion

UI: the results alert now reads "X of Z tools selected" with "Y tools filtered out" underneath, and renders as a warning instead of a success when nothing was filtered out, so a no-op filter is visible at a glance

Tests: test_semantic_filter_hook_filters_expanded_litellm_proxy_tools fails on the old hook (all 5 expanded tools forwarded, no stats emitted) and passes with the fix (top_k respected, stats 5->N); test_semantic_filter_hook_filters_expanded_tools_with_string_input covers the string input arm; test_semantic_filter_hook_expansion_skips_filter_when_disabled covers the disabled toggle; the panel specs assert the new copy including the zero filtered warning case

Link to Devin session: https://app.devin.ai/sessions/f03da2725ec94d28b3facf766871b102

---

Note

Medium Risk
Changes pre-LLM tool lists on a high-traffic proxy hook path; behavior is guarded by tests and mirrors the existing generic filter path, but wrong filtering could still affect provider tool limits.

Overview
Fixes LIT-4214: /v1/responses requests that pass litellm_proxy MCP tool references no longer skip semantic filtering after expansion.

Proxy hook: The litellm_proxy expansion branch used to attach every expanded tool and return without filtering or x-litellm-semantic-filter stats. It now runs expanded flat function dicts through filter_tools (with string input coerced for query extraction), honors filter.enabled (pass-through with no stats when off), and emits the same metadata via _emit_filter_metadata_safe so a metadata failure cannot abort the request.

Admin UI: The MCP semantic filter test panel shows “X of Z tools selected” and “Y tools filtered out”, and uses a warning alert when nothing was filtered so no-op runs are obvious.

Tests: Regression coverage for the expansion path (filtering, stats, disabled filter, string input) and updated panel copy expectations.

^{Reviewed by Cursor Bugbot for commit 14ff297. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai

[greptile-apps]

Greptile Summary

Fixes the MCP semantic filter hook silently skipping semantic filtering when tools are {"type": "mcp", "server_url": "litellm_proxy"} references: the expansion path now runs expanded tools through filter_tools, emits accurate N->M stats headers, and respects the enabled flag. The UI test panel alert text is improved to show filtered-out count and renders as a warning instead of success when the filter is a no-op.

• Backend (hook.py): _filter_expanded_tools applies semantic filtering post-expansion; _emit_filter_metadata_safe is extracted as a shared helper; filter.enabled guard added to the expansion branch.
• Tests: Three new regression tests cover the happy path, string input arm, and disabled-filter passthrough; all calls are fully mocked.
• UI: Results alert text updated to X of Y tools selected / Z tools filtered out, with a warning alert type when nothing is filtered.

Confidence Score: 5/5

Safe to merge — the change is scoped to the MCP expansion branch of one hook, all fallback paths are preserved, and all issues from the previous review thread are addressed.

The expansion path is well-guarded: the enabled check, the _emit_filter_metadata_safe wrapper, and the outer try/except each independently prevent a failure from corrupting the request. No pre-existing behavior is changed for non-MCP requests.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/hooks/mcp_semantic_filter/hook.py	Core fix: expansion path now applies semantic filtering and emits stats; shared _emit_filter_metadata_safe helper and enabled guard address the previously-flagged issues
tests/test_litellm/proxy/_experimental/mcp_server/test_semantic_tool_filter.py	Three new regression tests cover: expanded tools filtered correctly, string input arm, and disabled-filter passthrough — all fully mocked, no real network calls
ui/litellm-dashboard/src/components/Settings/AdminSettings/MCPSemanticFilterSettings/MCPSemanticFilterTestPanel.tsx	Alert now shows X of Y tools selected and Z tools filtered out, renders as warning instead of success when nothing is filtered out
ui/litellm-dashboard/src/components/Settings/AdminSettings/MCPSemanticFilterSettings/MCPSemanticFilterTestPanel.test.tsx	Test updated to assert new copy and added zero-filtered-out case; assertions are appropriately tightened, not weakened

_{Reviews (3): Last reviewed commit: "fix(mcp): guard expansion-path filtering..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 81.81818% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/hooks/mcp_semantic_filter/hook.py	81.81%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[tin-berri]

@greptileai

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 14ff297. Configure here.}

[mateo-berri]

LGTM; thanks!