feat(mcp): add mcp_tool_search virtual tools for large tool catalogs

[krrish-berri-2]

Relevant issues

N/A

Linear ticket

N/A

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

New Feature

Changes

Adds two virtual MCP tools (mcp_tool_search, mcp_tool_call) that activate when mcp_tool_search_enabled: true is set on a key's object_permission.

The problem: With 100+ MCP tools, sending the full catalog to the LLM wastes context and hurts tool selection accuracy.

The solution: When the flag is on, GET /mcp-rest/tools/list returns only two tools instead of the full catalog. The LLM calls mcp_tool_search with a keyword query to discover relevant tools, then calls mcp_tool_call with the discovered tool name to execute it. Both SSE (/mcp/tools/call) and REST (/mcp-rest/tools/call) handlers intercept these virtual tool names before the normal server routing.

mcp_tool_search uses token-based scoring against tool name and description fields. No new dependencies.

mcp_tool_call resolves the caller's allowed MCP servers and dispatches through the same execute_mcp_tool path the normal /tools/call route uses, so a tool discovered by search (e.g. math-add) executes with identical routing, auth and logging.

Setup:

curl http://localhost:4000/key/generate \
  -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"object_permission": {"mcp_tool_search_enabled": true, "mcp_servers": ["github", "slack"]}}'

Keys without the flag get the existing behavior unchanged.

Files changed

• litellm/proxy/_experimental/mcp_server/tool_search.py - new module: search_tools(), get_virtual_tool_definitions(), handle_mcp_tool_search(), handle_mcp_tool_call()
• litellm/proxy/_experimental/mcp_server/rest_endpoints.py - early-return in list_tool_rest_api; intercept in call_tool_rest_api
• litellm/proxy/_experimental/mcp_server/server.py - intercept in SSE mcp_server_tool_call
• litellm/proxy/_types.py + litellm/models/object_permission.py - add mcp_tool_search_enabled field
• schema.prisma, litellm/proxy/schema.prisma, litellm-proxy-extras/litellm_proxy_extras/schema.prisma - add mcp_tool_search_enabled Boolean? column to LiteLLM_ObjectPermissionTable
• litellm-proxy-extras/litellm_proxy_extras/migrations/20260626120000_add_mcp_tool_search_enabled/migration.sql - DB migration for the new column
• tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_tool_search.py - 19 tests
• tests/test_litellm/proxy/management_helpers/test_object_permission_utils.py - regression test that the flag is carried into the key-generation persist path

Screenshots / Proof of Fix

Live proxy with a math MCP server (add, multiply) configured, hitting the real server end to end.

Key with tool search enabled - tools/list returns only the two virtual tools, search finds the real tools, and a discovered tool executes:

$ KEY=$(curl -s -X POST http://localhost:4000/key/generate -H "Authorization: Bearer $ADMIN" \
    -d '{"object_permission":{"mcp_tool_search_enabled":true,"mcp_servers":["math"]}}' | jq -r .key)

$ curl -s http://localhost:4000/mcp-rest/tools/list -H "Authorization: Bearer $KEY" | jq '[.tools[].name]'
["mcp_tool_search", "mcp_tool_call"]

$ curl -s -X POST http://localhost:4000/mcp-rest/tools/call -H "Authorization: Bearer $KEY" \
    -d '{"name":"mcp_tool_search","arguments":{"query":"add numbers"}}' \
    | jq -r '.content[0].text | fromjson | [.[].name]'
["math-add", "math-multiply"]

$ curl -s -X POST http://localhost:4000/mcp-rest/tools/call -H "Authorization: Bearer $KEY" \
    -d '{"name":"mcp_tool_call","arguments":{"tool_name":"math-add","arguments":{"a":3,"b":4}}}' \
    | jq '{result: .content[0].text, isError}'
{
  "result": "7",
  "isError": false
}

Key without the flag is unchanged - full catalog on tools/list, and the virtual tools are rejected:

$ curl -s http://localhost:4000/mcp-rest/tools/list -H "Authorization: Bearer $KEY2" | jq '[.tools[].name]'
["add", "multiply"]

$ curl -s -X POST http://localhost:4000/mcp-rest/tools/call -H "Authorization: Bearer $KEY2" \
    -d '{"name":"mcp_tool_search","arguments":{"query":"add"}}' | jq .detail
{
  "error": "forbidden",
  "message": "mcp_tool_search requires mcp_tool_search_enabled on the key"
}

The flag persists to the DB through /key/generate:

$ psql "$DATABASE_URL" -t -c "SELECT opt.mcp_tool_search_enabled, opt.mcp_servers
    FROM \"LiteLLM_VerificationToken\" vt
    JOIN \"LiteLLM_ObjectPermissionTable\" opt USING (object_permission_id)
    WHERE opt.mcp_tool_search_enabled IS TRUE ORDER BY vt.created_at DESC LIMIT 1;"
 t | {8b38bf0c1d4fad25b013eb1304e851a8}

Access control on the virtual tools

Both virtual handlers go through the same filtered flow the normal MCP path uses, so the feature does not widen the access surface. Search lists tools via the filtered catalog (_list_mcp_tools) rather than the raw manager list, and call resolves the caller's allowed servers via _get_allowed_mcp_servers and dispatches through execute_mcp_tool. Both are threaded with the request client IP, so filter_server_ids_by_ip applies and a server marked available_on_public_internet: false stays unreachable from a public IP. execute_mcp_tool then enforces the server allowlist and per-key mcp_tool_permissions.

A key scoped to one server cannot call a tool on another server through mcp_tool_call:

$ curl -s -X POST http://localhost:4000/mcp-rest/tools/call -H "Authorization: Bearer $KEY" \
    -d '{"name":"mcp_tool_call","arguments":{"tool_name":"secret-server-delete_all","arguments":{}}}'
{"detail":"User not allowed to call this tool. Allowed MCP servers: [math]"}

Real MCP client over the protocol endpoint

The above uses the REST surface. A real MCP client connects over the streamable-http protocol endpoint (/mcp/), handled separately in server.py. Verified with the mcp Python SDK client against the same live math server:

async with streamablehttp_client("http://localhost:4000/mcp/",
        headers={"Authorization": f"Bearer {KEY}"}) as (r, w, _):
    async with ClientSession(r, w) as s:
        await s.initialize()
        print([t.name for t in (await s.list_tools()).tools])
        print((await s.call_tool("mcp_tool_search", {"query": "add numbers"})).content[0].text)
        print((await s.call_tool("mcp_tool_call",
            {"tool_name": "math-add", "arguments": {"a": 3, "b": 4}})).content[0].text)

Flagged key:

list_tools -> ['mcp_tool_search', 'mcp_tool_call']
mcp_tool_search -> [{"name": "math-add", ...}, {"name": "math-multiply", ...}]
mcp_tool_call(math-add 3+4) -> 7  isError=False

No-flag key over the same protocol endpoint is unchanged:

list_tools -> ['math-add', 'math-multiply']

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[krrish-berri-2]

@greptile review

[codecov]

Codecov Report

❌ Patch coverage is 93.69369% with 7 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/_experimental/mcp_server/server.py	87.50%	7 Missing ⚠️

📢 Thoughts on this report? Let us know!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4fd3125e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[greptile-apps]

Greptile Summary

Introduces two virtual MCP tools (mcp_tool_search and mcp_tool_call) activated via a per-key mcp_tool_search_enabled flag, replacing the full tool catalog with a keyword-search-then-execute pattern when enabled. The feature is cleanly scoped behind a new Boolean? database column, carries existing IP filtering, server allowlists, and spend logging through both the REST and SSE paths, and includes a 19-test mock suite.

• tool_search.py implements token-based scoring, virtual tool schema definitions, and the search/call handlers; server.py and rest_endpoints.py add flag-gated early-returns and a new _dispatch_virtual_mcp_tool helper that runs the full pre-call pipeline (guardrails + spend logging) for the SSE path.
• coerce_top_k allows negative integers to pass through, causing search_tools to slice the sorted result list incorrectly for negative values (e.g. [:-1] returns all but the last match); a max(1, int(value)) guard and a minimum: 1 JSON Schema constraint close this gap.
• The synthetic ASGI Request scope used in _build_virtual_call_logging_obj omits query_string, scheme, server, and root_path, so str(request.url) stored in spend-log metadata resolves to just the path string rather than a full URL.

Confidence Score: 5/5

Safe to merge; the feature is strictly additive, flag-gated, and leaves all existing key flows untouched.

The core access-control path (IP filtering, server allowlists, per-key tool permissions, spend logging) reuses proven helpers rather than reimplementing them, and every security boundary is covered by targeted tests. The two findings are quality concerns: a negative-top_k slice edge case that produces too many search results (not a security bypass), and an incomplete URL string in spend-log metadata for the SSE path. Neither affects correctness of tool dispatch or authorization.

tool_search.py — negative top_k coercion and missing minimum constraint in the JSON Schema; server.py — incomplete synthetic ASGI scope in _build_virtual_call_logging_obj.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/tool_search.py	New module implementing the virtual tool pair (mcp_tool_search, mcp_tool_call); clean separation of concerns, but coerce_top_k allows negative values through which causes incorrect slice behavior in search_tools.
litellm/proxy/_experimental/mcp_server/server.py	Adds list_tools early-return, _dispatch_virtual_mcp_tool, and _build_virtual_call_logging_obj; progress-callback extraction refactored; synthetic ASGI scope in _build_virtual_call_logging_obj is missing standard fields that produce incomplete URL strings in spend logs.
litellm/proxy/_experimental/mcp_server/rest_endpoints.py	Adds flag-gated early-returns for list and call REST endpoints; virtual tool interception correctly placed before server_id validation; flag check for mcp_tool_call correctly runs common_processing_pre_call_logic for spend logging.
litellm-proxy-extras/litellm_proxy_extras/migrations/20260626120000_add_mcp_tool_search_enabled/migration.sql	Adds nullable Boolean column mcp_tool_search_enabled to LiteLLM_ObjectPermissionTable; correct additive migration with no column removals.
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_tool_search.py	Comprehensive mock-only test suite covering 19 scenarios: unit tests for search_tools and coerce_top_k, integration mocks for list/call REST endpoints, SSE dispatch helper, and error conversion. No real network calls.
litellm/proxy/_types.py	Adds mcp_tool_search_enabled Optional[bool] field to LiteLLM_ObjectPermissionBase; consistent with other optional permission fields.
schema.prisma	Adds mcp_tool_search_enabled Boolean? to LiteLLM_ObjectPermissionTable in all three schema copies consistently.

_{Reviews (6): Last reviewed commit: "fix(mcp): mirror pre-call pipeline, guar..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR introduces two virtual MCP tools (mcp_tool_search, mcp_tool_call) behind a per-key mcp_tool_search_enabled flag, allowing LLMs to navigate large tool catalogs without receiving the full list upfront. The schema, type definitions, and migration are all consistent and additive.

• tool_search.py implements token-based search and the two virtual tool handlers, reusing _list_mcp_tools and execute_mcp_tool for access-control consistency.
• The REST path (rest_endpoints.py) correctly early-exits list_tool_rest_api and runs common_processing_pre_call_logic (guardrails + spend logging) before dispatching mcp_tool_call.
• The SSE/streamable-HTTP path (server.py) intercepts virtual tool names in _dispatch_virtual_mcp_tool but does not run the pre-call pipeline before calling handle_mcp_tool_call, so guardrails and spend logging are skipped for mcp_tool_call over SSE — inconsistent with both the REST path and the AGENTS.md requirement added in this same PR.

Confidence Score: 3/5

Safe to merge for the REST surface, but the SSE/streamable-HTTP surface for mcp_tool_call does not run guardrails or produce spend logs, contradicting the stated design contract.

The SSE handler in _dispatch_virtual_mcp_tool calls handle_mcp_tool_call without ever running common_processing_pre_call_logic, so any guardrails configured on the key are bypassed and the tool execution is not spend-logged when clients connect over the streamable-HTTP protocol endpoint. The REST path handles this correctly, so the gap is specific to one transport. Until the SSE path is brought into parity, mcp_tool_call behaves differently depending on how the client connects.

server.py — the _dispatch_virtual_mcp_tool function and its call site inside mcp_server_tool_call need a pre-call pipeline setup equivalent to what call_tool_rest_api does.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/tool_search.py	New module implementing search_tools, get_virtual_tool_definitions, handle_mcp_tool_search, and handle_mcp_tool_call; top-level MCP import lacks a guard, and litellm_logging_obj is omitted on the SSE dispatch path
litellm/proxy/_experimental/mcp_server/server.py	Adds _dispatch_virtual_mcp_tool and _capture_host_progress_callback; virtual tool call via SSE path omits litellm_logging_obj (no pre-call pipeline), and top_k coercion lacks error handling
litellm/proxy/_experimental/mcp_server/rest_endpoints.py	Early-return for list_tool_rest_api and virtual tool intercept in call_tool_rest_api are correct; REST path properly runs pre-call pipeline for mcp_tool_call, but top_k coercion can raise unhandled ValueError
litellm/proxy/_types.py	Adds mcp_tool_search_enabled: Optional[bool] = None to LiteLLM_ObjectPermissionBase; straightforward field addition
litellm/models/object_permission.py	Adds mcp_tool_search_enabled to the Pydantic ORM model; matches schema and types changes
litellm-proxy-extras/litellm_proxy_extras/migrations/20260626120000_add_mcp_tool_search_enabled/migration.sql	Additive nullable column migration; no data loss risk
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_tool_search.py	19 tests covering search logic, virtual tool definitions, REST and SSE paths; all mocked, no real network calls

_{Reviews (2): Last reviewed commit: "chore: trigger CI" | Re-trigger Greptile}

[krrish-berri-2]

@greptileai review

[krrish-berri-2]

@greptileai review

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[krrish-berri-2]

@greptileai review

[krrish-berri-2]

@greptileai review