fix: MCP semantic filter regression — top_k not limiting tools

[shin-bot-litellm]

Regression Fix

Failing Jobs: mcp_testing, litellm_mapped_tests_proxy
Caused By: PR #20296 (MCP Semantic Filtering)

What Broke

Semantic filter top_k not limiting tools — all tools returned instead of top_k subset. Two bugs:

1. filter_tools() never built the semantic router — when tool_router was None (tools passed directly, not via MCP registry startup), the method returned all tools unfiltered instead of lazily building the router from available_tools.

2. async_pre_call_hook() KeyError on missing metadata — when the request data dict had no metadata key, writing filter stats crashed with KeyError, causing the exception handler to return None (no filtering applied).

This Fix

• Lazy router initialization: filter_tools() now builds the semantic router from available_tools on first call if not pre-built via build_router_from_mcp_registry().
• Safe metadata access: async_pre_call_hook() initializes the metadata dict if missing before writing filter stats.

Testing

All 7 semantic filter tests pass locally:

• test_semantic_filter_basic_filtering ✅
• test_semantic_filter_top_k_limiting ✅
• test_semantic_filter_hook_triggers_on_completion ✅
• test_semantic_filter_disabled ✅
• test_semantic_filter_empty_tools ✅
• test_semantic_filter_extract_user_query ✅
• test_semantic_filter_hook_skips_no_tools ✅

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 3, 2026 4:34am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes two regression bugs from PR #20296 that broke the MCP semantic filtering feature:

Bug 1: top_k not limiting tools — The filter_tools() method returned all tools unfiltered when tool_router was None (i.e., when tools were passed directly instead of from the MCP registry). The fix adds lazy router initialization, building the semantic router from available_tools on first call.

Bug 2: KeyError crash on missing metadata — The hook crashed when the request data dict had no metadata key, causing the exception handler to return None (no filtering applied). The fix defensively initializes the metadata dict before writing filter stats.

Both fixes are logically sound and align with the PR description. All 7 semantic filter tests pass locally.

Confidence Score: 3/5

• PR fixes critical bugs but introduces lazy initialization in the request path which may impact performance
• The fixes address real bugs and tests pass, but lazy router initialization on first request violates the custom rule about avoiding object creation in the critical path (rule 0c2a17ad). The performance impact depends on whether startup initialization succeeds.
• Pay attention to semantic_tool_filter.py — lazy router initialization may add latency to first request

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/semantic_tool_filter.py	Adds lazy router initialization when tool_router is None, fixing regression where top_k parameter was not limiting tools. Performance concern: creates SemanticRouter on first request if not pre-built.
litellm/proxy/hooks/mcp_semantic_filter/hook.py	Safely initializes metadata dict if missing before writing filter stats, preventing KeyError crash. Clean defensive programming fix.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Hook as SemanticToolFilterHook
    participant Filter as SemanticMCPToolFilter
    participant Router as SemanticRouter
    participant LLM as LiteLLM Router

    Client->>Hook: async_pre_call_hook(data with tools)
    Hook->>Hook: Check if tools present
    Hook->>Hook: Check if metadata exists
    Note over Hook: Fix: Initialize metadata dict if missing
    Hook->>Filter: extract_user_query(messages)
    Filter-->>Hook: user_query
    Hook->>Filter: filter_tools(query, available_tools)
    
    alt Router not initialized (first call)
        Filter->>Filter: Lazy build router
        Note over Filter: Regression fix: Build router from available_tools
        Filter->>Router: SemanticRouter(routes, encoder)
        Router->>LLM: Generate embeddings
        LLM-->>Router: Embeddings
        Router-->>Filter: Router initialized
    else Router already initialized
        Note over Filter: Router pre-built on startup
    end
    
    Filter->>Router: semantic_router(query, limit=top_k)
    Router-->>Filter: Top K matched tool names
    Filter->>Filter: Get tools by names from available_tools
    Filter-->>Hook: Filtered tools (limited by top_k)
    
    Hook->>Hook: Store filter stats in metadata
    Hook-->>Client: Modified data with filtered tools

Loading

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Lazy router initialization happens in the critical request path on first call — creating SemanticRouter with embeddings can add significant latency. Consider pre-building the router during startup via build_router_from_mcp_registry() or add a startup check/warning if the router isn't initialized.

Context Used: Rule from dashboard - What: Avoid creating new database requests or Router objects in the critical request path.

Why: Cre... (source)

Prompt To Fix With AI

This is a comment left during a code review.
Path: litellm/proxy/_experimental/mcp_server/semantic_tool_filter.py
Line: 175:188

Comment:
Lazy router initialization happens in the critical request path on first call — creating `SemanticRouter` with embeddings can add significant latency. Consider pre-building the router during startup via `build_router_from_mcp_registry()` or add a startup check/warning if the router isn't initialized.

**Context Used:** Rule from `dashboard` - What: Avoid creating new database requests or Router objects in the critical request path.

Why: Cre... ([source](https://app.greptile.com/review/custom-context?memory=0c2a17ad-5f29-423f-a48b-371852ac4169))

How can I resolve this? If you propose a fix, please make it concise.