update 02 staging PR

[Sameerlite]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 3, 2026 11:38am

[greptile-apps]

Greptile Overview

Greptile Summary

This staging PR combines several significant changes:

Major Changes:

• MCP Semantic Tool Filter: New feature that semantically filters MCP tools before LLM calls using embeddings to reduce context window size. Includes pre-call hook, tool expansion logic, and response headers for filter stats.
• Access Group Filtering Removal: Completely reverts the access group filtering feature (issue [Bug]: CRITICAL - Models loadbalanced outside of Model Access Group #18333) that prevented cross-team load balancing. Removes filter_deployments_by_access_groups() function, related metadata extraction, and all associated tests.
• Database Migration Deletion: Removes the migration file that added indexes to LiteLLM_VerificationToken table, which could cause migration state issues in production.
• Key Management: Adds new /key/{key}/reset_spend endpoint allowing proxy/team admins to reset key spend to lower values with validation.
• Fallback Logic: Simplifies fallback exhaustion check by removing redundant model group length validation.
• UI Refactoring: Extracts community engagement buttons (Slack/GitHub) into separate component.

Key Concerns:

• The deleted migration file could break deployed instances that already ran it
• Semantic filter initialization happens during startup (violates performance rule about avoiding expensive startup operations)
• Removing access group filtering removes a security feature without explanation
• Good test coverage for new features (700+ lines of tests for reset_spend endpoint, comprehensive MCP semantic filter tests)

Confidence Score: 3/5

• This PR has some significant risks including migration deletion and feature removal that need careful review before merging
• Score reflects: (1) deleted migration file that could break production deployments, (2) complete removal of access group filtering security feature without documented justification, (3) startup initialization that may cause performance issues per custom rules, (4) good test coverage for new features partially offsets concerns
• Critical attention needed for migration file deletion (litellm-proxy-extras/litellm_proxy_extras/migrations/20260129103648_add_verificationtoken_indexes/migration.sql) and semantic filter startup initialization (litellm/proxy/proxy_server.py). Review removal of access group filtering (litellm/router_utils/common_utils.py, litellm/router.py)

Important Files Changed

Filename	Overview
litellm/router_utils/common_utils.py	Removed filter_deployments_by_access_groups function and its usage - reverting feature #18333
litellm/router.py	Removed access group filtering from router, removing cross-team load balancing protection
litellm/proxy/litellm_pre_call_utils.py	Removed access group metadata extraction for keys and teams from request processing
litellm/proxy/proxy_server.py	Added semantic tool filter initialization during proxy startup, creating Router object in startup path
litellm/proxy/hooks/mcp_semantic_filter/hook.py	New pre-call hook for semantic filtering of MCP tools using embeddings and semantic similarity
litellm/proxy/management_endpoints/key_management_endpoints.py	Added new /key/{key}/reset_spend endpoint with validation logic for resetting key spend to lower values
litellm/router_utils/fallback_event_handlers.py	Simplified fallback depth check by removing redundant model group length validation
litellm-proxy-extras/litellm_proxy_extras/migrations/20260129103648_add_verificationtoken_indexes/migration.sql	Deleted migration file that added database indexes - potentially breaking for deployed instances

Sequence Diagram

sequenceDiagram
    participant Client
    participant ProxyServer
    participant SemanticFilterHook
    participant SemanticMCPToolFilter
    participant Router as LiteLLM Router
    participant LLM as LLM Provider

    Note over ProxyServer: Startup Phase
    ProxyServer->>SemanticFilterHook: initialize_from_config()
    SemanticFilterHook->>SemanticMCPToolFilter: new(embedding_model, router, top_k)
    SemanticMCPToolFilter->>SemanticMCPToolFilter: build_router_from_mcp_registry()
    Note over SemanticMCPToolFilter: Fetches all MCP tools<br/>and builds semantic index
    SemanticFilterHook-->>ProxyServer: hook registered

    Note over Client,LLM: Request Phase
    Client->>ProxyServer: POST /chat/completions<br/>{messages, tools: [...]}
    ProxyServer->>SemanticFilterHook: async_pre_call_hook(data)
    
    alt MCP tools with litellm_proxy reference
        SemanticFilterHook->>SemanticFilterHook: _expand_mcp_tools()
        Note over SemanticFilterHook: Expands MCP references<br/>to full tool definitions
    end
    
    SemanticFilterHook->>SemanticFilterHook: extract_user_query(messages)
    SemanticFilterHook->>SemanticMCPToolFilter: filter_tools(query, tools)
    SemanticMCPToolFilter->>Router: aembedding(query)
    Router->>LLM: Generate embedding
    LLM-->>Router: embedding vector
    Router-->>SemanticMCPToolFilter: query embedding
    SemanticMCPToolFilter->>SemanticMCPToolFilter: Calculate similarity scores
    SemanticMCPToolFilter-->>SemanticFilterHook: filtered_tools (top_k)
    
    SemanticFilterHook->>SemanticFilterHook: Update data["tools"]<br/>Add filter stats to metadata
    SemanticFilterHook-->>ProxyServer: modified data
    
    ProxyServer->>LLM: completion(messages, filtered_tools)
    LLM-->>ProxyServer: response
    
    ProxyServer->>SemanticFilterHook: async_post_call_response_headers_hook()
    SemanticFilterHook-->>ProxyServer: headers with filter stats
    ProxyServer-->>Client: response + headers<br/>(x-litellm-semantic-filter)

Loading

[greptile-apps]

_{8 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

The semantic filter initialization occurs during startup and calls build_router_from_mcp_registry() which may involve network calls to fetch tools from MCP servers. This violates the custom rule about avoiding expensive operations during startup. Consider lazy initialization on first use instead.

Context Used: Rule from dashboard - What: Avoid creating new database requests or Router objects in the critical request path.

Why: Cre... (source)

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review.
Path: litellm/proxy/proxy_server.py
Line: 796:809

Comment:
The semantic filter initialization occurs during startup and calls `build_router_from_mcp_registry()` which may involve network calls to fetch tools from MCP servers. This violates the custom rule about avoiding expensive operations during startup. Consider lazy initialization on first use instead.

**Context Used:** Rule from `dashboard` - What: Avoid creating new database requests or Router objects in the critical request path.

Why: Cre... ([source](https://app.greptile.com/review/custom-context?memory=0c2a17ad-5f29-423f-a48b-371852ac4169))

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

[greptile-apps]

Additional Comments (1)

litellm-proxy-extras/litellm_proxy_extras/migrations/20260129103648_add_verificationtoken_indexes/migration.sql
Deleting a migration file is dangerous for production deployments. If this migration was already applied to deployed databases, removing it from the codebase can cause migration state inconsistencies. Consider reverting the indexes with a new DOWN migration instead of deleting the file.

If these indexes were causing issues, the proper approach is:

1. Keep this migration file
2. Create a new migration that drops the indexes
3. Document why they were removed

Prompt To Fix With AI

This is a comment left during a code review.
Path: litellm-proxy-extras/litellm_proxy_extras/migrations/20260129103648_add_verificationtoken_indexes/migration.sql
Line: 1:8

Comment:
Deleting a migration file is dangerous for production deployments. If this migration was already applied to deployed databases, removing it from the codebase can cause migration state inconsistencies. Consider reverting the indexes with a new DOWN migration instead of deleting the file.

If these indexes were causing issues, the proper approach is:
1. Keep this migration file
2. Create a new migration that drops the indexes
3. Document why they were removed

How can I resolve this? If you propose a fix, please make it concise.