fix: search tools not found when using per-request routers#19818
Merged
yuneng-jiang merged 1 commit intoBerriAI:litellm_router_search_fixfrom Jan 31, 2026
Merged
Conversation
**Root Cause:** When API keys or teams have router_settings configured, the proxy creates per-request Router instances from user_config. These new routers were missing search_tools from the main router, causing "search tool not found" errors despite search_tools being configured. **The Fix:** 1. **common_request_processing.py (lines 554-556):** Pass search_tools from main router to user_config so per-request routers inherit them 2. **proxy_server.py (line 3171):** Remove `if len(_model_list) > 0` check to allow router creation with empty model list (needed for search-tools-only use case) 3. **proxy_server.py (line 746):** Remove redundant search_tools loading code (already handled by _init_search_tools_in_db() called during startup)
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
mubashir1osmani
approved these changes
Jan 27, 2026
3 tasks
1 task
Member
|
The approach of creating a new router instance by key/team seems brittle (and also has poor performance as it initializes a new client on each request). @yuneng-jiang can you please take over this PR and come up with a better approach to team/key settings, where they just override global values instead of creating a separate instance per key/team? |
yuneng-jiang
merged commit 93383d0
into
BerriAI:litellm_router_search_fix
4 of 7 checks passed
shin-bot-litellm
added a commit
that referenced
this pull request
Feb 7, 2026
Previously, PR #19818 (via #20205) removed the model_list check entirely, causing Router to be created even with no models AND no search_tools. This fix adds back a conditional check that creates the Router only when: - There are models to route, OR - There are search_tools configured This preserves the PR #19818 goal (search-tools-only deployments) while avoiding unnecessary Router creation when there's nothing to route. Fixes test_add_and_delete_deployments[0-None]
ishaan-jaff
pushed a commit
that referenced
this pull request
Feb 7, 2026
) Previously, PR #19818 (via #20205) removed the model_list check entirely, causing Router to be created even with no models AND no search_tools. This fix adds back a conditional check that creates the Router only when: - There are models to route, OR - There are search_tools configured This preserves the PR #19818 goal (search-tools-only deployments) while avoiding unnecessary Router creation when there's nothing to route. Fixes test_add_and_delete_deployments[0-None]
7 tasks
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root Cause:
When API keys or teams have router_settings configured, the proxy creates per-request Router instances from user_config. These new routers were missing search_tools from the main router, causing "search tool not found" errors despite search_tools being configured.
The Fix:
common_request_processing.py (lines 554-556): Pass search_tools from main router to user_config so per-request routers inherit them
proxy_server.py (line 3171): Remove
if len(_model_list) > 0check to allow router creation with empty model list (needed for search-tools-only use case)proxy_server.py (line 746): Remove redundant search_tools loading code (already handled by _init_search_tools_in_db() called during startup)
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🐛 Bug Fix