[Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet#20607
Merged
ishaan-jaff merged 19 commits intomainfrom Feb 7, 2026
Merged
[Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet#20607ishaan-jaff merged 19 commits intomainfrom
ishaan-jaff merged 19 commits intomainfrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile OverviewGreptile Summary
Confidence Score: 2/5
|
| Filename | Overview |
|---|---|
| CLAUDE.md | Adds style guidance to avoid inline imports (module-level imports preferred). |
| litellm/proxy/_experimental/mcp_server/auth/litellm_auth_handler.py | Extends MCPAuthenticatedUser to store client_ip in auth context. |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Adds available_on_public_internet flag support and IP-based filtering helpers; introduces request-path inline import of proxy_server settings that can break MCP gating/cause circular import. |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | Applies IP filtering to REST MCP server/tool listing; single-server branch still authorizes against pre-filtered set allowing external callers to access private servers by ID. |
| litellm/proxy/_experimental/mcp_server/server.py | Plumbs client IP into MCP auth context and applies IP filtering when listing allowed servers; still has minor risk if other call sites don't pass client_ip when looking up servers by name. |
| litellm/proxy/auth/ip_address_utils.py | Introduces IPAddressUtils to classify internal IPs and extract client IP using proxy settings; uses inline import of proxy_server.general_settings which can fail when proxy_server import isn't safe. |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Filters registry.json results based on client IP, but does so by importing IPAddressUtils inside the endpoint (inline import) and depends on proxy_server settings access. |
| litellm/types/mcp_server/mcp_server_manager.py | Adds available_on_public_internet boolean field to MCPServer pydantic model. |
| tests/test_litellm/proxy/auth/test_mcp_ip_filtering.py | Adds unit tests for internal/public IP classification and server ID filtering behavior. |
Sequence Diagram
sequenceDiagram
autonumber
participant Client
participant Proxy as LiteLLM Proxy (FastAPI)
participant IPUtils as IPAddressUtils
participant AuthCtx as MCP auth_context_var
participant Manager as MCPServerManager
participant Settings as proxy_server.general_settings
Client->>Proxy: GET /mcp-rest/tools/list (or POST /mcp-rest/tools/call)
Proxy->>IPUtils: get_mcp_client_ip(request)
IPUtils->>Settings: read use_x_forwarded_for, mcp_internal_ip_ranges
IPUtils-->>Proxy: client_ip
Proxy->>Manager: get_allowed_mcp_servers(user_api_key_auth)
Manager-->>Proxy: allowed_server_ids
Proxy->>Manager: filter_server_ids_by_ip(allowed_server_ids, client_ip)
Manager->>Manager: _is_server_accessible_from_ip(server, client_ip)
Manager->>Settings: read mcp_internal_ip_ranges
Manager-->>Proxy: filtered_server_ids
Proxy-->>Client: tools/allowed servers response
Note over Proxy,AuthCtx: For /mcp (streamable-http/SSE) routes
Client->>Proxy: /mcp streamable-http or SSE request
Proxy->>IPUtils: get_mcp_client_ip(StarletteRequest(scope))
Proxy->>AuthCtx: set_auth_context(..., client_ip)
Proxy->>Manager: _list_mcp_tools() / allowed-server filtering uses client_ip
Proxy-->>Client: MCP response
Contributor
Additional Comments (1)
In The membership check should be against the post-filtered set/list (e.g., |
Member
Author
|
@greptile-apps review this again |
Contributor
Greptile OverviewGreptile Summary
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| CLAUDE.md | Adds a style-guide note discouraging inline imports inside methods; no runtime impact. |
| litellm/proxy/_experimental/mcp_server/auth/litellm_auth_handler.py | Extends MCPAuthenticatedUser to carry client_ip through auth context; straightforward change. |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Implements IP-based filtering for MCP server visibility and adds lazy import of proxy_server general_settings; introduces request-path dependency on proxy_server and potential auth bypass when client_ip missing. |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | Applies IP filtering to REST tool list/call endpoints using derived client IP; duplicates header extraction logic in one route. |
| litellm/proxy/_experimental/mcp_server/server.py | Plumbs client_ip into auth context and filters allowed server IDs in _list_mcp_tools; client_ip extraction depends on request scope and may be absent in some paths. |
| litellm/proxy/auth/ip_address_utils.py | Adds new IPAddressUtils helper for internal network parsing and client IP extraction; imports proxy_server.general_settings lazily when settings not provided. |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Filters registry.json output based on client IP; introduces inline import of IPAddressUtils in endpoint handler. |
| litellm/types/mcp_server/mcp_server_manager.py | Adds available_on_public_internet boolean to MCPServer pydantic model; safe schema extension. |
| tests/test_litellm/proxy/auth/test_mcp_ip_filtering.py | Adds unit tests for IP classification and server filtering; uses patching of proxy_server.general_settings to avoid imports during tests. |
Sequence Diagram
sequenceDiagram
participant Client
participant Proxy as LiteLLM Proxy
participant IPU as IPAddressUtils
participant MSM as MCPServerManager
participant MCP as Upstream MCP Server
Client->>Proxy: GET /mcp-rest/tools/list (or /mcp tools)
Proxy->>IPU: get_mcp_client_ip(request)
IPU-->>Proxy: client_ip
Proxy->>MSM: get_allowed_mcp_servers(auth_context)
MSM-->>Proxy: allowed_server_ids
Proxy->>MSM: filter_server_ids_by_ip(allowed_server_ids, client_ip)
MSM->>MSM: _is_server_accessible_from_ip(server, client_ip)
MSM->>IPU: parse_internal_networks(general_settings.mcp_internal_ip_ranges)
MSM->>IPU: is_internal_ip(client_ip, networks)
IPU-->>MSM: true/false
MSM-->>Proxy: filtered_server_ids
Proxy->>MCP: list_tools/call_tool (only for filtered servers)
MCP-->>Proxy: tools/result
Proxy-->>Client: response
Contributor
Additional Comments (3)
OAuth discovery routes ( Also appears in: |
Contributor
Greptile OverviewGreptile Summary
Confidence Score: 2/5
|
| Filename | Overview |
|---|---|
| CLAUDE.md | No functional code changes relevant to MCP filtering; documentation/guidance update only. |
| litellm/proxy/_experimental/mcp_server/auth/litellm_auth_handler.py | Adds request-IP based post-filtering of allowed MCP servers; potential correctness depends on which request IP source is passed in. |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Implements available_on_public_internet filtering across MCP access paths; risk of using request-derived IP without trusted proxy constraints and potential runtime issues in new access checks. |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | Applies IP-based server filtering to REST MCP endpoints; correctness depends on consistent IP extraction and passing through request context. |
| litellm/proxy/_experimental/mcp_server/server.py | Wires IP filtering into dynamic MCP routes/tool listing; verify that all call sites supply the same client IP semantics. |
| litellm/proxy/auth/ip_address_utils.py | Introduces IP classification + XFF parsing; security depends on trusted proxy assumptions—if XFF is honored without a trust boundary, external callers can spoof private IPs. |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Adds IP filtering to management endpoints; ensure it does not rely on unsafe header-derived client IP for access decisions. |
| litellm/types/mcp_server/mcp_server_manager.py | Extends MCP server config schema/types to include available_on_public_internet; low risk. |
| tests/test_litellm/proxy/auth/test_mcp_ip_filtering.py | Adds tests for private/public IP filtering and XFF parsing; may not cover spoofing/trusted-proxy scenarios. |
Sequence Diagram
sequenceDiagram
participant Client
participant Proxy as LiteLLM Proxy
participant IPU as IPAddressUtils
participant MSM as MCPServerManager
participant MCP as MCP Server
Client->>Proxy: Request (headers + remote addr)
Proxy->>IPU: extract_client_ip(request)
IPU-->>Proxy: client_ip + is_internal?
Proxy->>MSM: get_allowed_mcp_servers(user)
MSM->>MSM: filter_server_ids_by_ip(client_ip)
MSM-->>Proxy: filtered server/tool list
Proxy->>MCP: tool call / registry discovery (if allowed)
MCP-->>Proxy: response
Proxy-->>Client: response
Contributor
Additional Comments (1)
|
Member
Author
|
@greptile review again pls |
Contributor
Greptile OverviewGreptile Summary
Confidence Score: 2/5
|
| Filename | Overview |
|---|---|
| CLAUDE.md | Adds a brief instruction; no functional impact. |
| litellm/proxy/_experimental/mcp_server/auth/litellm_auth_handler.py | Extends MCPAuthenticatedUser with client_ip field for downstream filtering; no obvious breakage. |
| litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py | Adds client IP extraction and passes it into server lookup for OAuth discovery/authorize/token routes; access-denied now indistinguishable from not-found (404) for hidden servers. |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Implements available_on_public_internet flag and IP-based filtering helpers; introduces request-path lazy import of proxy_server general_settings (previously flagged) and server-name lookup ignores aliases. |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | Applies IP filtering to REST list/call tool endpoints using extracted client IP; logic appears consistent with manager filtering. |
| litellm/proxy/_experimental/mcp_server/server.py | Propagates client_ip via auth context and enforces IP filtering; but handle_streamable_http_mcp exception handler now unconditionally re-raises (dead code after), breaking intended graceful error responses. |
| litellm/proxy/auth/ip_address_utils.py | Adds IP parsing/classification + client IP extraction; relies on use_x_forwarded_for without trusted-proxy boundary (previously flagged) and does lazy proxy_server import in request path (previously flagged). |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Filters MCP registry results by client IP via get_filtered_registry(); import of IPAddressUtils inside request handler mirrors existing patterns. |
| litellm/proxy/proxy_server.py | Dynamic MCP route now applies IP-based visibility via get_mcp_client_ip + get_mcp_server_by_name(client_ip=...); access-denied will surface as 404. |
| litellm/types/mcp_server/mcp_server_manager.py | Adds available_on_public_internet field to MCPServer model; straightforward schema extension. |
| tests/test_litellm/proxy/auth/test_mcp_ip_filtering.py | Adds unit tests for internal/public IP classification and manager filtering; does not cover streamable HTTP error-path regression. |
Sequence Diagram
sequenceDiagram
autonumber
participant Client
participant Proxy as LiteLLM Proxy
participant IPU as IPAddressUtils
participant Mgr as MCPServerManager
participant Ctx as auth_context_var
participant Sess as MCP Session Manager
Client->>Proxy: MCP request (StreamableHTTP/SSE)
Proxy->>Proxy: extract_mcp_auth_context(scope,path)
Proxy->>IPU: get_mcp_client_ip(request)
IPU-->>Proxy: client_ip
Proxy->>Ctx: set_auth_context(..., client_ip)
alt list tools / call tool
Proxy->>Proxy: _get_allowed_mcp_servers()
Proxy->>Ctx: get() (read client_ip)
Proxy->>Mgr: get_allowed_mcp_servers(user_api_key_auth)
Mgr-->>Proxy: allowed_server_ids
Proxy->>Mgr: filter_server_ids_by_ip(allowed_server_ids, client_ip)
Mgr->>IPU: is_internal_ip(client_ip, internal_networks)
IPU-->>Mgr: internal? / external?
Mgr-->>Proxy: filtered_server_ids
Proxy->>Sess: handle_request(...)
Sess-->>Proxy: response
end
alt discovery / dynamic routes
Client->>Proxy: GET /.well-known/... or /mcp/{server}
Proxy->>IPU: get_mcp_client_ip(request)
IPU-->>Proxy: client_ip
Proxy->>Mgr: get_mcp_server_by_name(server_name, client_ip)
Mgr->>IPU: is_internal_ip(client_ip,...)
Mgr-->>Proxy: MCPServer|None
Proxy-->>Client: 200 or 404
end
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+2287
to
2290
| def get_mcp_server_by_name( | ||
| self, server_name: str, client_ip: Optional[str] = None | ||
| ) -> Optional[MCPServer]: | ||
| """ |
Contributor
There was a problem hiding this comment.
Server alias lookup regression
get_mcp_server_by_name() only matches server.server_name == server_name and ignores alias even though servers can be addressed via alias elsewhere. After this PR, multiple request paths (e.g., OAuth endpoints and dynamic MCP routes) rely on get_mcp_server_by_name(..., client_ip=...), so any client using an alias will now get a 404 even if the server exists and is allowed.
Member
Author
There was a problem hiding this comment.
what do you need me to change / edit @greptile ?
Contributor
There was a problem hiding this comment.
You need to add alias matching to get_mcp_server_by_name(). Change:
Suggested change
| def get_mcp_server_by_name( | |
| self, server_name: str, client_ip: Optional[str] = None | |
| ) -> Optional[MCPServer]: | |
| """ | |
| def get_mcp_server_by_name( | |
| self, server_name: str, client_ip: Optional[str] = None | |
| ) -> Optional[MCPServer]: | |
| """ | |
| Get an MCP server by name or alias, with optional IP filtering. | |
| """ | |
| registry = self.get_registry() | |
| for server in registry.values(): | |
| if server.server_name == server_name or server_name in (server.alias or []): | |
| if client_ip and not IPAddressUtils.is_private_ip(client_ip): | |
| if not server.available_on_public_internet: | |
| return None | |
| return server | |
| return None |
This preserves the existing alias lookup behavior while adding IP filtering.
Member
Author
|
@greptile review again |
Contributor
Greptile OverviewGreptile SummaryThis PR adds IP-based access control for MCP servers, allowing admins to mark servers as public or private via the Key Changes:
Critical Issues:
Regression from previous feedback: Confidence Score: 1/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/ip_address_utils.py | New IP classification utility with critical XFF trust boundary vulnerability allowing external clients to spoof internal IPs |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Adds IP filtering logic and filter_server_ids_by_ip() method; circular import in _get_general_settings() called on request path |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | Applies IP filtering to REST tool listing and tool calling endpoints; refactored header extraction into helper |
| litellm/proxy/_experimental/mcp_server/server.py | Extracts client IP in SSE/HTTP MCP handlers and threads it through auth context; removed error re-raise losing client visibility |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Registry endpoint now uses get_filtered_registry(client_ip) to hide non-public servers from external IPs |
Sequence Diagram
sequenceDiagram
participant Client
participant FastAPI
participant IPAddressUtils
participant MCPServerManager
participant MCPServer
Client->>FastAPI: HTTP Request (with XFF header)
FastAPI->>IPAddressUtils: get_mcp_client_ip(request)
IPAddressUtils->>IPAddressUtils: Extract from XFF or client.host
IPAddressUtils-->>FastAPI: client_ip
FastAPI->>MCPServerManager: get_allowed_mcp_servers(user_auth)
MCPServerManager-->>FastAPI: allowed_server_ids[]
FastAPI->>MCPServerManager: filter_server_ids_by_ip(server_ids, client_ip)
MCPServerManager->>IPAddressUtils: is_internal_ip(client_ip)
IPAddressUtils-->>MCPServerManager: is_internal=true/false
alt Internal IP
MCPServerManager-->>FastAPI: All servers (public + private)
else External IP
MCPServerManager->>MCPServer: Check available_on_public_internet
MCPServerManager-->>FastAPI: Only public servers
end
FastAPI-->>Client: Filtered server list/tools
Comment on lines
+83
to
+101
| Uses the proxy's use_x_forwarded_for setting to determine whether | ||
| to trust X-Forwarded-For headers. | ||
|
|
||
| Args: | ||
| request: FastAPI request object | ||
| general_settings: Optional settings dict. If not provided, imports from proxy_server. | ||
| """ | ||
| if general_settings is None: | ||
| try: | ||
| from litellm.proxy.proxy_server import ( | ||
| general_settings as proxy_general_settings, | ||
| ) | ||
| general_settings = proxy_general_settings | ||
| except ImportError: | ||
| # Fallback if proxy_server not available (e.g., MCP gated deployments) | ||
| general_settings = {} | ||
|
|
||
| use_xff = general_settings.get("use_x_forwarded_for", False) | ||
| return _get_request_ip_address(request, use_x_forwarded_for=use_xff) |
Contributor
There was a problem hiding this comment.
XFF parsing without trust boundary creates bypass risk. External clients can spoof X-Forwarded-For: 10.0.0.1 and gain internal-only MCP server access. Only parse XFF when request.client.host is a trusted proxy (e.g., loadbalancer IP allowlist), otherwise use socket peer IP.
The existing use_x_forwarded_for flag doesn't validate trust - it's either always-trust or never-trust globally. Need per-request trust validation based on immediate peer.
Comment on lines
1988
to
1989
| except Exception as e: | ||
| raise e | ||
| verbose_logger.exception(f"Error handling MCP request: {e}") |
Contributor
There was a problem hiding this comment.
Error swallowed without re-raising. This was previously raise e but now just logs. Clients won't get proper HTTP error responses on failures.
Contributor
Additional Comments (1)
Pass |
Member
Author
|
@greptile review again, if u need changes pls be clear on what you want me to change |
Contributor
Greptile OverviewGreptile Summary
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| CLAUDE.md | Adds style guidance discouraging inline imports; no functional code changes. |
| litellm/proxy/_experimental/mcp_server/auth/litellm_auth_handler.py | Extends MCPAuthenticatedUser to carry client_ip for downstream filtering; appears consistent with auth context usage. |
| litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py | OAuth endpoints now resolve MCP server with client_ip-based filtering via IPAddressUtils; relies on IP extraction working safely in request context. |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Adds available_on_public_internet flag, IP-based filtering helpers, and filtered registry/name lookup. Introduces lazy import of proxy_server settings in request path which can still trigger ImportError/circular imports depending on environment. |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | Applies IP-based server filtering to tool listing/calling and refactors header extraction. Behavior change depends on client_ip extraction and filtering correctness. |
| litellm/proxy/_experimental/mcp_server/server.py | Threads client_ip through MCP request handling and filters allowed servers; changes exception handling to re-raise HTTPException and JSON-500 others (could alter error propagation). |
| litellm/proxy/auth/ip_address_utils.py | Introduces IP extraction/classification for MCP access control with optional trusted proxy ranges; still imports proxy_server.general_settings inside request helper and ignores the function parameter, risking ImportError/circular import in non-proxy contexts. |
| litellm/proxy/management_endpoints/mcp_management_endpoints.py | Registry endpoint now filters returned servers by client_ip; adds inline import of IPAddressUtils and depends on safe client_ip extraction in management request path. |
| litellm/proxy/proxy_server.py | Dynamic MCP route now applies IP-based filtering via get_mcp_server_by_name(..., client_ip=...); depends on correct alias matching and IP extraction. |
| litellm/types/mcp_server/mcp_server_manager.py | Adds available_on_public_internet field to MCPServer model; straightforward schema extension. |
| tests/test_litellm/proxy/auth/test_mcp_ip_filtering.py | Adds unit tests covering IP classification and server filtering; tests patch proxy_server.general_settings which may not reflect real runtime import behavior. |
Sequence Diagram
sequenceDiagram
autonumber
participant Client
participant Proxy as LiteLLM Proxy (FastAPI/Starlette)
participant IPU as IPAddressUtils
participant MSM as global_mcp_server_manager
participant AS as Auth (get_allowed_mcp_servers)
Client->>Proxy: Request (list tools / call tool / registry / dynamic route)
Proxy->>IPU: get_mcp_client_ip(request)
IPU-->>Proxy: client_ip (direct or XFF-derived)
Proxy->>AS: build_effective_auth_contexts(user key)
AS-->>Proxy: auth_contexts
Proxy->>MSM: get_allowed_mcp_servers(auth_context)
MSM-->>Proxy: allowed_server_ids
Proxy->>MSM: filter_server_ids_by_ip(allowed_server_ids, client_ip)
MSM->>IPU: is_internal_ip(client_ip, internal_networks)
IPU-->>MSM: internal? (true/false)
MSM-->>Proxy: filtered_server_ids
Proxy->>MSM: get_mcp_server_by_name(name, client_ip) / get_filtered_registry(client_ip)
MSM-->>Proxy: server(s) visible for caller
Proxy-->>Client: Response limited by available_on_public_internet
Comment on lines
+104
to
+110
| addr = ipaddress.ip_address(client_ip.strip()) | ||
| except ValueError: | ||
| return False | ||
|
|
||
| return any(addr in network for network in networks) | ||
|
|
||
| @staticmethod |
Contributor
There was a problem hiding this comment.
Unused settings parameter
get_mcp_client_ip(..., general_settings=...) ignores its general_settings argument and unconditionally does from litellm.proxy.proxy_server import general_settings, which will break callers that try to supply settings (and also reintroduces the request-path import/circular-import issue this PR is trying to avoid). Consider using the passed-in general_settings when provided and only falling back to the proxy_server import when it’s None.
Suggested change
| addr = ipaddress.ip_address(client_ip.strip()) | |
| except ValueError: | |
| return False | |
| return any(addr in network for network in networks) | |
| @staticmethod | |
| settings = general_settings | |
| if settings is None: | |
| from litellm.proxy.proxy_server import general_settings as settings | |
| use_xff = settings.get("use_x_forwarded_for", False) |
Member
Author
There was a problem hiding this comment.
@greptile but we follow the pattern of from litellm.proxy.proxy_server import general_settings through the codebase to ensure it's always in sync with general settings
Contributor
There was a problem hiding this comment.
You're correct—that pattern is standard throughout the codebase. The get_mcp_client_ip function includes general_settings: Optional[Dict[str, Any]] = None as a parameter but unconditionally imports from proxy_server and ignores the passed-in value. This is consistent with how other utilities in litellm/proxy/auth/ handle general_settings (e.g., auth_utils.py, handle_jwt.py, user_api_key_auth.py all do the same import).
The pattern ensures all functions see the same shared state, so this is intentional and matches the codebase standard. The parameter signature is misleading but harmless—it's just not used. If you want to remove the unused parameter to avoid confusion, that would be a valid cleanup, but the current approach aligns with the established pattern in the codebase.
Comment on lines
+726
to
+734
| user_api_key_auth: Optional[UserAPIKeyAuth], | ||
| mcp_servers: Optional[List[str]], | ||
| ) -> List[MCPServer]: | ||
| """Return allowed MCP servers for a request after applying filters.""" | ||
| """Return allowed MCP servers for a request after applying filters. | ||
|
|
||
| Reads client_ip from the auth context (set by handle_streamable_http_mcp/handle_sse_mcp) | ||
| to enforce IP-based access control on MCP servers. | ||
| """ | ||
| # Extract client_ip from auth context for IP-based filtering |
Contributor
There was a problem hiding this comment.
Filtering skipped without auth context
_get_allowed_mcp_servers() pulls client_ip from auth_context_var, but set_auth_context() is only called in handle_streamable_http_mcp()/handle_sse_mcp(). Any other call sites that invoke _get_allowed_mcp_servers() (e.g., internal helpers/tests/future endpoints) will have client_ip=None and therefore skip IP filtering (filter_server_ids_by_ip returns input unchanged). If IP filtering is meant to be enforced for all MCP access paths, consider explicitly passing client_ip into _get_allowed_mcp_servers() from request handlers instead of relying on ambient context.
Contributor
Additional Comments (1)
|
Member
Author
|
@greptile review again, if not satisified tell me what you want/need me to change |
Contributor
Greptile OverviewGreptile SummaryThis PR adds IP-based access control for MCP servers, allowing admins to mark servers as private (internal network only) or public (internet-accessible). The implementation addresses all critical security concerns from previous review rounds. Key changes:
Configuration: general_settings:
use_x_forwarded_for: true
mcp_trusted_proxy_ranges: ["10.0.0.0/8"] # Only trust XFF from these IPs
mcp_internal_ip_ranges: ["10.0.0.0/8", "192.168.0.0/16"] # Optional custom internal ranges
mcp_servers:
- name: public_server
available_on_public_internet: true # Visible to all
- name: internal_server
available_on_public_internet: false # Only visible from internal IPsSecurity model:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/ip_address_utils.py | New IP utility class with XFF trust validation for MCP access control - addresses previous XFF spoofing concerns |
| litellm/proxy/_experimental/mcp_server/mcp_server_manager.py | Added IP filtering methods and alias matching to get_mcp_server_by_name - addresses previous alias lookup regression |
| litellm/proxy/_experimental/mcp_server/server.py | Integrated client IP extraction and added to auth context, HTTPException now properly re-raised |
| litellm/proxy/_experimental/mcp_server/rest_endpoints.py | REST API endpoints extract client IP and apply filtering correctly |
| litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py | OAuth discovery endpoints extract client IP and filter server lookups by accessibility |
Sequence Diagram
sequenceDiagram
participant Client
participant FastAPI as FastAPI Request Handler
participant IPUtils as IPAddressUtils
participant AuthContext as Auth Context
participant Manager as MCPServerManager
participant Filter as IP Filter
participant Server as MCP Server
Client->>FastAPI: Request to MCP endpoint
FastAPI->>IPUtils: get_mcp_client_ip(request)
alt XFF enabled & trusted proxy
IPUtils->>IPUtils: Validate request.client.host in trusted_proxy_ranges
alt Trusted proxy
IPUtils->>IPUtils: Parse X-Forwarded-For header
IPUtils-->>FastAPI: Return leftmost IP from XFF
else Untrusted source
IPUtils->>IPUtils: Log warning
IPUtils-->>FastAPI: Return request.client.host (ignore XFF)
end
else XFF disabled or no XFF header
IPUtils-->>FastAPI: Return request.client.host
end
FastAPI->>AuthContext: set_auth_context(client_ip=...)
FastAPI->>Manager: get_allowed_mcp_servers(user_api_key_auth)
Manager-->>FastAPI: List of allowed server IDs
FastAPI->>Filter: filter_server_ids_by_ip(server_ids, client_ip)
loop For each server_id
Filter->>Manager: get_mcp_server_by_id(server_id)
Manager-->>Filter: MCPServer object
Filter->>Filter: _is_server_accessible_from_ip(server, client_ip)
alt client_ip is None
Filter->>Filter: Return True (no filtering)
else server.available_on_public_internet = True
Filter->>Filter: Return True (public server)
else Check IP classification
Filter->>IPUtils: is_internal_ip(client_ip, internal_networks)
IPUtils->>IPUtils: Parse IP, check against RFC1918/ULA ranges
IPUtils-->>Filter: True (internal) or False (external)
Filter->>Filter: Return True if internal, False if external
end
end
Filter-->>FastAPI: Filtered list of accessible server IDs
FastAPI->>Server: Execute MCP operation with filtered servers
Server-->>Client: Response
| request: FastAPI request object | ||
| general_settings: Optional settings dict. If not provided, uses cached reference. | ||
| """ | ||
| from litellm.proxy.proxy_server import general_settings |
Contributor
There was a problem hiding this comment.
Unused parameter - general_settings is imported from proxy_server on line 126 but the parameter is never used. Consider removing the parameter or using it when provided (fall back to import only when None).
Collaborator
|
skill issue @ishaan-jaff 😭😭 |
krrishdholakia
added a commit
that referenced
this pull request
Feb 7, 2026
…ng format for Claude Code (#20631) * Add http support to custom code guardrails + Unified guardrails for MCP + Agent guardrail support (#20619) * fix: fix styling * fix(custom_code_guardrail.py): add http support for custom code guardrails allows users to call external guardrails on litellm with minimal code changes (no custom handlers) Test guardrail integrations more easily * feat(a2a/): add guardrails for agent interactions allows the same guardrails for llm's to be applied to agents as well * fix(a2a/): support passing guardrails to a2a from the UI * style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails * feat(mcp/): support custom code guardrails for mcp calls allows custom code guardrails to work on mcp input * feat(chatui.tsx): support guardrails on mcp tool calls on playground * fix(mypy): resolve missing return statements and type casting issues (#20618) * fix(mypy): resolve missing return statements and type casting issues * fix(pangea): use elif to prevent UnboundLocalError and handle None messages Address Greptile review feedback: - Make branches mutually exclusive using elif to prevent input_messages from being overwritten - Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API --------- Co-authored-by: Shin <shin@openclaw.ai> * [Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet (#20607) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * fixing user_id * [Feat] IP-Based Access Control for MCP Servers (#20620) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * test fix * fix linting * inint ui types * add ui for managing MCP private/public * add ui * fixes * add to schema * add types * fix endpoint * add endpoint * update manager * test mcp * dont use external party for ip address * Add OpenAI/Azure release test suite with HTTP client lifecycle regression detection (#20622) * docs (#20626) * docs * fix(mypy): resolve type checking errors in 5 files (#20627) - a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment - caching/redis_cache.py: Add type ignore for async ping return type - caching/redis_cluster_cache.py: Add type ignore for async ping return type - llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text - proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument All changes add appropriate type: ignore comments to handle library typing inconsistencies. * fix(test): update deprecated gemini embedding model (#20621) Replace text-embedding-004 with gemini-embedding-001. The old model was deprecated and returns 404: 'models/text-embedding-004 is not found for API version v1beta' Co-authored-by: Shin <shin@openclaw.ai> * ui new buil * fix(websearch_interception): convert agentic loop response to streaming format when original request was streaming Fixes #20187 - When using websearch_interception in Bedrock with Claude Code: 1. Output tokens were showing as 0 because the agentic loop response wasn't being converted back to streaming format 2. The response from the agentic loop (follow-up request) was returned as a non-streaming dict, but Claude Code expects a streaming response This fix adds streaming format conversion for the agentic loop response when the original request was streaming (detected via the websearch_interception_converted_stream flag in logging_obj). The fix ensures: - Output tokens are correctly included in the message_delta event - stop_reason is properly preserved - The response format matches what Claude Code expects --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Shin <shin@openclaw.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
krrishdholakia
added a commit
that referenced
this pull request
Feb 7, 2026
…iohttp tracing (#20630) * Add http support to custom code guardrails + Unified guardrails for MCP + Agent guardrail support (#20619) * fix: fix styling * fix(custom_code_guardrail.py): add http support for custom code guardrails allows users to call external guardrails on litellm with minimal code changes (no custom handlers) Test guardrail integrations more easily * feat(a2a/): add guardrails for agent interactions allows the same guardrails for llm's to be applied to agents as well * fix(a2a/): support passing guardrails to a2a from the UI * style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails * feat(mcp/): support custom code guardrails for mcp calls allows custom code guardrails to work on mcp input * feat(chatui.tsx): support guardrails on mcp tool calls on playground * fix(mypy): resolve missing return statements and type casting issues (#20618) * fix(mypy): resolve missing return statements and type casting issues * fix(pangea): use elif to prevent UnboundLocalError and handle None messages Address Greptile review feedback: - Make branches mutually exclusive using elif to prevent input_messages from being overwritten - Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API --------- Co-authored-by: Shin <shin@openclaw.ai> * [Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet (#20607) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * fixing user_id * [Feat] IP-Based Access Control for MCP Servers (#20620) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * test fix * fix linting * inint ui types * add ui for managing MCP private/public * add ui * fixes * add to schema * add types * fix endpoint * add endpoint * update manager * test mcp * dont use external party for ip address * Add OpenAI/Azure release test suite with HTTP client lifecycle regression detection (#20622) * docs (#20626) * docs * fix(mypy): resolve type checking errors in 5 files (#20627) - a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment - caching/redis_cache.py: Add type ignore for async ping return type - caching/redis_cluster_cache.py: Add type ignore for async ping return type - llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text - proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument All changes add appropriate type: ignore comments to handle library typing inconsistencies. * fix(test): update deprecated gemini embedding model (#20621) Replace text-embedding-004 with gemini-embedding-001. The old model was deprecated and returns 404: 'models/text-embedding-004 is not found for API version v1beta' Co-authored-by: Shin <shin@openclaw.ai> * ui new buil * fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing When users pass a shared_session with trace_configs to acompletion(), the get_async_httpx_client() function was ignoring it and returning a cached client without the user's tracing configuration. This fix bypasses the cache when shared_session is provided, ensuring the user's ClientSession (with its trace_configs, connector settings, etc.) is actually used for the request. Fixes #20174 --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Shin <shin@openclaw.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com> Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
Sameerlite
pushed a commit
that referenced
this pull request
Feb 12, 2026
…iohttp tracing (#20630) * Add http support to custom code guardrails + Unified guardrails for MCP + Agent guardrail support (#20619) * fix: fix styling * fix(custom_code_guardrail.py): add http support for custom code guardrails allows users to call external guardrails on litellm with minimal code changes (no custom handlers) Test guardrail integrations more easily * feat(a2a/): add guardrails for agent interactions allows the same guardrails for llm's to be applied to agents as well * fix(a2a/): support passing guardrails to a2a from the UI * style(code-editor): allow editing custom code guardrails on ui + add examples of pre/post calls for custom code guardrails * feat(mcp/): support custom code guardrails for mcp calls allows custom code guardrails to work on mcp input * feat(chatui.tsx): support guardrails on mcp tool calls on playground * fix(mypy): resolve missing return statements and type casting issues (#20618) * fix(mypy): resolve missing return statements and type casting issues * fix(pangea): use elif to prevent UnboundLocalError and handle None messages Address Greptile review feedback: - Make branches mutually exclusive using elif to prevent input_messages from being overwritten - Handle case where data.get('messages') returns None to avoid passing invalid payload to Pangea API --------- Co-authored-by: Shin <shin@openclaw.ai> * [Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet (#20607) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * fixing user_id * [Feat] IP-Based Access Control for MCP Servers (#20620) * update MCPAuthenticatedUser * add available_on_public_internet for MCPs * update claude.md * init IPAddressUtils * init available_on_public_internet * add on REST endpoints * filter with IP * TestIsInternalIp * _extract_mcp_headers_from_request * init get_mcp_client_ip * _get_general_settings * allowed_server_ids * address PR comments * get_mcp_server_by_name fix * fix server * fix review comments * get_public_mcp_servers * address _get_allowed_mcp_servers * test fix * fix linting * inint ui types * add ui for managing MCP private/public * add ui * fixes * add to schema * add types * fix endpoint * add endpoint * update manager * test mcp * dont use external party for ip address * Add OpenAI/Azure release test suite with HTTP client lifecycle regression detection (#20622) * docs (#20626) * docs * fix(mypy): resolve type checking errors in 5 files (#20627) - a2a_protocol/exception_mapping_utils.py: Fix type ignore comment for None assignment - caching/redis_cache.py: Add type ignore for async ping return type - caching/redis_cluster_cache.py: Add type ignore for async ping return type - llms/deprecated_providers/palm.py: Add type ignore for palm.generate_text - proxy/auth/handle_jwt.py: Add type ignore for jwt.decode options argument All changes add appropriate type: ignore comments to handle library typing inconsistencies. * fix(test): update deprecated gemini embedding model (#20621) Replace text-embedding-004 with gemini-embedding-001. The old model was deprecated and returns 404: 'models/text-embedding-004 is not found for API version v1beta' Co-authored-by: Shin <shin@openclaw.ai> * ui new buil * fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing When users pass a shared_session with trace_configs to acompletion(), the get_async_httpx_client() function was ignoring it and returning a cached client without the user's tracing configuration. This fix bypasses the cache when shared_session is provided, ensuring the user's ClientSession (with its trace_configs, connector settings, etc.) is actually used for the request. Fixes #20174 --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Shin <shin@openclaw.ai> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com> Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
Contributor
|
Warning about this being a breaking change: #20620 (comment) |
Member
Author
|
@andresC98 fixed in #22331 — default is now back to public. Also patching onto stable as 1.81.12.stable.2, migration script in the PR for any existing servers that got set to private. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Feat] MCP Gateway - Allow setting MCP Servers as Private/Public available on Internet
Adds
available_on_public_internetflag to MCP server config — admins mark which MCP servers should be visible to external callersavailable_on_public_internet: trueHow it works
IPAddressUtilsclass inlitellm/proxy/auth/ip_address_utils.pyhandles IP classification (RFC 1918, IPv6 ULA, XFF chain parsing, fails closed on bad input)MCPServerManager.filter_server_ids_by_ip()is a post-filter callers chain afterget_allowed_mcp_servers()— zero changes to the existing auth logicget_mcp_server_by_name()andget_filtered_registry()enforce IP filtering for dynamic routes and the registry endpointConfig example
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
✅ Test
Changes