feat(proxy): add per-MCP-server RPM rate limiting for keys and teams

[Sameerlite]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Booted a proxy with one stdio MCP server (uvx mcp-server-fetch, aliased fetch_mcp) and generated two keys that both have full access to that server. They differ only in which server name their mcp_rpm_limit targets: the limited key caps fetch_mcp at 2/min, while the control key caps a different name (other_mcp) so its fetch_mcp calls stay uncapped. Firing four fetch_mcp tool calls with each key shows the limited key trips at the third call while the control key is never limited, proving the cap is keyed per MCP server name rather than globally per key.

=== limited key (caps fetch_mcp at 2/min) ===
   call 1 -> HTTP 200
   call 2 -> HTTP 200
   call 3 -> HTTP 429
   call 4 -> HTTP 429

=== control key (caps other_mcp, so fetch_mcp is uncapped) ===
   call 1 -> HTTP 200
   call 2 -> HTTP 200
   call 3 -> HTTP 200
   call 4 -> HTTP 200

=== result ===
PASS: fetch_mcp tripped at call 3 (429) under the limited key; the control key (which caps other_mcp) was never rate limited on fetch_mcp.

The exact key generation and tool-call requests behind the run above (master key and host elided):

# limited key: caps fetch_mcp at 2/min
curl -sf -X POST "$BASE/key/generate" \
  -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"mcp_rpm_limit": {"fetch_mcp": 2}, "object_permission": {"mcp_servers": ["fetch_mcp"]}}'

# control key: caps a different server name, so fetch_mcp is uncapped
curl -sf -X POST "$BASE/key/generate" \
  -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"mcp_rpm_limit": {"other_mcp": 2}, "object_permission": {"mcp_servers": ["fetch_mcp"]}}'

# fire the same fetch_mcp call four times per key
curl -s -o /dev/null -w "%{http_code}\n" -X POST "$BASE/mcp-rest/tools/call" \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"server_id": "fetch_mcp", "name": "fetch", "arguments": {"url": "'"$BASE"'/health/readiness", "max_length": 100}}'

The full reproducer is committed at scripts/test_mcp_rpm_limit.sh.

Type

🆕 New Feature

Changes

Adds mcp_rpm_limit, a dict keyed by MCP server name (the alias if one is set, otherwise the configured server name) that caps requests per minute per server for a key or team. It can be set on /key/generate, /key/update, /team/new, and /team/update, and is stored in key/team metadata.

The v3 rate limiter builds a per-server descriptor (mcp_per_key / mcp_per_team, keyed {api_key}:{server_name}) only when a limit is configured for the specific server being called, so other servers stay uncapped and no TPM reservation is engaged. The server identity is surfaced into the request data as mcp_rate_limit_server_name (resolved from server.alias or server.server_name or server.name) so the limiter can resolve which descriptor applies.
The descriptors are only built when the call is an actual MCP tool call (call_type == call_mcp_tool), so a mcp_server_name field injected into a normal LLM request body cannot consume another server's MCP quota.

---

Note

Medium Risk
Touches proxy rate limiting and auth metadata for MCP tool calls; mis-keyed server names or hook gating could block or under-limit legitimate traffic, but scope is narrow and guarded by call_type and server_id checks.

Overview
Adds mcp_rpm_limit, a per-MCP-server requests-per-minute cap on API keys and teams (dict keyed by server alias if set, else configured name), wired through management types, metadata hoisting (LiteLLM_ManagementEndpoint_MetadataFields), and key/team/user endpoint docs.

The v3 parallel request limiter gains mcp_per_key / mcp_per_team descriptors (RPM only, no TPM). Limits apply only for call_mcp_tool when the hook data is the trusted synthetic payload (no server_id in the body), so injected mcp_server_name on LLM routes or raw REST bodies cannot consume another server’s quota. Server identity is set in pre_call_tool_check as mcp_rate_limit_server_name and surfaced into hook data as mcp_server_name for enforcement.

get_key_mcp_rpm_limit / get_team_mcp_rpm_limit resolve limits from key metadata (over team). Tests cover alias resolution, descriptor gating, 429 behavior, and metadata round-trip.

^{Reviewed by Cursor Bugbot for commit a34126a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

Codecov Report

❌ Patch coverage is 87.23404% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/hooks/parallel_request_limiter_v3.py	85.18%	4 Missing ⚠️
...y/management_endpoints/key_management_endpoints.py	33.33%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds mcp_rpm_limit, a per-MCP-server requests-per-minute cap settable on keys and teams, stored in metadata and enforced by the v3 parallel request limiter. The feature is cleanly gated by call_type == call_mcp_tool and a "server_id" not in data guard to prevent raw REST payloads or injected LLM-request fields from consuming MCP quota.

• Adds mcp_rpm_limit: Optional[Dict[str, int]] to request/team types, LiteLLM_ManagementEndpoint_MetadataFields, and generate_key_helper_fn; hoisting into metadata works correctly for both key and team management paths.
• Surfaces the resolved server identity (alias or server_name or name) as mcp_rate_limit_server_name in pre-hook kwargs, then as mcp_server_name in the synthetic data dict the limiter reads.
• New mcp_per_key / mcp_per_team descriptors set tokens_per_unit: None so the TPM reservation path is never engaged for tool calls.

Confidence Score: 5/5

Safe to merge — the new rate-limiting path is narrowly scoped to MCP tool calls and guarded by two independent checks that prevent it from affecting any existing request flow.

The call_type guard and 'server_id' not in data check form independent layers that correctly isolate the new MCP rate-limiting logic. Server name resolution always produces a non-None value. Metadata hoisting follows the established LiteLLM_ManagementEndpoint_MetadataFields pattern. Tests cover all meaningful code paths including the security-relevant raw-REST-body gating and cross-key/team precedence. No existing behavior is altered.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/hooks/parallel_request_limiter_v3.py	Adds _add_mcp_per_key_rate_limit_descriptor / _add_mcp_per_team_rate_limit_descriptor methods and wires call_type through _create_rate_limit_descriptors; MCP descriptors are guarded by call_type == call_mcp_tool and 'server_id' not in data.
litellm/proxy/auth/auth_utils.py	Adds get_key_mcp_rpm_limit (key metadata first, team metadata fallback) and get_team_mcp_rpm_limit; follows the same precedence pattern as the existing get_key_model_rpm_limit helpers.
litellm/proxy/_experimental/mcp_server/mcp_server_manager.py	Injects mcp_rate_limit_server_name (resolved as alias or server_name or name) into the pre-hook kwargs.
litellm/proxy/_types.py	Adds mcp_rpm_limit to GenerateRequestBase, NewTeamRequest, UpdateTeamRequest, and LiteLLM_ManagementEndpoint_MetadataFields.
litellm/proxy/management_endpoints/key_management_endpoints.py	Adds mcp_rpm_limit param to generate_key_helper_fn with doc string updates.
litellm/proxy/utils.py	Surfaces mcp_rate_limit_server_name into synthetic data as mcp_server_name.
tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py	Comprehensive tests for descriptor creation, skip logic, 429 enforcement, and precedence.
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_hook_extra_headers.py	Tests mcp_server_name surfacing and alias resolution.
tests/test_litellm/proxy/management_endpoints/test_common_utils.py	Regression test for mcp_rpm_limit metadata hoisting.

_{Reviews (5): Last reviewed commit: "fix(proxy): accept mcp_rpm_limit in gene..." | Re-trigger Greptile}

[veria-ai]

PR overview

This PR adds proxy support for per-MCP-server requests-per-minute limits for keys and teams, including the configuration/type fields needed to enforce those limits. It extends key and team rate-limit handling for MCP server usage.

There is one open security issue remaining. Users with key-update access may be able to change the MCP RPM limit metadata for their own key, allowing them to weaken or remove administrator-configured per-server caps. One issue has already been addressed, but this remaining update-path control should be tightened before merging.

Open issues (1)

• Medium: MCP rate limit bypass through key metadata updates — litellm/proxy/_types.py:4291

Fixed/addressed: 1 · PR risk: 5/10

[Sameerlite]

Addressed the review feedback: the MCP rate-limit descriptors are now gated on call_type == call_mcp_tool so a normal LLM request cannot inject mcp_server_name to drain another server's quota, with a regression test added. Also documented the new mcp_rpm_limit parameter on the update_key, new_user, and user_update endpoints so the API-docs validator passes.

@greptileai

[Sameerlite]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: Injected mcp_server_name in MCP request body drains another server's quota
  • Raw REST MCP request bodies are now excluded from per-server MCP descriptors until the resolved synthetic hook payload supplies the server name.
• ✅ Fixed: Truthy check prevents explicit empty-dict override of team limits
  • MCP RPM limit lookup now treats empty dictionaries as explicit configured values by checking for None instead of truthiness.

_{You can send follow-ups to the cloud agent here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ Sameerlite
✅ mateo-berri
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Sameerlite]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 73c61e5. Configure here.}

[Sameerlite]

@greptileai

[Sameerlite]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a34126a. Configure here.}

[veria-ai]

Medium: MCP rate limit bypass through key metadata updates

mcp_rpm_limit is now treated as generic key metadata, and /key/update preserves that path for key owners and delegated team members on non-budget fields. A caller with key-update access can set their key's mcp_rpm_limit to {} or a higher value and remove the admin-configured per-server key cap that get_key_mcp_rpm_limit() enforces; make this field admin-only/immutable on key updates or preserve the existing value unless the caller has key administration rights.

[mateo-berri]

This finding flags intended, pre-existing behavior rather than a regression introduced here. mcp_rpm_limit is plumbed exactly like the existing per-key rate-limit metadata fields model_rpm_limit and model_tpm_limit: same LiteLLM_ManagementEndpoint_MetadataFields hoisting, same get_key_* key-over-team resolution. It is not handled any differently from those fields on /key/update.

The /key/update authorization model is deliberate (see _validate_update_key_data, the GHSA-q775-qw9r-2r4g hardening). A non-admin key owner or an authorized team member may update non-budget fields on their own key; only max_budget and spend are gated behind the admin check. Under that model, rpm_limit, tpm_limit, model_rpm_limit, and model_tpm_limit are already self-modifiable by the key owner today. mcp_rpm_limit inherits the same property, so it does not open a new bypass.

Making just this one field admin-only or immutable would be inconsistent with every other per-key rate limit, was not part of this feature's scope, and would not actually change the threat model since the same caller could still raise their own rpm_limit / model_rpm_limit. Budget and spend remain the hard, admin-only controls. If per-key rate limits should become admin-immutable, that is a separate, system-wide design change covering all rate-limit fields, not something specific to mcp_rpm_limit.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

LGTM; thanks!