fix(mcp): apply outbound concurrency limit to OBO tool calls

[tin-berri]

Relevant issues

Stacked on #31762 (litellm_mcp_v2_obo_endpoint_discovery), which is where the OBO token-exchange tool-call path and the per-server _limit_outbound_concurrency limiter were introduced. #31983 stacks on the same base, so this fix flows up to it once it syncs with #31762

Linear ticket

N/A (found during review of the OBO stack)

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Root cause: in _call_regular_mcp_tool the oauth2_token_exchange (OBO) branch built its coroutine by calling _obo_call_tool_with_retry directly, outside the _limit_outbound_concurrency(mcp_server) context manager that the regular branch wraps its call in. OBO tool calls, and the on-401 re-mint retry (a second upstream call_tool), therefore never acquired the per-server max_concurrent_requests semaphore, so a caller with access to an OBO MCP server could run unlimited concurrent tool calls against it regardless of the admin-configured cap

Reproduced against a live proxy on localhost:4011 backed by real Postgres, a real RFC 8693 token-exchange round-trip to a local IdP, and a real upstream MCP server (streamable-http). The upstream records how many call_tool invocations overlap. The MCP server is configured with auth_type: oauth2_token_exchange and max_concurrent_requests: 2; the proxy key travels in x-litellm-api-key so Authorization carries the OBO subject token. The same subject token is reused across the batch, so the exchanger single-flights to one exchange and all six calls pile onto the upstream semaphore

Command (identical before and after):

for i in $(seq 1 6); do
  curl -s http://localhost:4011/mcp-rest/tools/call \
    -H "x-litellm-api-key: sk-1234" \
    -H "Authorization: Bearer subject-jwt-caller-001" \
    -H "Content-Type: application/json" \
    -d "{\"server_id\":\"$SID\",\"name\":\"slow_echo\",\"arguments\":{\"text\":\"c$i\"}}" \
    -w "call $i -> http %{http_code} in %{time_total}s\n" &
done
wait

Before the fix, all six calls run at once (peak upstream concurrency 6, every call returns in ~1.6s) despite the limit of 2:

call 1 -> http 200 in 1.589s   call 2 -> http 200 in 1.601s
call 3 -> http 200 in 1.609s   call 4 -> http 200 in 1.613s
call 5 -> http 200 in 1.608s   call 6 -> http 200 in 1.610s

# upstream MCP inflight timeline (peak reaches 6)
ENTER inflight=1   ENTER inflight=2   ENTER inflight=3
ENTER inflight=4   ENTER inflight=5   ENTER inflight=6
EXIT  inflight=5 ... EXIT inflight=0

After the fix, the six calls serialize into three waves of two (peak upstream concurrency 2, completions bucket at ~1.6s / ~3.2s / ~4.7s):

call 1 -> http 200 in 1.612s   call 2 -> http 200 in 1.617s
call 3 -> http 200 in 3.154s   call 4 -> http 200 in 3.155s
call 5 -> http 200 in 4.694s   call 6 -> http 200 in 4.701s

# upstream MCP inflight timeline (peak stays at 2)
ENTER inflight=1   ENTER inflight=2   EXIT inflight=1   EXIT inflight=0
ENTER inflight=1   ENTER inflight=2   EXIT inflight=1   EXIT inflight=0
ENTER inflight=1   ENTER inflight=2   EXIT inflight=1   EXIT inflight=0

Type

🐛 Bug Fix

Changes

_call_regular_mcp_tool now wraps the OBO coroutine in _limit_outbound_concurrency(mcp_server), mirroring the regular branch's _call_tool_via_client. One permit is held across the whole logical call: the initial client.call_tool, the on-401 credential invalidation and client re-mint, and the retry. The initial client build stays outside the semaphore, matching the regular path, so OBO and non-OBO tool calls now enforce max_concurrent_requests identically

Added TestOBOConcurrencyLimit.test_obo_dispatch_respects_max_concurrent_requests, which drives concurrent OBO dispatches through _call_regular_mcp_tool against a blocking client and asserts the observed peak equals the configured cap. It fails on the unfixed code (peak equals the number of callers) and passes with the fix

---

Note

Low Risk
Targeted concurrency fix aligned with existing non-OBO behavior; regression test covers the OBO dispatch path with no auth or data-model changes.

Overview
OBO (oauth2_token_exchange) MCP tool calls now respect each server’s max_concurrent_requests cap, matching the non-OBO dispatch path.

In _call_regular_mcp_tool, the token-exchange branch previously scheduled _obo_call_tool_with_retry without entering _limit_outbound_concurrency, so concurrent OBO calls could bypass the per-server semaphore. The fix wraps that work in _obo_call_tool_limited(), holding one permit for the full logical call (initial call_tool, optional 401 re-mint, and retry).

Adds TestOBOConcurrencyLimit.test_obo_dispatch_respects_max_concurrent_requests to assert peak in-flight calls equals the configured limit.

^{Reviewed by Cursor Bugbot for commit 0534fde. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai

[greptile-apps]

Greptile Summary

The OBO (OAuth2 token-exchange) tool-call path in _call_regular_mcp_tool was building its coroutine outside the _limit_outbound_concurrency context manager that the non-OBO path correctly wraps its call in, so any call using auth_type: oauth2_token_exchange could run unlimited concurrent upstream tool calls regardless of the admin-configured max_concurrent_requests cap.

• Production fix (mcp_server_manager.py): introduces _obo_call_tool_limited, a local async function that wraps _obo_call_tool_with_retry inside async with self._limit_outbound_concurrency(mcp_server), mirroring the non-OBO path. One permit is held across the full logical call — initial attempt, optional credential invalidation and client re-mint, and the retry — matching the regular path's semantics.
• Test (test_mcp_server_manager.py): adds TestOBOConcurrencyLimit.test_obo_dispatch_respects_max_concurrent_requests, which drives 5 concurrent OBO dispatches through a mock client that blocks on an asyncio.Event, polls until the inflight count stabilises, and then asserts the observed peak equals the configured cap of 2.

Confidence Score: 5/5

Safe to merge — the change is a narrow, well-tested fix to a single missing context-manager wrapping, with no behavioural impact on the non-OBO path.

The fix is a small, surgical closure of a missing semaphore acquisition. The logic is structurally identical to the non-OBO branch it mirrors. The new test uses only mocks, reproduces the failure mode, and directly asserts the corrected peak concurrency. No pre-existing tests are modified, and no production-code paths other than the OBO branch are touched.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/mcp_server_manager.py	Wraps the OBO call coroutine in _limit_outbound_concurrency, fixing the missing semaphore acquisition that let OBO tool calls bypass the per-server max_concurrent_requests cap
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py	Adds TestOBOConcurrencyLimit with a mock-only asyncio concurrency test that blocks on an event until the peak inflight count has stabilised, then asserts the peak equals the configured cap — correctly fails on the unfixed code

_{Reviews (3): Last reviewed commit: "fix(mcp): apply outbound concurrency lim..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes a concurrency-limit bypass in the MCP OBO (OAuth2 token-exchange) tool-call path. The OBO branch in _call_regular_mcp_tool was building its coroutine outside _limit_outbound_concurrency, so per-server max_concurrent_requests semaphore was never acquired for OBO calls or their 401-triggered retries.

• mcp_server_manager.py: Introduces _obo_call_tool_limited(), a local async wrapper that acquires _limit_outbound_concurrency(mcp_server) before delegating to _obo_call_tool_with_retry, exactly mirroring the _call_tool_via_client wrapper used by the non-OBO branch. The client build and semaphore entry point are unchanged relative to the non-OBO path.
• test_mcp_server_manager.py: Adds TestOBOConcurrencyLimit.test_obo_dispatch_respects_max_concurrent_requests, which runs 5 concurrent OBO dispatches through a blocking mock client configured with max_concurrent_requests=2 and asserts the observed peak inflight count equals the cap; fully mocked, no network calls.

Confidence Score: 5/5

Safe to merge — the change is minimal and targeted, the fix exactly mirrors the already-trusted non-OBO code path, and the new test demonstrates the corrected behavior.

The OBO wrapper is a straightforward one-liner delta: a local async def with async with self._limit_outbound_concurrency(mcp_server) around the existing _obo_call_tool_with_retry call. The non-OBO path uses the identical pattern and has been in place since the base branch, so the fix has a proven template. The test directly exercises the regression path — blocking mock client, semaphore stabilization check, and a final peak assertion — and would fail on unfixed code. No existing tests were modified and no unrelated code was touched.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/mcp_server_manager.py	Wraps the OBO tool-call coroutine in _limit_outbound_concurrency, mirroring the existing non-OBO _call_tool_via_client pattern; the semaphore is held across the initial call and any 401-triggered retry, fixing the bypass.
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py	Adds TestOBOConcurrencyLimit.test_obo_dispatch_respects_max_concurrent_requests: drives 5 concurrent OBO dispatches through a blocking mock client with max_concurrent_requests=2, observes peak inflight, and asserts it stays at 2; all mocked, no real network calls.

_{Reviews (2): Last reviewed commit: "fix(mcp): apply outbound concurrency lim..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tin-berri]

@greptileai

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0534fde. Configure here.}

[codspeed-hq]

Merging this PR will improve performance by 17.11%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 29 untouched benchmarks

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	test_completion_simple_message	4.8 ms	4.1 ms	+17.11%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing litellm_mcp_v2_obo_concurrency_limit (0534fde) with litellm_internal_staging (ff6dc33)}

[mateo-berri]

LGTM; thanks!