fix(responses): fail loudly on MCP gateway failures (initial call, mid-stream, zero resolved tools)

[thibault-linktree]

Relevant issues

Fixes #32561, fixes #32562, fixes #32563

Combines #32565, #32566 and #32567 into a single PR per review logistics (the three fixes share the _stream_error stash / _make_stream_error_event helper and their tests interact). All Greptile feedback from those PRs is already incorporated; see the closed PRs for the per-fix review threads.

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem (three swallow-points of the same defect: MCP gateway failures are invisible to clients)
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Real proxy (litellm --config, real OpenAI calls, deepwiki MCP server registered). Commit hashes are from the original per-fix branches; the combined branch is the same changes rebased onto litellm_oss_staging.

1. Initial LLM call fails (bogus previous_response_id), streaming

Before (main @ 999637883c) — HTTP 200 + discovery events with no response.created (crashes openai-node: expected 'response.created' event, got response.mcp_list_tools.in_progress):

HTTP/1.1 200 OK
data: {"type":"response.mcp_list_tools.in_progress",...}
data: {"type":"response.mcp_list_tools.completed",...}
data: {"type":"response.output_item.done","item":{"type":"mcp_list_tools",...}}
data: [DONE]

After (@ b6bbee9c92) — real 400 with the provider error body, before any SSE bytes:

HTTP/1.1 400 Bad Request
{"error":{"message":"litellm.BadRequestError: OpenAIException - {...\"message\": \"Previous response with id 'resp_bogus_e2e_capture' not found.\"...}","code":"400"}}

2. Tool execution / follow-up failure mid-stream (requires fault injection; captures drive the real iterator with only the failing dependency mocked)

Before — tool events, then the stream ends with no terminal event; a batch tool-execution failure additionally attempted the doomed follow-up (follow-up model call attempted: True), rejected by the provider with "No tool output found for function call ...".

After (@ bd9e13914e) — terminal error event; doomed follow-up skipped:

... mcp_call events ...
error  code=400 message=litellm.BadRequestError: {"error": {"message": "No tool output found for function call ..."}}
<< stream ended >>   (follow-up model call attempted: False in the tool-failure case)

3. MCP tools requested, zero resolved (key lacks server access / unknown server)

Before (main @ 999637883c) — HTTP 200, model called with tools=None, fabricated answer ("It seems I can't access external tools or links, including Linktree. You can check the number of links on your Linktree by logging into your account ..."); only trace is a "success" list_mcp_tools spend log with [].

After (@ 60dc62fb84) — 400 with an actionable message:

{"error":{"message":"litellm.BadRequestError: MCP gateway resolved 0 tools for the requested MCP tool(s) (server_url(s): ['litellm_proxy/mcp/linktree_omnibot']). Likely causes: the API key/team does not have access to the MCP server (server has allow_all_keys=false and no key/team object-permission grant), the server name does not exist, or allowed_tools matches no tool on the server. ...","code":"400"}}

Happy path, fully e2e (real deepwiki read_wiki_structure call): unchanged on every fix — full two-phase flow, real answer ("The first topic name is "Overview.""), zero error events.

Type

🐛 Bug Fix

Changes

Three swallow-points in the MCP gateway made failures invisible to clients (HTTP 200 + broken/silent streams, or silent no-tools model calls):

1. Initial LLM call (aresponses_api_with_mcp + MCPEnhancedStreamingIterator): the initial call is now made eagerly, before any SSE bytes are written, and a stashed creation failure is re-raised so the proxy returns a real 4xx/5xx. If a failure still surfaces during iteration, the stream emits a single terminal error event instead of mcp_list_tools discovery events with no response.created.
2. Tool execution / follow-up call: failures are stashed and surfaced as a terminal error event instead of silently ending the stream; after a batch tool-execution failure the doomed follow-up call (function_calls with no outputs → provider 400) is skipped entirely.
3. Zero resolved MCP tools: requests that asked for litellm_proxy MCP tools but resolved none (and carry no other tools to fall back on) now raise a 400 naming the requested server URLs and likely causes. Mixed requests with function tools fall back to those tools (previous behaviour). Opt-out: litellm.reject_empty_mcp_resolved_tools = False (default True per maintainer guidance).

Tests: 9 new tests across tests/test_litellm/responses/mcp/test_mcp_streaming_iterator.py (new file) and tests/test_litellm/responses/mcp/test_mcp_empty_resolved_tools.py (new file); the auth-header pass-through test in tests/mcp_tests/test_aresponses_api_with_mcp.py now resolves a dummy tool (its purpose is header propagation, not zero-tool behaviour). Full local run: 50 passed.

[greptile-apps]

Greptile Summary

This PR fixes three silent failure modes in the MCP gateway where client-visible errors were swallowed, producing HTTP 200 responses with broken or empty streams, or a no-tools model call that hallucinated. The fixes share a common stash-and-surface pattern: failures are stashed in _stream_error/_initial_creation_error and re-raised as real HTTP errors (eager path) or emitted as terminal OpenAI-format error stream events (mid-stream path).

• Initial LLM call: aresponses_api_with_mcp now calls _create_initial_response_iterator() eagerly and re-raises any stashed failure before writing any SSE bytes, replacing the old HTTP-200-with-broken-stream behaviour.
• Tool execution / follow-up failure: a batch tool-execution failure now clears queued per-tool events (preventing orphaned mcp_call.in_progress events), stashes the error, and skips the doomed follow-up call; both tool-execution and follow-up failures surface as a monotonically-numbered terminal error event.
• Zero resolved MCP tools: a new litellm.reject_empty_mcp_resolved_tools flag (default True) gates a pre-call guard that raises BadRequestError with an actionable message when the gateway resolves no tools and the request has no other tools to fall back on.

Confidence Score: 5/5

Safe to merge — changes are well-scoped to the MCP streaming path, add an opt-out flag for the behavioural change, and are backed by 9 targeted mock tests covering every new code branch.

All three failure modes are addressed through a coherent stash-and-surface pattern with no unsafe assumptions. The zero-tool guard is gated behind a module-level flag so existing users can opt out. The eager initial call correctly prevents double-invocation via the existing guard in _anext_impl. Sequence-number monotonicity is maintained by the outer __anext__ wrapper. No pre-existing regression detectors were softened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/responses/mcp/mcp_streaming_iterator.py	Core fix: adds error stashing, monotonic sequence tracking, and terminal error event emission for initial-call, tool-execution, and follow-up failures; drops orphaned per-tool events on batch failure and skips the doomed follow-up call.
litellm/responses/main.py	Adds the zero-resolved-tools guard (raises BadRequestError before any model call) and the eager initial-call creation that re-raises pre-stream failures as real HTTP errors instead of broken HTTP-200 SSE streams.
litellm/init.py	Adds reject_empty_mcp_resolved_tools: bool = True module-level flag with a clear comment; follows the existing opt-out flag pattern used elsewhere in the module.
tests/test_litellm/responses/mcp/test_mcp_streaming_iterator.py	New test file with 6 well-focused tests covering the three failure scenarios (initial call, tool execution, follow-up) plus the happy path, with assertions on sequence-number monotonicity and absence of orphaned events.
tests/test_litellm/responses/mcp/test_mcp_empty_resolved_tools.py	New test file with 5 tests covering: zero-tool raises (stream and non-stream), fallback to function tools, flag-off restores old behaviour, happy path with resolved tools, and plain function-tool requests being unaffected.
tests/mcp_tests/test_aresponses_api_with_mcp.py	Legitimate test fix: mock_process now returns a dummy tool so the auth-header propagation test avoids the new zero-tool guard; the test's purpose (verifying mcp_server_auth_headers forwarding) is unchanged and coverage is not weakened.
litellm/proxy/dev_config.yaml	Adds the deepwiki MCP server entry used for manual e2e validation of the happy path; dev-only config, no production impact.

_{Reviews (2): Last reviewed commit: "fix(responses): monotonic sequence numbe..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 94.73684% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/responses/main.py	87.50%	1 Missing ⚠️
litellm/responses/mcp/mcp_streaming_iterator.py	96.55%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[thibault-linktree]

Both review points addressed in the latest commit:

• Monotonic sequence numbers: __anext__ now tracks the highest sequence_number seen across all emitted chunks (provider passthrough and synthesized events alike), and the terminal error event is numbered after it. Covered by an assertion in test_follow_up_failure_emits_error_event.
• Orphaned in-progress events: a batch tool-execution failure now drops the queued per-tool events instead of emitting mcp_call.in_progress items that never terminate; the terminal error event carries the failure. Covered by an assertion in test_tool_execution_failure_emits_error_event_and_skips_follow_up.

@greptileai