fix(responses): emit terminal error event on MCP tool-execution / follow-up failures

[thibault-linktree]

Relevant issues

Fixes #32562

Stacked on #32565 (shares the _stream_error stash and _make_stream_error_event helper) — review the last commit only; will rebase once #32565 lands.

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

These failures require fault injection mid-stream (batch tool-execution failure / follow-up provider failure), so the before/after captures drive the real MCPEnhancedStreamingIterator with only the failing dependency mocked; the happy-path proof is fully e2e with real LLM + MCP calls.

Before (main @ 999637883c), tools succeed then follow-up call 400s — stream ends with no terminal event:

response.output_item.added
response.mcp_list_tools.in_progress
response.mcp_list_tools.completed
response.completed            (initial response: output=[function_call])
response.mcp_call.in_progress
response.mcp_call_arguments.delta
response.mcp_call_arguments.done
response.mcp_call.completed
response.output_item.done
<< stream ended >>

Batch tool-execution failure additionally attempted the doomed follow-up call (follow-up model call attempted: True), which the provider rejects with "No tool output found for function call ...".

After (this branch @ bd9e13914e) — terminal error event, and the doomed follow-up is skipped:

... same events as above ...
error  code=400 message=litellm.BadRequestError: {"error": {"message": "No tool output found for function call ..."}}
<< stream ended >>
follow-up model call attempted: False   (tool-execution failure case)

Happy-path regression proof, fully e2e (this branch @ bd9e13914e, real proxy + real OpenAI + real deepwiki MCP call): full two-phase flow (response.created ×2, executed mcp_call, final output_text), 0 error events.

Type

🐛 Bug Fix

Changes

• Stash tool-execution and follow-up failures on the iterator instead of dropping them after the stderr log.
• After a batch tool-execution failure, skip the follow-up call entirely (it would carry function_call items with no outputs and be rejected by the provider).
• When the follow-up cannot start, emit a single terminal OpenAI-style error stream event carrying the mapped failure instead of ending the stream silently.

Tests: tests/test_litellm/responses/mcp/test_mcp_streaming_iterator.py — tool-execution failure emits error event and skips follow-up, follow-up failure emits error event, multi-round happy path stays error-free. Full tests/test_litellm/responses/ suite passes (343 tests).

[greptile-apps]

Greptile Summary

This PR fixes silent stream terminations in the MCP streaming path by ensuring that failures during the initial LLM call, tool execution, or follow-up LLM call all surface as a terminal OpenAI-style error event rather than an abrupt end-of-stream; it also eagerly surfaces initial-call failures as HTTP-level errors before any SSE bytes are written.

• mcp_streaming_iterator.py: introduces _stream_error/_initial_creation_error/_error_event_emitted fields and a _make_stream_error_event helper; wires all three failure paths to emit the error event and skips the doomed follow-up call when tool execution has already failed.
• responses/main.py: calls _create_initial_response_iterator() eagerly before returning the streaming response so an invalid previous_response_id (or other pre-stream failure) becomes a normal HTTP 4xx/5xx instead of an HTTP 200 with a broken stream.
• New tests in tests/test_litellm/responses/mcp/test_mcp_streaming_iterator.py cover all three failure scenarios and a happy-path regression guard using monkeypatched mocks (no real network calls).

Confidence Score: 5/5

Safe to merge — the change is tightly scoped to MCP streaming failure paths and adds no new happy-path logic.

All three failure scenarios are correctly handled and tested. The eager-creation guard prevents double LLM calls, the _error_event_emitted flag prevents duplicate terminal events, and no existing test semantics were weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/responses/mcp/mcp_streaming_iterator.py	Core fix: adds _stream_error/_initial_creation_error/_error_event_emitted fields, _make_stream_error_event helper, and wires all three failure scenarios to emit a terminal OpenAI-style error event; also skips the doomed follow-up call when tool execution has already failed.
litellm/responses/main.py	Eagerly calls _create_initial_response_iterator() before returning the streaming response so a pre-stream failure becomes a regular HTTP error instead of an HTTP 200 with a broken stream; checks _initial_creation_error and re-raises it if set.
tests/test_litellm/responses/mcp/test_mcp_streaming_iterator.py	New test file covering all three failure paths plus happy-path regression guard; all tests use monkeypatch mocks with no real network calls.
litellm/proxy/dev_config.yaml	Adds deepwiki MCP server entry for local development and happy-path e2e testing.

_{Reviews (2): Last reviewed commit: "fix(responses): emit terminal error even..." | Re-trigger Greptile}

[greptile-apps]

The sequence_number for the terminal error event is hardcoded to 0, but other events emitted earlier in the same stream carry positive, incrementing sequence numbers. Clients that enforce monotonically-increasing sequence ordering (or that use the sequence number to deduplicate/reorder events) will see an out-of-order value. Adding a simple counter (e.g. self._next_sequence_number) that is bumped each time an event is yielded and reused here would keep the stream contract intact.

Suggested change

	return ErrorEvent(
	type=ResponsesAPIStreamEvents.ERROR,
	sequence_number=0,
	error=ErrorEventError(
	return ErrorEvent(
	type=ResponsesAPIStreamEvents.ERROR,
	sequence_number=self._next_sequence_number,
	error=ErrorEventError(

[codecov]

Codecov Report

❌ Patch coverage is 70.96774% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/responses/main.py	0.00%	8 Missing ⚠️
litellm/responses/mcp/mcp_streaming_iterator.py	95.65%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will degrade performance by 11.8%

❌ 1 regressed benchmark
✅ 29 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_completion_simple_message	4.2 ms	4.8 ms	-11.8%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing thibault-linktree:litellm_fix_mcp_gateway_midstream_failure_error_event (bd9e139) with main (9996378)}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 5 committers have signed the CLA.

✅ fernando-izar
✅ seph-barker
✅ Sameerlite
✅ thibault-linktree
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[thibault-linktree]

Rebased onto litellm_oss_staging (still stacked on #32565 — review the last commit only). Review feedback addressed:

• CodeQL cyclic import: ErrorEvent/ErrorEventError now imported at module level alongside the other litellm.types.llms.openai imports (no new function-level import)
• On sequence_number: kept the constant 1 rather than a per-iterator counter — the terminal error event is fatal by definition and every SDK we tested treats error events positionally (openai-node raises immediately without consulting sequence numbers); threading a monotonic counter through the passthrough chunks would touch every yield site for no observable behaviour change. Happy to add the counter if maintainers prefer.
• The CodSpeed regression flag (test_completion_simple_message, 4.2ms → 4.8ms) looks like simulation noise: this PR touches only mcp_streaming_iterator.py failure paths, which are not on the completion code path.

@greptileai

[thibault-linktree]

Consolidated into #32579 (all three MCP gateway fail-loudly fixes in one PR, with the review feedback from this PR incorporated).