fix(otel): stamp an MCP tool failure on the request that carried it

[yassin-berriai]

TLDR

Problem this solves:

• a failed MCP tool call stamps its error.* on the span of whichever request opened the session, which has already ended, so the SDK drops the write and the POST that actually failed is left unmarked
• the identity attributes seeded onto the server span go to the same dead span

How it solves it:

• carry the live transport span of the current message across the session-task boundary and stamp that
• re-anchor the request root to it for the message, so guardrail spans and identity seeding follow, and only ever anchor or stamp a span still open for writes

Relevant issues

Follows #34537, which fixed the trace placement of the same span and has merged. This branch is rebased onto litellm_internal_staging and carries only the change below.

Linear ticket

Resolves LIT-4784

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Same rig as #34537: live proxy with otel_v2 exporting OTLP/HTTP to a local collector that prints one line per span, one upstream MCP server, and a session driven with curl the way MCP Inspector drives it, ending in a tools/call for a tool that fails upstream. The wire is unchanged throughout, HTTP 200 with isError: true.

Before, on litellm_internal_staging with #34537 in it. Every failed tool call logs:

$ grep -c 'ended span' proxy.log
7

Setting attribute on ended span.
Setting attribute on ended span.
Setting attribute on ended span.
Setting attribute on ended span.
Setting attribute on ended span.
Tried calling set_status on an ended span.
Tried calling _add_event on an ended span.

and the transport carries nothing, because all of it was aimed at the initialize request's finished span:

SPAN name='tools/call speech_to_text'   trace_id=d06e2f1b...4d99 parent=1eb9977868fe260b status=ERROR
SPAN name='POST /{mcp_server_name}/mcp' trace_id=d06e2f1b...4d99 span_id=1eb9977868fe260b status=UNSET

After, on this branch, same sequence:

$ grep -c 'ended span' proxy.log
0

SPAN name='auth /mcp/probe'             trace_id=57c23926...2aea span_id=009fb6caf751a09b parent=5aecd6be09dc547d status=UNSET
SPAN name='tools/list'                  trace_id=57c23926...2aea span_id=5023e47b7a3eb810 parent=5aecd6be09dc547d status=UNSET
SPAN name='tools/call speech_to_text'   trace_id=57c23926...2aea span_id=f6358c06818d116b parent=5aecd6be09dc547d status=ERROR
SPAN name='POST /{mcp_server_name}/mcp' trace_id=63e55470...0a1f span_id=df2f661f6cf54d69 parent=-                status=UNSET
SPAN name='POST /{mcp_server_name}/mcp' trace_id=c6a9c05d...a32b span_id=22166cd5011dc910 parent=-                status=UNSET
SPAN name='POST /{mcp_server_name}/mcp' trace_id=57c23926...2aea span_id=5aecd6be09dc547d parent=-                status=ERROR

No writes to ended spans, and the POST that carried the call is the one marked ERROR. The two POSTs that did not fail, initialize and notifications/initialized, stay UNSET rather than absorbing another request's failure.

Type

🐛 Bug Fix

Changes

async_post_call_failure_hook now prefers the transport span published for the message being handled, falling back to the request-root anchor and then to user_api_key_dict.parent_otel_span exactly as before, so the pre-call failure paths (auth rejection, malformed body) are untouched.

The transport is published on the ASGI scope of the request being handled and read back through req_ctx.request, the Request the streamable-HTTP transport attaches to every message. That makes it per-message rather than per-session, which matters because a JSON-RPC response POST deliberately skips the per-session lock: it can arrive while the tool call awaiting it is still in flight, so a field on the shared auth object would be overwritten mid-call and the tool call's telemetry would land on the response's request. A scope belongs to one request and dies with it, so nothing keeps a finished span alive on idle session state either. This replaces the auth-context field #34537 introduced, which is removed.

It is the live Span, not the SpanContext #34537 carried. Parenting and linking only need the context, but stamping needs a span still open for writes, so one value now serves both and the context accessor derives from it. mcp_message_transport_span gates on is_recording(), not is_recordable_span: a finished span keeps a valid context forever, so the latter would hand back exactly the spans the SDK then refuses to write. The POST carrying a tools/call stays open until the result is written, so it is live for the duration of the call, while a notification POST can answer first, which is the case the gate exists for.

Publishing also re-anchors the request root for that message, so the identity attributes seeded onto the server span and any guardrail span emitted during the call land on the same request rather than on the session opener. The MCP SDK dispatches each message on its own task, so the anchor is scoped to the message. An already-finished transport is never anchored, since that would only move the dropped writes from one dead span to another.

Tests in test_otel_v2_logger.py add three cases, each emitting from a task spawned before the current request so the stale anchor is real rather than simulated: the failure lands on this message's transport and not on the session opener, a transport that already answered is skipped in favor of the anchor, and a guardrail emitted during the message parents to this message's transport.

QA runbook

1. Start an MCP server the proxy can reach, register it under mcp_servers, and start the proxy with LITELLM_OTEL_V2=1 and callbacks: ["otel"] pointed at any span exporter
2. From MCP Inspector over Streamable HTTP, run initialize and then tools/call for a tool that fails, so the result is HTTP 200 with isError: true
3. Confirm the proxy logs no Setting attribute on ended span or Tried calling set_status on an ended span warnings
4. In the exported spans, confirm the POST /<mcp_server_name>/mcp that carried the call has ERROR status and the error.* attributes, and that the initialize POST does not
5. Run a second failing tools/call on the same session and confirm the error follows the second POST, not the first

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR moves MCP transport-span propagation from shared session authentication state to each message's ASGI request scope, uses the live span for failure attribution and request-root anchoring, and adds regression coverage for stale and completed spans

Confidence Score: 5/5

The PR appears safe to merge

No blocking failures remain

Important Files Changed

Filename	Overview
litellm/integrations/otel/logger.py	Prefers the current MCP message's writable transport span when recording proxy-level failures
litellm/integrations/otel/plumbing/context.py	Carries the live per-message transport span through context and re-anchors request-level telemetry while it is recording
litellm/proxy/_experimental/mcp_server/server.py	Publishes transport spans on request-local ASGI scope and scopes their use to MCP message handling
litellm/proxy/_experimental/mcp_server/auth/litellm_auth_handler.py	Removes transport telemetry state from the session-retained authentication object
tests/test_litellm/integrations/otel/test_otel_v2_logger.py	Adds focused regression tests for failure attribution, completed-span fallback, and guardrail parenting

_{Reviews (2): Last reviewed commit: "fix(otel): stamp an MCP tool failure on ..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 87.80488% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/_experimental/mcp_server/server.py	76.19%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai please review the current head 4f61825. Both findings are addressed: the transport span moved off the shared session auth object onto the per-request ASGI scope, read back through req_ctx.request, so a lock-skipping JSON-RPC response POST can no longer overwrite an in-flight message's transport, and nothing retains a finished span on idle session state.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_mcp_failure_span_current_request (0515579) with litellm_internal_staging (a66bac3)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (96f58fa) during the generation of this report, so a66bac3 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩