feat(mcp): share compatibility-aware result conversion across tool surfaces

[devin-ai-integration]

TLDR

Problem this solves:

• MCP callers build results independently and lose OpenAPI structure
• Compatibility conversion has no shared owner

How it solves it:

• Share typed conversion with explicit legacy or modern compatibility
• Preserve OpenAPI response text and admit compatible structured values
• Preserve native content and metadata when converting SDK results
• Prepare interim-result propagation without enabling modern production access
• Keep SSE on the SDK legacy-only serving loop

User Flow

Before: an OpenAPI tool returns JSON only as text

1. A client calls reports-list_reports through POST http://localhost:4000/mcp/
2. The response contains JSON text without structuredContent
3. The client parses that text to use the object

After: the same tool also returns the parsed object

1. A client calls reports-list_reports through POST http://localhost:4000/mcp/
2. The response retains identical text and includes object structuredContent
3. Arrays remain readable text on legacy connections

Linear ticket

Resolves LIT-7742

Pre-Submission checklist

• I have added meaningful tests
• The affected test files pass locally
• My PR passes all required CI/CD checks
• My PR's scope is isolated to shared result conversion
• Current-tip Greptile confidence is at least 4/5

Validation

Review found a real exception to the version gate: headerless SSE can select modern mode through its opening message. Commit 72331cb uses the SDK legacy-only loop while preserving its lifespan. New regressions fail before the fix and pass afterward; live testing confirms modern rejection and unchanged legacy listing, calls, authentication, accounting, and cancellation. Greptile rates this commit 5/5, Veria confirms both prior concerns resolved with zero open security findings, and Cursor Bugbot reports no issues. All review threads are resolved

Commit 72331cb7bb7836109e66dfc27caca1c6b13db831: 2,045 affected tests pass with line and branch coverage. Local changed executable-line coverage is 122/123, 99.19%, up from the prior completed Codecov patch report's 97.52%. The completed hosted report covers 124/125 changed executable lines, 99.20%. All 16 uploaded reports are merged, including the MCP rerun; 15 unchanged reports are carried forward. Repository-wide coverage is 77.36%, a separate measure

The sole uncovered changed line is assert_never after every declared input arm returns. No coverage exclusions, thresholds, or budgets changed. All reachable branches in the new conversion modules are covered. Across entire touched functions, including existing authentication and request handling, local branch coverage is 181/216, 83.80%; this is separate from changed-line coverage

Fifteen added cases cover unsupported methods without HTTP traffic, request-context restoration, exact HTTP body classification, falsy structured values, preservation of image blocks and metadata, input immutability, and headerless modern SSE envelopes on both existing mount variants. The terminal exhaustiveness check now sits after the match to make the explicit return visible to CodeQL

Full make check passed in an independent managed checkout using the frozen canonical environment, with the changed files matching this commit byte-for-byte. This includes full-tree source/test Ruff, formatting, strict-rule, type-discipline, test-quality and basedpyright budget gates, operation-boundary and import checks, E2E type checking, and dashboard API-type synchronization. Independent Devin verification of the exact pushed tree also passes: live legacy MCP, the REST endpoint, SSE admission/authentication/cancellation, and the separately labeled private modern lanes below. CI is green: 89 checks pass and 13 are skipped. The MCP shard passed unchanged on its targeted rerun (380 passed, 5 skipped) after a transient timeout in an existing 200 ms connection-setup deadline; no assertion or timeout was weakened

Screenshots / Proof of Fix

Independent verification used Python 3.12.13 and MCP 2.2.0, real local HTTP upstreams, and source-built proxies at the hashes below. No LLM calls were needed. The normal gateway uses legacy upstream negotiation; modern results below use a separate, explicitly modified test process.

Shared setup: reference MCP upstream on 8765, OpenAPI service on 8766, proxy on 4000. LITELLM_KEY is a local test master key. Save the fixtures below in a separate directory, set TREE to the selected checkout, and use the frozen proxy development environment.

# config_base.yaml; no callbacks and no modern harness
model_list:
  - model_name: fake
    litellm_params:
      model: openai/fake
      api_key: fake
      api_base: http://127.0.0.1:1
general_settings:
  master_key: os.environ/LITELLM_KEY
litellm_settings:
  user_url_allowed_hosts: ["127.0.0.1"]
mcp_servers:
  ref:
    url: http://127.0.0.1:8765/mcp/
    transport: http
  reports:
    url: http://127.0.0.1:8766
    transport: http
    spec_path: http://127.0.0.1:8766/openapi.json

python upstream.py
# Separate terminal, restart this process for each checkout/lane:
cd "$TREE"
env -u DATABASE_URL DISABLE_SCHEMA_UPDATE=true PYTHONPATH="$TREE" \
  python litellm/proxy/proxy_cli.py --config "$FIXTURES/config_base.yaml" --port 4000

For the private modern lane only, copy config_base.yaml to config.yaml, add callbacks: harness_callbacks.tracker under litellm_settings, and add $FIXTURES to PYTHONPATH. This harness changes both the downstream version gate and upstream negotiation. It is not shipped or evidence that production enables modern MCP.

Reproduction fixtures, including the explicit private modern harness

upstream.py

"""Reference upstream servers for the LIT-7742 live proof.

Port 8765: SDK 2.x lowlevel MCP server (streamable HTTP, legacy handshake and modern discover)
with tools returning object, array and input_required results.
Port 8766: tiny JSON REST API with an OpenAPI spec, registered in the proxy as an OpenAPI MCP server.
"""
import contextlib
import sys
import threading

import uvicorn
from fastapi import FastAPI
from mcp.server.lowlevel import Server
from mcp.server.streamable_http_manager import StreamableHTTPSessionManager
from mcp.types import (
    CallToolResult,
    ListToolsResult,
    ElicitRequest,
    ElicitRequestFormParams,
    InputRequiredResult,
    TextContent,
    Tool,
)
from starlette.applications import Starlette
from starlette.routing import Mount

async def list_tools(ctx, params):
    return ListToolsResult(tools=[
        Tool(name="object_result", description="structured object", inputSchema={"type": "object"}),
        Tool(name="array_result", description="structured array", inputSchema={"type": "object"}),
        Tool(name="needs_input", description="interim input_required", inputSchema={"type": "object"}),
    ])


async def call_tool(ctx, params):
    name = params.name
    if name == "object_result":
        return CallToolResult(content=[TextContent(type="text", text="ok")], structuredContent={"answer": 42})
    if name == "array_result":
        return CallToolResult(content=[TextContent(type="text", text="Done")], structuredContent=[1, 2])
    if name == "needs_input":
        return InputRequiredResult(
            inputRequests={
                "req-1": ElicitRequest(
                    params=ElicitRequestFormParams(
                        message="Pick one", requestedSchema={"type": "object", "properties": {}}
                    )
                )
            },
            requestState="state-1",
        )
    raise ValueError(f"unknown tool {name}")


server = Server("ref-upstream", on_list_tools=list_tools, on_call_tool=call_tool)


manager = StreamableHTTPSessionManager(app=server, stateless=True)


@contextlib.asynccontextmanager
async def lifespan(_app):
    async with manager.run():
        yield


async def handle_mcp(scope, receive, send):
    await manager.handle_request(scope, receive, send)


mcp_app = Starlette(routes=[Mount("/mcp", app=handle_mcp)], lifespan=lifespan)

rest = FastAPI(title="reports", version="1.0.0")


@rest.get("/reports", operation_id="list_reports")
async def list_reports() -> dict:
    return {"total": 1.10, "items": []}


@rest.get("/tags", operation_id="list_tags")
async def list_tags() -> list[str]:
    return ["a", "b"]


if __name__ == "__main__":
    t = threading.Thread(
        target=lambda: uvicorn.run(rest, host="127.0.0.1", port=8766, log_level="warning"), daemon=True
    )
    t.start()
    uvicorn.run(mcp_app, host="127.0.0.1", port=8765, log_level="warning")
    sys.exit(0)

legacy_proof.sh

#!/usr/bin/env bash
# Legacy live proof: normal gateway on :4000, four handshake revisions, object / array / interim / OpenAPI tools
set -u
KEY="${LITELLM_KEY:?set LITELLM_KEY}"
URL=http://127.0.0.1:4000/mcp/
H=(-s -H "Authorization: Bearer $KEY" -H 'content-type: application/json' -H 'accept: application/json, text/event-stream')
call() { # $1=protocolVersion $2=session $3=id $4=method $5=params
  curl "${H[@]}" -H "mcp-protocol-version: $1" -H "mcp-session-id: $2" -X POST "$URL" \
    -d "{\"jsonrpc\":\"2.0\",\"id\":$3,\"method\":\"$4\",\"params\":$5}" | sed -n 's/^data: //p'
}
for V in 2024-11-05 2025-03-26 2025-06-18 2025-11-25; do
  echo "=== protocolVersion $V"
  INIT=$(curl "${H[@]}" -D - -X POST "$URL" -d "{\"jsonrpc\":\"2.0\",\"id\":0,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"$V\",\"capabilities\":{},\"clientInfo\":{\"name\":\"curl\",\"version\":\"1\"}}}")
  SID=$(printf '%s' "$INIT" | tr -d '\r' | awk 'tolower($1)=="mcp-session-id:"{print $2}')
  printf '%s' "$INIT" | sed -n 's/^data: //p' | jq -c '{negotiated: .result.protocolVersion}'
  curl "${H[@]}" -H "mcp-protocol-version: $V" -H "mcp-session-id: $SID" -X POST "$URL" -d '{"jsonrpc":"2.0","method":"notifications/initialized"}' >/dev/null
  for T in ref-object_result ref-array_result ref-needs_input reports-list_reports reports-list_tags; do
    echo "--- tools/call $T"
    call "$V" "$SID" 1 tools/call "{\"name\":\"$T\",\"arguments\":{}}" | jq -c '.result // .error'
  done
done

modern_harness.py

"""Test-only private modern harness for the LIT-7742 proof. Never ships; loaded into the proxy
process through PYTHONPATH by harness_callbacks so nothing in config.yaml or litellm changes.

1. Lifts the gateway's ASGI reject of non-handshake MCP-Protocol-Version headers so a 2026-07-28
   downstream client reaches the SDK's modern single-exchange path.
2. Makes the upstream MCPClient try the SDK's modern `server/discover` negotiation first and fall back
   to the legacy `initialize()`, logging the revision the upstream actually adopted.
"""
import json

from mcp.client.session import ClientSession
from mcp.shared.exceptions import MCPError

from litellm.proxy._experimental.mcp_server import server as gateway_server

LOG = "/tmp/lit7742-hooks.log"


def _write(event: str, detail: object) -> None:
    with open(LOG, "a") as fh:
        fh.write(json.dumps({"event": event, "detail": detail}) + "\n")


gateway_server.unsupported_protocol_version = lambda scope: None

_legacy_initialize = ClientSession.initialize


async def _modern_first_initialize(self: ClientSession):
    try:
        await self.discover()
    except MCPError:
        result = await _legacy_initialize(self)
        _write("upstream_negotiated", self.protocol_version)
        return result
    _write("upstream_negotiated", self.protocol_version)
    return self.initialize_result


ClientSession.initialize = _modern_first_initialize


import asyncio as _asyncio
from litellm.litellm_core_utils.litellm_logging import Logging as _Logging

_orig_success = _Logging.async_success_handler


async def _traced_success(self, *args, **kwargs):
    chain = []
    coro = _asyncio.current_task().get_coro()
    while coro is not None:
        frame = getattr(coro, "cr_frame", None)
        if frame is not None:
            chain.append(f"{frame.f_code.co_filename.rsplit('/',1)[-1]}:{frame.f_lineno} {frame.f_code.co_name}")
        coro = getattr(coro, "cr_await", None)
    st = getattr(_asyncio.current_task(), "_source_traceback", None)
    _write("success_chain", chain + ["SRC:"] + ([f"{f.filename.rsplit('/',1)[-1]}:{f.lineno} {f.name}" for f in st[-12:]] if st else ["no-src"]))
    return await _orig_success(self, *args, **kwargs)


_Logging.async_success_handler = _traced_success

modern_proof.py

"""Downstream 2026-07-28 client for the private modern proof: SDK ClientSession over streamable HTTP
against the gateway on :4000, negotiated with `server/discover`, calling the interim tool with
`allow_input_required=True` and printing the raw result frame."""
import asyncio
import os
import sys

from mcp.client.session import ClientSession
from mcp.client.streamable_http import streamable_http_client
import httpx

KEY = os.environ["LITELLM_KEY"]


async def main() -> None:
    headers = {"Authorization": f"Bearer {KEY}"}
    async with httpx.AsyncClient(headers=headers, timeout=60) as http:
        async with streamable_http_client("http://127.0.0.1:4000/mcp/", http_client=http) as (read, write, *_):
            async with ClientSession(read, write) as session:
                discover = await session.discover()
                print("downstream discover supportedVersions:", discover.supported_versions)
                print("downstream negotiated protocolVersion:", session.protocol_version)
                for tool in sys.argv[1:]:
                    result = await session.call_tool(tool, {}, allow_input_required=True)
                    print(f"--- tools/call {tool} ->", type(result).__name__)
                    print(result.model_dump_json(by_alias=True, exclude_none=True))


asyncio.run(main())

harness_callbacks.py

"""Test-only accounting tracker: appends one line per fired hook to /tmp/lit7742-hooks.log."""
import json
import traceback

from litellm.integrations.custom_logger import CustomLogger

LOG = "/tmp/lit7742-hooks.log"


def _write(event: str, detail: object) -> None:
    with open(LOG, "a") as fh:
        fh.write(json.dumps({"event": event, "detail": detail}) + "\n")


class Tracker(CustomLogger):
    async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
        _write("success", {"model": kwargs.get("model") or kwargs.get("call_type"), "type": type(response_obj).__name__, "stack": [l.strip() for l in traceback.format_stack()[-14:-1]]})

    async def async_log_failure_event(self, kwargs, response_obj, start_time, end_time):
        _write("failure", str(kwargs.get("exception"))[:200])

    async def async_post_mcp_tool_call_hook(self, kwargs, response_obj, start_time, end_time):
        _write("post_mcp_tool_call_hook", type(response_obj).__name__)
        return None


tracker = Tracker()

import modern_harness  # noqa: E402,F401  test-only: modern bypass, see module docstring

SSE probe, against the normal gateway without the modern harness: open an authenticated SSE connection without a protocol header, then post to its returned session URL. Use a fresh connection for each opening request. The independent run additionally used a logging-only callback and a raw slow upstream to check unchanged cancellation and hook behavior.

curl -s -N -H "Authorization: Bearer $LITELLM_KEY" \
  -H 'accept: text/event-stream' http://127.0.0.1:4000/mcp/sse > /tmp/sse-frames &
# Read the URL following event: endpoint in /tmp/sse-frames into ENDPOINT
curl -s -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -X POST "http://127.0.0.1:4000$ENDPOINT" -d '{"jsonrpc":"2.0","id":0,"method":"server/discover","params":{"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientInfo":{"name":"modern-client","version":"1"},"io.modelcontextprotocol/clientCapabilities":{}}}}'
cat /tmp/sse-frames

REST probe: look up reports with GET /v1/mcp/server, then use its ID with the actual mounted REST route:

SID=$(curl -s -H "Authorization: Bearer $LITELLM_KEY" \
  http://127.0.0.1:4000/v1/mcp/server | jq -r '.[]|select(.server_name=="reports").server_id')
curl -s -H "Authorization: Bearer $LITELLM_KEY" -H 'content-type: application/json' \
  -X POST http://127.0.0.1:4000/mcp-rest/tools/call \
  -d "{\"server_id\":\"$SID\",\"name\":\"list_reports\",\"arguments\":{}}"

Before (27d1974)

Legacy tool calls

1. Run bash legacy_proof.sh against the normal gateway. Each requested revision (2024-11-05, 2025-03-26, 2025-06-18, 2025-11-25) is echoed by initialization
2. reports-list_reports returns only text:

   {"content":[{"text":"{\"total\":1.1,\"items\":[]}","type":"text"}],"isError":false}

3. reports-list_tags returns text ["a","b"] without structuredContent. Native ref-object_result retains {"answer":42}; ref-array_result and ref-needs_input both return isError:true with MCPError: Handler returned an invalid result because the upstream SDK rejects them under legacy negotiation

REST result shape

1. Run the REST probe above; repeat with list_tags. Both return HTTP 200
2. list_reports contains the exact JSON text and "structuredContent":null; list_tags also has null structure. Both already contain "resultType":"complete" and null metadata fields through FastAPI serialization

Private modern propagation

1. Restart with the private harness; run python modern_proof.py needs_input ref-array_result reports-list_tags reports-list_reports ref-object_result. Both connections negotiate 2026-07-28
2. needs_input returns an error CallToolResult: RuntimeError: Server returned InputRequiredResult; pass allow_input_required=True to receive it and retry call_tool(..., input_responses=..., request_state=result.request_state). The failure handler and post-call hook run
3. OpenAPI arrays and objects remain text-only; the native array retains structuredContent:[1,2]

Private modern upstream with legacy downstream

1. Keep the private harness enabled; run bash legacy_proof.sh again. The downstream uses each legacy revision while the upstream negotiates 2026-07-28
2. ref-array_result fails on the gateway wire:

   {"code":-32603,"message":"Handler returned an invalid result"}

SSE protocol admission

1. Run the SSE server/discover probe above. The merge base accepts the headerless modern envelope and advertises supportedVersions:["2026-07-28"]
2. Opening with enveloped tools/list instead also succeeds and emits resultType:"complete". This admission gap already exists at the merge base
3. Standard legacy initialization, list, and tool calls work; cancelling the slow request suppresses its response. Unauthenticated GET /mcp/sse returns HTTP 401

After (72331cb)

Legacy tool calls

1. Run the same bash legacy_proof.sh against the normal gateway. All four initialization revisions match the baseline
2. reports-list_reports retains identical text and adds the parsed object:

   {"content":[{"text":"{\"total\":1.1,\"items\":[]}","type":"text"}],"isError":false,"structuredContent":{"total":1.1,"items":[]}}

3. OpenAPI array text, native object structure, and upstream SDK error results are unchanged. No resultType is added to these legacy MCP frames

REST result shape

1. Run the same REST probe and repeat with list_tags. Both still return HTTP 200
2. Only the object structure changes: list_reports now has "structuredContent":{"total":1.1,"items":[]}. Original text, resultType, null metadata, and the array response shape remain unchanged. Independent logging captures confirm the same success/error accounting and post-call hooks

Private modern propagation

1. Restart with the same private harness and run the same modern_proof.py command. Both connections negotiate 2026-07-28
2. needs_input now returns the original interim result:

   {"_meta":{"io.modelcontextprotocol/serverInfo":{"name":"ref-upstream","version":""}},"resultType":"input_required","inputRequests":{"req-1":{"method":"elicitation/create","params":{"mode":"form","message":"Pick one","requestedSchema":{"properties":{},"type":"object"}}}},"requestState":"state-1"}

   This interim result triggers zero completed-success, failure, or post-call hooks
3. OpenAPI arrays gain structuredContent:["a","b"]; objects gain structuredContent:{"total":1.1,"items":[]}. Both retain original text. Native array structure remains unchanged

Private modern upstream with legacy downstream

1. Keep the same private harness enabled and run bash legacy_proof.sh again
2. ref-array_result now retains its existing block and appends the serialized array, without legacy-incompatible structure:

   {"_meta":{"io.modelcontextprotocol/serverInfo":{"name":"ref-upstream","version":""}},"content":[{"text":"Done","type":"text"},{"text":"[1, 2]","type":"text"}],"isError":false}

SSE protocol admission

1. Run the same SSE server/discover probe. The response is now:

   {"jsonrpc":"2.0","id":0,"error":{"code":-32601,"message":"Method not found","data":"server/discover"}}

2. Opening with the same enveloped tools/list instead returns -32602, Invalid request parameters. No tool calls reach the upstream and no tool hooks run
3. Standard legacy initialization, list, tool calls, cancellation, and unauthenticated HTTP 401 are unchanged

Type

New feature, refactoring, tests

Caveats

The live hybrid-upstream probe did not reproduce interim accounting or guardrail bypass: the SDK strips later-revision fields on legacy upstream connections and normal hooks run. The confirmed defect was SSE modern admission, now closed. Interim charging and output inspection remain requirements for any future modern activation

Medium

• Production modern access remains disabled on both transports
• Upstream connections still initialize using legacy negotiation
• Invalid native arrays can be rejected before gateway conversion
• Interim orchestration and output-policy handling await later activation

Low

• Private modern proof explicitly changes both negotiation paths
• JSON null remains text without a structured claim
• REST keeps its existing model serialization shape

Final Attestation

• Current-tip CI, coverage, independent verification and required bot reviews are complete

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Refactors tool result conversion to support protocol versions.

No outstanding finding or established new defect blocks merging.

Summary

The PR centralizes compatibility-aware MCP tool-result conversion, preserves OpenAPI response text while admitting structured JSON where supported, and changes legacy SSE serving to use the SDK loop directly.

• Added conversion coverage for legacy and modern result shapes.
• Added SSE coverage for endpoint delivery, rejection of modern discovery, per-request authentication context, and session ownership.

_{Reviews (3) · Last reviewed commit: "fix(mcp): keep SSE connections on the le..."}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_mcp_shared_result_conversion (72331cb) with main (8b3939d)¹}

Footnotes

1. No successful run was found on main (2701e20) during the generation of this report, so 8b3939d was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[codecov]

Codecov Report

❌ Patch coverage is 99.20000% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...roxy/_experimental/mcp_server/result_conversion.py	97.87%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[joshua-berri]

@greptileai @veria-ai Please review 87f9a28 for compatibility, accounting, and lossless conversion. Added boundary regressions and clarified the exhaustive return

[joshua-berri]

bugbot run

Please review commit 87f9a28, including legacy conversion, interim result handling, error accounting, and the added regression tests

[cursor]

Stale Bugbot comment from a previous run.

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

[joshua-berri]

@greptileai @veria-ai Please review 72331cb: SSE now uses the SDK legacy-only loop. Live tests confirm modern rejection and unchanged legacy calls, accounting, authentication, cancellation

[joshua-berri]

bugbot run

Please review 72331cb, especially the SSE legacy-only SDK loop, preserved lifespan handling, modern-envelope regression, and existing initialization behavior

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 72331cb. Configure here.}