Skip to content

fix: preserve structured_content when ResponseLimitingMiddleware truncates#3740

Closed
voidborne-d wants to merge 3 commits intoPrefectHQ:mainfrom
voidborne-d:fix/response-limiting-preserve-structured-content
Closed

fix: preserve structured_content when ResponseLimitingMiddleware truncates#3740
voidborne-d wants to merge 3 commits intoPrefectHQ:mainfrom
voidborne-d:fix/response-limiting-preserve-structured-content

Conversation

@voidborne-d
Copy link
Copy Markdown
Contributor

@voidborne-d voidborne-d commented Apr 1, 2026

Fixes #3717.

ResponseLimitingMiddleware drops structured_content when truncating, causing outputSchema defined but no structured output returned for tools with return type annotations.

Fix: Pass structured_content through _truncate_to_result() instead of discarding it. Subtract its serialized size from the text budget so combined response respects max_size.

MRE:

from pydantic import BaseModel
from fastmcp import FastMCP, Client
from fastmcp.server.middleware import ResponseLimitingMiddleware

class Answer(BaseModel):
    text: str

mcp = FastMCP()
mcp.add_middleware(ResponseLimitingMiddleware(max_size=500))

@mcp.tool()
def big() -> Answer:
    return Answer(text="x" * 2000)

async with Client(mcp) as c:
    r = await c.call_tool("big")  # raises output validation error without fix
    assert r.structured_content is not None

…cates

When a tool declares an outputSchema (via return type annotation or
explicit output_schema), its response includes structured_content.
ResponseLimitingMiddleware dropped structured_content during truncation,
producing a ToolResult with only text content. This caused downstream
MCP SDK output validation to fail:
  'outputSchema defined but no structured output returned'

The fix preserves the original structured_content through truncation.
The serialized size of structured_content is subtracted from the text
budget so the combined response stays within max_size.

Closes PrefectHQ#3717
@marvin-context-protocol marvin-context-protocol Bot added the too-long Excessively verbose or unedited LLM output. Condense before triage. label Apr 1, 2026
@marvin-context-protocol
Copy link
Copy Markdown
Contributor

Thanks for the report. This issue goes beyond what our contributor guidelines ask for — we just need a short problem description and an MRE. Please see our contributing guidelines and condense this issue. We'll triage it once it's trimmed down.

@marvin-context-protocol
Copy link
Copy Markdown
Contributor

Test Failure Analysis

Summary: The static analysis ( type checker) fails because a new test accesses .text on result.content[0] without narrowing the union type first.

Root Cause: In tests/server/middleware/test_response_limiting.py:204, the test does:

assert "[Response truncated" in result.content[0].text

result.content is typed as a list of TextContent | ImageContent | AudioContent | ResourceLink | EmbeddedResource. Since only TextContent has a .text attribute, ty raises error[unresolved-attribute]. The same pattern likely appears in the other two new tests at similar lines.

Suggested Solution: Narrow the type before accessing .text by adding an isinstance assertion. In each place where result.content[0].text is accessed in the new tests, add a guard:

# Before:
assert "[Response truncated" in result.content[0].text

# After:
assert isinstance(result.content[0], TextContent)
assert "[Response truncated" in result.content[0].text

This requires importing TextContent in the test file if it is not already imported (it likely is, given the existing tests use it).

Detailed Analysis

Failing check: ty check (type checker hook in prek)

Log excerpt:

error[unresolved-attribute]: Attribute `text` is not defined on `ImageContent`, `AudioContent`, `ResourceLink`, `EmbeddedResource` in union `TextContent | ImageContent | AudioContent | ResourceLink | EmbeddedResource`
  --> tests/server/middleware/test_response_limiting.py:204:41
    |
202 |         result = middleware._truncate_to_result("a" * 1000, structured_content=sc)
203 |         assert result.structured_content == sc
204 |         assert "[Response truncated" in result.content[0].text
    |                                         ^^^^^^^^^^^^^^^^^^^^^^
    |
info: rule `unresolved-attribute` is enabled by default
Found 1 diagnostic

The same .content[0].text pattern is also used in the integration tests added in this PR (e.g. test_structured_content_preserved_on_truncation and test_truncation_without_structured_content_still_works). Those did not trigger a separate ty error in this run, possibly because ty stops at the first error per file — worth checking and fixing all occurrences preemptively.

Related Files
  • tests/server/middleware/test_response_limiting.py — test file with the failing assertion (line 204 and similar lines in the other two new tests)
  • src/fastmcp/server/middleware/middleware.pyResponseLimitingMiddleware._truncate_to_result() implementation being tested

voidborne-d and others added 2 commits April 1, 2026 20:54
Narrow content[0] to TextContent before accessing .text in
test_structured_content_preserved_on_truncation, fixing ty
unresolved-attribute error.
@jlowin
Copy link
Copy Markdown
Member

jlowin commented Apr 3, 2026

Thanks for the PR. The diagnosis is right — truncation breaks outputSchema tools — but the original PR (#3072) intentionally excluded structured_content from truncated results. Structured content is often the bulk of the payload, and you can't safely truncate JSON without understanding the schema — cutting it wrong silently corrupts data. Passing it through unmodified defeats the middleware's size-limiting purpose.

We're going to fix this by stripping the output schema when truncation kicks in, so the SDK doesn't expect structured content that no longer applies. Closing in favor of that approach.

@jlowin jlowin closed this Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

too-long Excessively verbose or unedited LLM output. Condense before triage.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ResponseLimitingMiddleware truncation breaks tools with outputSchema

2 participants