Skip to content

fix(anthropic): use native structured output for Sonnet 5 - #36082

Open
le-shi wants to merge 1 commit into
BerriAI:mainfrom
le-shi:fix-anthropic-sonnet-5-structured-output
Open

le-shi wants to merge 1 commit into
BerriAI:mainfrom
le-shi:fix-anthropic-sonnet-5-structured-output

Conversation

@le-shi

@le-shi le-shi commented Aug 6, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • Sonnet 5 structured responses can leak tool envelopes

How it solves it:

  • Route Sonnet 5 through Anthropic native output_format

Relevant issues

Fixes #35677

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

Not available in this environment because Anthropic credentials are not configured

Type

Bug Fix

Changes

Adds supports_native_structured_output to the direct Anthropic claude-sonnet-5 entry in both model maps and adds a regression test covering Responses structured output routing

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds native structured-output capability metadata for the direct Anthropic claude-sonnet-5 model so JSON-schema requests use Anthropic's output_format instead of the tool workaround.

  • Keeps the primary and bundled backup model maps synchronized.
  • Adds a focused regression test asserting native output-format routing without generated tools or tool choice.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code failure identified.

The synchronized capability metadata is consumed by the existing Anthropic structured-output mapping, and the focused regression test verifies that the exact Sonnet 5 model selects native output formatting rather than the tool workaround.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Marks the direct Anthropic Sonnet 5 entry as supporting native structured output.
litellm/model_prices_and_context_window_backup.json Mirrors the Sonnet 5 capability flag in the bundled backup model map.
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py Adds a regression test confirming Sonnet 5 maps JSON-schema response formats to native output format without tool fallback.

Reviews (1): Last reviewed commit: "fix(anthropic): use native structured ou..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing le-shi:fix-anthropic-sonnet-5-structured-output (26f1a3b) with litellm_internal_staging (b66d4e6)

Open in CodSpeed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Responses API — Anthropic structured output leaks the tool-call envelope as output_text for models off the native allowlist (claude-sonnet-5)

2 participants