fix(anthropic): fix response_format for claude-fable-5-1 on Vertex AI and Bedrock

[devin-ai-integration]

TLDR

Problem this solves:

• response_format 400s on claude-fable-5-1 via Vertex AI and Bedrock
• Vertex and Invoke overrides stub the model before capability checks
• Converse sent native output_config, which Bedrock Fable rejects
• Tool fallback forced tool_choice, which Fable also rejects

How it solves it:

• Vertex override skips the stub for native structured output models
• Bedrock Fable map entries stop advertising native structured output
• Converse and Invoke use the tool fallback without forced tool_choice
• Backstop keys off supports_forced_tool_use, no hardcoded model names

User Flow

Before: a developer asking for JSON output from claude-fable-5-1 through Bedrock gets a 400

1. They send POST https://litellm-domain/v1/chat/completions with model fable-bedrock-converse and a response_format json_schema
2. They get HTTP 400: output_config.format: Extra inputs are not permitted
3. They retry with model fable-bedrock-invoke and the same body
4. They get HTTP 400: tool_choice: type "tool" and "any" are not supported for this model.

After: the same requests return schema-conforming JSON

1. They send POST https://litellm-domain/v1/chat/completions with model fable-bedrock-converse and a response_format json_schema
2. They get HTTP 200 with content like {"city": "Lisbon", "country": "Portugal"}
3. They retry with model fable-bedrock-invoke and the same body
4. They get HTTP 200 with schema-conforming JSON as well

Relevant issues

Follow-up to #39148 (see the Bugbot thread at #39148 (comment))

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup: proxy on localhost:4000 with model_list entries fable-anthropic (anthropic/claude-fable-5-1), fable-bedrock-converse (bedrock/us.anthropic.claude-fable-5-1) and fable-bedrock-invoke (bedrock/invoke/us.anthropic.claude-fable-5-1), all hitting real provider APIs. Every request is the same curl:

curl -sS -X POST http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model":"<model>","messages":[{"role":"user","content":"Give me a random city and its country."}],
       "response_format":{"type":"json_schema","json_schema":{"name":"city","schema":{"type":"object",
       "properties":{"city":{"type":"string"},"country":{"type":"string"}},
       "required":["city","country"],"additionalProperties":false}}}}'

Before (a3e115f)

bedrock converse

1. curl above with "model":"fable-bedrock-converse"
2. HTTP 400: BedrockException - {"message":"The model returned the following errors: output_config.format: Extra inputs are not permitted"}

bedrock invoke

1. curl above with "model":"fable-bedrock-invoke"
2. HTTP 400: BedrockException - {"message":"tool_choice: type \"tool\" and \"any\" are not supported for this model."}

After (d568bbe)

bedrock converse

1. curl above with "model":"fable-bedrock-converse"
2. HTTP 200, content {"city": "Lisbon", "country": "Portugal"} (real Bedrock call, non-zero spend)

bedrock invoke

1. curl above with "model":"fable-bedrock-invoke"
2. HTTP 200, content {"city": "Lisbon", "country": "Portugal"} (real Bedrock call, non-zero spend)

anthropic direct (regression control)

1. curl above with "model":"fable-anthropic"
2. HTTP 200, content {"city":"Porto","country":"Portugal"}, still on the native structured output path

Type

🐛 Bug Fix

Caveats (if any)

Medium

• Vertex AI path is unit-tested only, live quota for the Fable base model is 0 in our test project
• Without a forced tool_choice the model can occasionally answer in prose instead of calling the JSON tool, same limitation as the existing thinking-enabled fallback

Link to Devin session: https://app.devin.ai/sessions/91161c53a340462b8b5ca0d9ad54359b
Open in Devin Desktop: https://app.devin.ai/desktop/session/91161c53a340462b8b5ca0d9ad54359b?variant=devin
Requested by: @mateo-berri

---

Note

Medium Risk
Changes how response_format is translated across Anthropic, Bedrock, and Vertex based on model-map flags; wrong flags could break structured output or reintroduce provider 400s, but scope is limited to capability checks and tests.

Overview
Fixes HTTP 400 when response_format is used with claude-fable-5-1 (and similar models with supports_forced_tool_use: false) on Bedrock and Vertex.

Capability-driven routing: Adds AnthropicModelInfo.forced_tool_use_unsupported() and uses it so the JSON-schema tool fallback no longer sets a forced tool_choice on models that reject it (same idea as the existing thinking-enabled path). Bedrock Converse and direct Anthropic mapping get this guard; Bedrock Invoke also strips a response-format tool_choice left over from the parent stub-model path.

Provider overrides: Vertex only forces the tool-based stub when native structured output is not advertised for that model. Bedrock Invoke skips the stub when supports_native_structured_output is true for bedrock. Model map updates set Bedrock Fable entries to supports_native_structured_output: false so Converse stops sending native outputConfig that Bedrock rejects, while Vertex can still use output_format where the map says native SO is supported.

Regression tests cover Anthropic, Bedrock Converse/Invoke, and Vertex paths.

^{Reviewed by Cursor Bugbot for commit d568bbe. Bugbot is set up for automated code reviews on this repo. Configure here.}

• d568bbe passes /live-pr-risk

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

PR #39184 (BerriAI/litellm, author devin-ai-integration[bot]) has no enterprise label — out of scope. No changes made; no risk label applied, no routing.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fable_5_1_structured_output (d568bbe) with litellm_internal_staging (c913b09)}

[greptile-apps]

Greptile Summary

This PR corrects structured-output capability handling for Claude Fable 5.1 across Anthropic, Bedrock, and Vertex AI

• Uses model capability metadata to avoid unsupported forced tool selection
• Routes Bedrock Fable requests through the tool fallback instead of native output configuration
• Preserves native structured output for supported Anthropic and Vertex AI routes
• Adds focused regression coverage for the affected provider transformations

Confidence Score: 5/5

The PR appears safe to merge, with the remaining unforced-tool limitation explicitly documented

The changed provider transformations consistently use capability metadata to avoid request parameters rejected by Fable 5.1, and no unacknowledged blocking failure remains

Important Files Changed

Filename	Overview
litellm/llms/anthropic/chat/transformation.py	Skips generated forced tool selection when model metadata explicitly marks forced tool use unsupported
litellm/llms/anthropic/common_utils.py	Adds a shared capability helper and reuses it in forced-tool downgrade handling
litellm/llms/bedrock/chat/converse_transformation.py	Prevents unsupported forced tool choice on Bedrock Converse structured-output fallbacks
litellm/llms/bedrock/chat/invoke_transformations/anthropic_claude3_transformation.py	Selects native output from provider metadata and removes only the synthetic response-format tool choice when unsupported
litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/transformation.py	Preserves the original model for Vertex models explicitly advertising native structured output
model_prices_and_context_window.json	Aligns Bedrock and Vertex Fable 5.1 structured-output capabilities with provider behavior
litellm/model_prices_and_context_window_backup.json	Mirrors the primary model catalog capability updates

_{Reviews (1): Last reviewed commit: "fix(bedrock): use tool fallback without ..." | Re-trigger Greptile}

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d568bbe. Configure here.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

Lgtm