fix(bedrock): send json_schema as a forced tool on Claude Opus 4.7 and 4.8 Converse

[devin-ai-integration]

TLDR

Problem this solves:

• json_schema on Bedrock Claude Opus 4.7 and 4.8 returns HTTP 400
• Bedrock rejects output_config.format on those two models ("Extra inputs are not permitted")
• The cost map still marked them supports_native_structured_output: true

How it solves it:

• Flip that flag to false on all 15 Bedrock Opus 4.7 and 4.8 entries
• LiteLLM then sends the schema as a forced json_tool_call tool, like other non-native models
• Regression test covers all 15 entries, stream and non-stream; Sonnet 4.6 stays native

User Flow

Before: a developer calling Bedrock Claude Opus 4.8 from the litellm Python SDK's Responses API with a pydantic text_format gets a 400 from Bedrock instead of JSON

1. They run litellm.responses(model="bedrock/us.anthropic.claude-opus-4-8", input=[one user message "why is the sky blue? format your response as json"], text_format=TextResponse), where TextResponse is a pydantic model with a single text: str field; the SDK sends POST https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-opus-4-8/converse
2. The call raises litellm.BadRequestError: BedrockException - {"message":"The model returned the following errors: output_config.format: Extra inputs are not permitted"}, so TextResponse.model_validate_json(response.output[0].content[0].text) never runs
3. They retry with stream=True and the same BadRequestError is raised before any event arrives
4. The same call with model="bedrock/us.anthropic.claude-sonnet-4-6" returns a response whose output[0].content[0].text validates as TextResponse, so only Opus 4.7 and 4.8 fail

After: the same Opus 4.8 call returns a response whose text validates against the pydantic model

1. They run the same litellm.responses(model="bedrock/us.anthropic.claude-opus-4-8", ..., text_format=TextResponse) call; the SDK sends the same POST https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-opus-4-8/converse
2. The call returns, response.output[0].content[0].text is a JSON object with the text key the schema asked for, and TextResponse.model_validate_json(...) succeeds
3. They retry with stream=True; the response.output_text.delta events add up to the same JSON object and the stream ends with response.completed
4. The same call with model="bedrock/us.anthropic.claude-sonnet-4-6" still validates as TextResponse

Relevant issues

Fixes #27846

Affected release

Linear ticket

Resolves LIT-8378

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Two legs against real Bedrock (account 888602223428, us-east-1, real $$$), identical except for the LiteLLM commit: the merge base with main (1953603) and commit 6135e76. Each leg ran two things

b214a41 landed after this run and only rewrites one tests/unit Invoke test (nothing under litellm/ or the cost maps changed), so it cannot change behavior and the proof keeps 6135e76

1. The reporter's own client: the litellm Python SDK responses() call from [Bug]: Structured Output fails for Anthropic models on bedrock/converse #27846 with a pydantic text_format, run as a plain script with LITELLM_LOCAL_MODEL_COST_MAP=True so each leg reads its own checkout's cost map
2. Every unified endpoint through a proxy booted from that commit with python litellm/proxy/proxy_cli.py --config config.yaml --port $PORT --num_workers 2 (one proxy process, 2 uvicorn workers, no database, LITELLM_LOCAL_MODEL_COST_MAP=True): /v1/chat/completions, /v1/responses, and /v1/messages, stream and non-stream, with Sonnet 4.6 as the control on each

config.yaml maps bedrock-opus-4-8, bedrock-opus-4-7, and bedrock-sonnet-4-6 to bedrock/us.anthropic.claude-opus-4-8, bedrock/us.anthropic.claude-opus-4-7, and bedrock/us.anthropic.claude-sonnet-4-6 with drop_params: true

SDK script (the reporter's flow):

# the reporter's snippet from #27846, model swapped to the Opus 4.8 US cross-region id (our creds live in us-east-1)
from litellm import responses
from pydantic import BaseModel


class TextResponse(BaseModel):
    text: str


response = responses(
    model="bedrock/us.anthropic.claude-opus-4-8",
    input=[{"role": "user", "content": [{"type": "input_text", "text": "why is the sky blue? format your response as json"}]}],
    text_format=TextResponse,
)
TextResponse.model_validate_json(response.output[0].content[0].text)
# then the same call with stream=True (collecting response.output_text.delta events),
# then with model="bedrock/us.anthropic.claude-opus-4-7", then the Sonnet 4.6 control

Proxy requests (the leg loops MODEL over the three deployments; the output blocks below print the content, finish reason, and HTTP status of each):

SCHEMA='{"type":"object","properties":{"city":{"type":"string"},"country":{"type":"string"}},"required":["city","country"],"additionalProperties":false}'
# /v1/chat/completions, non-stream and with "stream": true, for bedrock-opus-4-8, bedrock-opus-4-7, bedrock-sonnet-4-6
curl -sS -w '\nHTTP %{http_code}\n' "http://localhost:${PORT}/v1/chat/completions" -H "Authorization: Bearer ${KEY}" -H 'Content-Type: application/json' \
  -d "{\"model\":\"${MODEL}\",\"messages\":[{\"role\":\"user\",\"content\":\"Name the capital of France.\"}],\"response_format\":{\"type\":\"json_schema\",\"json_schema\":{\"name\":\"capital\",\"schema\":${SCHEMA},\"strict\":true}}}"
# /v1/responses with the reporter's text.format json_schema (TextResponse), non-stream and with "stream": true
curl -sS -w '\nHTTP %{http_code}\n' "http://localhost:${PORT}/v1/responses" -H "Authorization: Bearer ${KEY}" -H 'Content-Type: application/json' \
  -d '{"model":"bedrock-opus-4-8","input":[{"role":"user","content":[{"type":"input_text","text":"why is the sky blue? format your response as json"}]}],"text":{"format":{"type":"json_schema","name":"TextResponse","schema":{"type":"object","properties":{"text":{"type":"string"}},"required":["text"],"additionalProperties":false},"strict":true}}}'
# /v1/messages with output_config.format json_schema, for bedrock-opus-4-8 and bedrock-sonnet-4-6
curl -sS -w '\nHTTP %{http_code}\n' "http://localhost:${PORT}/v1/messages" -H "Authorization: Bearer ${KEY}" -H 'Content-Type: application/json' \
  -d "{\"model\":\"${MODEL}\",\"max_tokens\":200,\"messages\":[{\"role\":\"user\",\"content\":\"Name the capital of France.\"}],\"output_config\":{\"format\":{\"type\":\"json_schema\",\"schema\":${SCHEMA}}}}"

Before (1953603)

SDK, the reporter's flow: Opus 4.8 and 4.7 raise the 400, Sonnet 4.6 validates

===== [BASE] litellm.responses(model='bedrock/us.anthropic.claude-opus-4-8', text_format=TextResponse, stream=False)

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

BadRequestError: litellm.BadRequestError: BedrockException - {"message":"The model returned the following errors: output_config.format: Extra inputs are not permitted"}
===== [BASE] litellm.responses(model='bedrock/us.anthropic.claude-opus-4-8', text_format=TextResponse, stream=True)

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

BadRequestError: litellm.BadRequestError: BedrockException - b'{"message":"The model returned the following errors: output_config.format: Extra inputs are not permitted"}'
===== [BASE] litellm.responses(model='bedrock/us.anthropic.claude-opus-4-7', text_format=TextResponse, stream=False)

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

BadRequestError: litellm.BadRequestError: BedrockException - {"message":"The model returned the following errors: output_config.format: Extra inputs are not permitted"}
===== [BASE] litellm.responses(model='bedrock/us.anthropic.claude-sonnet-4-6', text_format=TextResponse, stream=False)
output[0].content[0].text: {"text": "The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters Earth's atmosphere, it collides with gas molecules. Sunlight is made up of all colors of the visible spectrum, each with different wavelengths. Blue light has a shorter wavelength compared to other co
TextResponse.model_validate_json -> {"text": "The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters Earth's atmosphere, it collides with gas molecules. Sunlight is made up of all colors of the visible

Proxy, every unified endpoint: 7 x HTTP 400 on Opus 4.8 and 4.7, 3 x HTTP 200 on the Sonnet 4.6 control

===== [BASE] bedrock-opus-4-8 chat/completions json_schema non-stream
{"error": {"message": "litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: output_config.format: Extra inputs are not permitted\"}\n\nLiteLLM: model group 'bedrock-opus-4-8' failed with the error above. No fallback was attempted.", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400
===== [BASE] bedrock-opus-4-8 chat/completions json_schema stream
streamed content: <none>
non-delta lines: ['{"error":{"message":"litellm.BadRequestError: BedrockException - {\\"message\\":\\"The model returned the following errors: output_config.format: Extra inputs are not permitted\\"}\\n\\nLiteLLM: model group \'bedrock-opus-4-8\' failed with the error above. No fallback was attempted.","type":"invalid_request']
HTTP 400
===== [BASE] bedrock-opus-4-7 chat/completions json_schema non-stream
{"error": {"message": "litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: output_config.format: Extra inputs are not permitted\"}\n\nLiteLLM: model group 'bedrock-opus-4-7' failed with the error above. No fallback was attempted.", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400
===== [BASE] bedrock-opus-4-7 chat/completions json_schema stream
streamed content: <none>
non-delta lines: ['{"error":{"message":"litellm.BadRequestError: BedrockException - {\\"message\\":\\"The model returned the following errors: output_config.format: Extra inputs are not permitted\\"}\\n\\nLiteLLM: model group \'bedrock-opus-4-7\' failed with the error above. No fallback was attempted.","type":"invalid_request']
HTTP 400
===== [BASE] bedrock-sonnet-4-6 chat/completions json_schema non-stream
content: {"city":"Paris","country":"France"}
finish_reason: stop
HTTP 200
===== [BASE] bedrock-sonnet-4-6 chat/completions json_schema stream
streamed content: {"city":"Paris","country":"France"}
HTTP 200
===== [BASE] bedrock-opus-4-8 /v1/responses text.format json_schema (reporter flow)
{"error": {"message": "litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: output_config.format: Extra inputs are not permitted\"}\n\nLiteLLM: model group 'bedrock-opus-4-8' failed with the error above. No fallback was attempted.", "type": "invalid_request_error", "param": null, "code": "400"}}
HTTP 400
===== [BASE] bedrock-opus-4-8 /v1/responses text.format json_schema stream (reporter flow)
streamed output_text: <none>
last event: <none>
{"error":{"message":"litellm.BadRequestError: BedrockException - {\"message\":\"The model returned the following errors: output_config.format: Extra inputs are not permitted\"}\n\nLiteLLM: model group 'bedrock-opus-4-8' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400
===== [BASE] bedrock-opus-4-8 /v1/messages output_config.format json_schema
{"type": "error", "error": {"type": "invalid_request_error", "message": "litellm.BadRequestError: BedrockException - {\"message\":\"output_config.format: Extra inputs are not permitted\"}\n\nLiteLLM: model group 'bedrock-opus-4-8' failed with the error above. No fallback was attempted."}}
HTTP 400
===== [BASE] bedrock-sonnet-4-6 /v1/messages output_config.format json_schema
content: [{"type": "text", "text": "{\"city\":\"Paris\",\"country\":\"France\"}"}]
stop_reason: end_turn
HTTP 200

After (6135e76)

SDK, the reporter's flow: every call returns and TextResponse.model_validate_json succeeds, the stream ends in response.completed

===== [TIP] litellm.responses(model='bedrock/us.anthropic.claude-opus-4-8', text_format=TextResponse, stream=False)
output[0].content[0].text: {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"answer\": {\n    \"summary\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, in which molecules in Earth's atmosphere scatter shorter-wavelength (blue) light more than longer-wavelength (red) light.\",\n    \"exp
TextResponse.model_validate_json -> {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"answer\": {\n    \"summary\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, in which molecules in Earth's atmos
===== [TIP] litellm.responses(model='bedrock/us.anthropic.claude-opus-4-8', text_format=TextResponse, stream=True)
streamed output_text: {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"answer\": {\n    \"short_explanation\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, in which the shorter blue wavelengths of sunlight are scattered more strongly by molecules in Earth's atmosphere than the lon
last event: ResponsesAPIStreamEvents.RESPONSE_COMPLETED
TextResponse.model_validate_json -> {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"answer\": {\n    \"short_explanation\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, in which the shorter blue
===== [TIP] litellm.responses(model='bedrock/us.anthropic.claude-opus-4-7', text_format=TextResponse, stream=False)
output[0].content[0].text: {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"short_answer\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, in which shorter (blue) wavelengths of sunlight are scattered by molecules in Earth's atmosphere much more than longer (red) wavelengths.\",\n  \"det
TextResponse.model_validate_json -> {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"short_answer\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, in which shorter (blue) wavelengths of sunlight a
===== [TIP] litellm.responses(model='bedrock/us.anthropic.claude-sonnet-4-6', text_format=TextResponse, stream=False)
output[0].content[0].text: {"text": "The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters Earth's atmosphere, it collides with gas molecules. Sunlight is made up of all colors of the visible spectrum, each with different wavelengths. Blue light has a shorter wavelength compared to other co
TextResponse.model_validate_json -> {"text": "The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters Earth's atmosphere, it collides with gas molecules. Sunlight is made up of all colors of the visible

Proxy, every unified endpoint: 10 x HTTP 200, schema-matching JSON on each

===== [TIP] bedrock-opus-4-8 chat/completions json_schema non-stream
content: {"city": "Paris", "country": "France"}
finish_reason: stop
HTTP 200
===== [TIP] bedrock-opus-4-8 chat/completions json_schema stream
streamed content: {"city": "Paris", "country": "France"}
HTTP 200
===== [TIP] bedrock-opus-4-7 chat/completions json_schema non-stream
content: {"city": "Paris", "country": "France"}
finish_reason: stop
HTTP 200
===== [TIP] bedrock-opus-4-7 chat/completions json_schema stream
streamed content: {"city": "Paris", "country": "France"}
HTTP 200
===== [TIP] bedrock-sonnet-4-6 chat/completions json_schema non-stream
content: {"city":"Paris","country":"France"}
finish_reason: stop
HTTP 200
===== [TIP] bedrock-sonnet-4-6 chat/completions json_schema stream
streamed content: {"city":"Paris","country":"France"}
HTTP 200
===== [TIP] bedrock-opus-4-8 /v1/responses text.format json_schema (reporter flow)
output_text: {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"short_answer\": \"The sky appears blue because of a phenomenon called Rayleigh scattering, where air molecules scatter shorter (blue) wavelengths of sunlight more than longer (red) wavelengths.\",\n  \"detailed_explanation\": {\n    \"sunli
HTTP 200
===== [TIP] bedrock-opus-4-8 /v1/responses text.format json_schema stream (reporter flow)
streamed output_text: {"text": "{\n  \"question\": \"Why is the sky blue?\",\n  \"answer\": \"The sky appears blue due to a phenomenon called Rayleigh scattering.\",\n  \"explanation\": {\n    \"cause\": \"Sunlight is made up of all colors of the visible spectrum. As sunlight passes through Earth's atmosphere, it collide
last event: response.completed
HTTP 200
===== [TIP] bedrock-opus-4-8 /v1/messages output_config.format json_schema
content: [{"type": "text", "text": "{\"city\": \"Paris\", \"country\": \"France\"}"}]
stop_reason: end_turn
HTTP 200
===== [TIP] bedrock-sonnet-4-6 /v1/messages output_config.format json_schema
content: [{"type": "text", "text": "{\"city\":\"Paris\",\"country\":\"France\"}"}]
stop_reason: end_turn
HTTP 200

Observations

• Opus nests extra JSON inside the schema's text string; the prompt asks for JSON twice, this PR leaves it alone
• /v1/messages on Opus 4.7 and 4.8 is prompt-hinted JSON, not tool-enforced; this PR causes it (the Medium caveat)
• Sonnet 4.6 identical on every endpoint in both legs; this PR leaves it alone
• Both legs ran 2 uvicorn workers; no worker-dependent behavior seen

Live PR risk

Ran at 6135e76 (b214a41 is test-only) against real Bedrock, merge base 1953603 vs tip, then again on the merged tree ecaff1b943 (b214a41 plus origin/main b395bfe, a clean merge): the nine requests of the proof plus the legs named below, one proxy process per side with 2 uvicorn workers and --detailed_debug, so the outbound Bedrock request body was diffed per request (SigV4 signs the host, so a forwarding recorder cannot sit in front of Bedrock). Every request reached Bedrock exactly once on every side

Breaking

None observed. Sonnet 4.6 sends the same output_config.format (Invoke) and outputConfig.textFormat (Converse) bodies at base, tip, and merged and answers 200 with schema JSON on all three; json_object requests carry no structured-output key on any side and return the same fenced JSON on all three; the direct anthropic/claude-opus-4-7 and claude-opus-4-8 keys still resolve to true at head

Backward incompatible

Every entry below is a request that got Bedrock's 400 (output_config.format: Extra inputs are not permitted) at base and now gets 200 through the non-native path every other non-native Bedrock Claude model already takes, so no client that worked before answers differently. Decision: ship them on that existing path instead of building new enforcement for these two models; approved by mateo-berri in this PR's approving review

• /v1/messages with output_config.format on Opus 4.7 and 4.8 (Invoke route): the schema is inlined as text into the last user message and nothing enforces it. Observed: 400 at base; 200 at tip and merged with prose plus a fenced JSON block; the outbound body carries the schema text in the last user message and no output_config.format
• /v1/chat/completions with an Anthropic-style output_config.format on the Converse route: the key is dropped with a warning log and nothing constrains the answer. Observed: 400 at base; 200 with plain prose (The capital of France is **Paris**.) at tip and merged; the outbound body has no outputConfig and no toolConfig
• /v1/chat/completions json_schema with reasoning_effort on the Converse route: json_tool_call is offered but not forced, since a forced tool choice is not allowed with thinking on, so the model may answer in prose. Observed: 400 at base; 200 at tip and merged over three runs each with identical outbound bodies (json_tool_call in toolConfig, no toolChoice, adaptive thinking), schema JSON twice and prose once at tip, prose three times on the merged tree
• /v1/chat/completions json_schema together with the caller's own tools on Converse: toolChoice is forced to json_tool_call, so the model cannot call the caller's tools on that request. Observed (request sent tools and response_format, no tool_choice): 400 at base; 200 at tip and merged with schema JSON, the outbound toolConfig carrying json_tool_call and get_weather with toolChoice set to json_tool_call, no tool call in the answer
• stream: true with json_schema on Converse now runs as one non-streamed Bedrock call re-chunked to the client, so the first delta arrives after the whole completion. Observed: 400 at base; 200 at tip and merged, deltas assembling to the same JSON as the non-stream call and ending in [DONE]
• GET /v1/model/info reports supports_native_structured_output: false for these deployments (true at base) with supports_response_schema still true; /model_group/info never carried the field (null on every side)

Regression risk

• Strings with no cost-map entry of their own (any other region prefix such as apac., and bedrock_mantle/ names) fall through to the bare anthropic.claude-opus-4-7 / -4-8 key, so the flag flips for them too. No mantle code path reads it (mantle rides the OpenAI-compatible transformer), so the delta there is the reported value only; not driven
• The /v1/messages Invoke branch for these models has no unit test of its own beyond the shared apply_bedrock_invoke_structured_output tests and the label-gated e2e suites (tests/e2e/claude_code/structured_outputs/test_bedrock_{converse,invoke}.py), so it is covered here by the live legs only; the Converse and Invoke chat branches have cost-map-driven unit tests in this PR

Dependency graph

Readers of supports_native_structured_output (complete grep over litellm/, enterprise/, litellm/proxy/client/, the dashboard, and the litellm-docs checkout; no string dispatch, registry, constant, or dashboard reader exists):

• litellm.utils.supports_native_structured_output and _supports_factory, key resolution for bedrock/, bedrock/invoke/, bedrock/converse/, region-prefixed and us-gov keys: verified-live through /v1/model/info, plus every prefix form resolved at head
• ModelInfo.supports_native_structured_output and get_model_info behind /v1/model/info: verified-live
• Bedrock Converse gate (converse_transformation.py), chat completions and the Responses bridge on top of it: verified-live (non-stream, stream, tools plus schema, json_object, output_config.format, reasoning plus schema), tested
• Bedrock Invoke chat gate (anthropic_claude3_transformation.py): verified-live (Opus 4.7, Opus 4.8, Sonnet 4.6 control), tested at b214a41
• Bedrock Invoke /v1/messages gate (common_utils.apply_bedrock_invoke_structured_output): verified-live, untested at unit level for these models
• Router and proxy pre-call checks: no reader (they filter on supports_response_schema and supports_function_calling, both still true), so no deployment is dropped or rejected
• /model_group/info, /v1/models, the dashboard, litellm/proxy/client/: no reader, unchanged
• Anthropic-direct and Vertex partner gates: read the flag for their own keys, which are untouched
• Cost-map reload paths (/reload/model_cost_map, periodic reload, adopt_model_cost_map): not driven; a running proxy picks the flip up at the next reload event or restart

Not verified

• eu., jp., au., us-gov. and us-gov-east-1 Opus 4.7 and 4.8 keys: not routable from the us-east-1 credentials this run has; global. was driven through the bedrock/* wildcard
• The bare anthropic.claude-opus-4-8 id and the bedrock/converse/ prefix: resolved at head, not called
• /v1/messages with the legacy top-level output_format field: same fallback by code, not driven
• Cost-map pickup on a running proxy through POST /reload/model_cost_map: not driven

Type

🐛 Bug Fix

Caveats (if any)

Low

• Requests that got Bedrock's 400 before now take the existing non-native path
  • Details and the recorded decision are under Live PR risk in Screenshots / Proof of Fix
  • /v1/messages with output_config.format: schema inlined as a prompt hint, not enforced
  • Anthropic-style output_config.format on Converse chat completions: key dropped with a warning
  • reasoning_effort plus json_schema on Converse: json_tool_call offered, not forced
  • json_schema plus the caller's tools on Converse: toolChoice forced to json_tool_call
  • json_schema with stream: true on Converse: one Bedrock call, re-chunked to the client
  • Left as is on purpose: each is how every non-native Bedrock Claude model (Opus 5.5, Sonnet 5, every unset key) already behaves, so enforcing JSON on /v1/messages or under thinking is its own feature with its own QA, and the 400 it replaces served nobody
• misc / Run tests is red on a pre-existing failure and is not a required check
  • test_wrapper_async_does_not_fire_failure_hook_for_post_success_error fails on main run 35807522965 and at the merge base
  • test(utils): raise the post-success hook error from a guardrail in the failure-hook regression #42646 carries the fix on main
• Hosted cost-map deployments pick this up at the next reload or restart, no upgrade needed
  • LITELLM_LOCAL_MODEL_COST_MAP=True deployments need the release
• A deployment model_info override pinning supports_native_structured_output: true keeps the native path and the 400 (by code, not driven)
• Strings without their own key (apac., bedrock_mantle/) fall through to the bare key, so they flip too
  • No mantle code path reads the flag; not driven
• Not driven: eu., jp., au., and us-gov keys (no credentials for those regions), POST /reload/model_cost_map pickup, legacy output_format on /v1/messages
  • If any of these is wrong, that request keeps today's 400 rather than losing a working path
• If AWS enables structured outputs on Opus 4.7 and 4.8, these 15 flags flip back to true

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
• b214a41 passes /live-pr-risk (legs at 6135e76, b214a41 is test-only; merged tree ecaff1b943 = b214a41 plus origin/main b395bfe; report under Screenshots / Proof of Fix)

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule issues

Findings

1.  Direct schemas are dropped ▶

Summary

This PR corrects Bedrock Claude Opus 4.7 and 4.8 capability metadata so structured-output requests use supported fallback paths instead of sending rejected native fields

• Marks all 15 affected model identifiers as lacking native structured-output support in both cost maps
• Verifies Converse requests use a forced JSON tool for Opus while Sonnet 4.6 retains native output configuration
• Verifies the Invoke path inlines schemas for Opus while preserving supported effort settings

_{Reviews (2) · Last reviewed commit: "test(bedrock): assert Opus 4.7 and 4.8 i..."}

[greptile-apps]

Direct schemas are dropped

On Opus 4.7/4.8, this false flag drops output_config.format schemas, allowing successful unconstrained responses. Use the same tool fallback here

Knowledge Base Used: Provider adapters and capabilities

[mateo-berri]

The Messages route already inlines the schema for every non-native model (#22797); a forced-tool fallback there is a separate change. Caveat noted

[greptile-apps]

That distinction is correct. The Messages route already preserves the schema through its existing non-native inline path, so this flag change does not introduce the schema loss I called out. A forced-tool fallback there is a separate enhancement, and the caveat accurately documents the current best-effort behavior. I’m withdrawing the prior finding.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_opus_4_8_structured_output (b214a41) with main (b395bfe)¹}

Footnotes

1. No successful run was found on main (1bc7d34) during the generation of this report, so b395bfe was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit b214a41. Configure here.}

[mateo-berri]

LGTM