feat(bedrock): support native structured outputs API (outputConfig.textFormat)

[ndgigliotti]

Relevant issues

Fixes #21208

Pre-Submission checklist

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🆕 New Feature

Changes

For Bedrock models that support native structured outputs, use the Converse API's outputConfig.textFormat field (true constrained decoding) instead of the synthetic json_tool_call workaround. Unsupported models automatically fall back to the existing tool-call approach.

What changed

• New TypedDicts in litellm/types/llms/bedrock.py — OutputConfigBlock, OutputFormat, OutputFormatStructure, JsonSchemaDefinition matching the AWS Converse API schema
• Model detection — BEDROCK_NATIVE_STRUCTURED_OUTPUT_MODELS set with substring matching (same pattern as Anthropic's direct API)
• Schema normalization — _add_additional_properties_to_schema() recursively ensures additionalProperties: false on all object types, which Bedrock requires
• Branching in _translate_response_format_param — supported models get outputConfig (no synthetic tool, no fake_stream); unsupported models use the existing tool-call path unchanged
• Request/response plumbing — outputConfig flows through _prepare_request_params → _transform_request_helper → final API request; response handling passes JSON text content through directly when no synthetic tool call is present
• 12 new unit tests covering model detection, schema normalization, outputConfig creation, request building, response handling, and streaming behavior

Supported models (native path)

Anthropic Claude 4.5+, Qwen3, DeepSeek V3.1, Gemma 3, MiniMax M2, Mistral Large 3, Ministral, Voxtral, Moonshot Kimi K2, NVIDIA Nemotron Nano.

GPT-OSS and Magistral Small are excluded despite AWS listing them — their constrained decoding is broken on Bedrock (produces invalid JSON). They fall back to tool-call which works reliably for GPT-OSS.

Why

• No tool conflicts — structured output is decoupled from tool calling, so response_format and tools work together cleanly (helps with [Bug]: Bedrock returns fake json_tool_call tool when using response_format + tools together, breaking OpenAI Agents SDK #18381)
• True schema guarantees — constrained decoding at inference level vs relying on model to call a tool correctly
• Broader schema support — handles schema shapes that are tricky to wrap in tool definitions (helps with [Bug]: AWS Bedrock fails with top-level union in custom response format #1348 #17574)
• Real streaming — no fake_stream needed for native path

AWS docs: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 17, 2026 1:29am

[CLAassistant]

All committers have signed the CLA.

[ndgigliotti]

@greptileai

[greptile-apps]

Greptile Overview

Greptile Summary

Adds native structured outputs support for Bedrock models via the Converse API's outputConfig.textFormat field, replacing the synthetic tool-call workaround for supported models. Unsupported models automatically fall back to the existing tool-call approach.

• Native path for supported models: Anthropic Claude 4.5+, Qwen3, DeepSeek V3.1, Gemma 3, MiniMax M2, Mistral Large 3, Ministral, Voxtral, Moonshot Kimi K2, and NVIDIA Nemotron Nano use outputConfig.textFormat with json_schema type for true constrained decoding. This eliminates fake_stream, enables real streaming, and decouples structured output from tool calling.
• Fallback path preserved: Models without native support (including GPT-OSS and Magistral Small, which have broken constrained decoding on Bedrock) continue using the existing synthetic tool-call approach unchanged.
• Schema normalization: Recursively injects additionalProperties: false on all object types, as required by Bedrock's API. Handles properties, items, $defs, and anyOf/allOf/oneOf keywords.
• Request/response plumbing: outputConfig flows cleanly through _prepare_request_params → _transform_request_helper → final API payload. Response handling correctly passes JSON text content through when no synthetic tool call is present (via the elif tools: guard on line 1829).
• TypedDict types: New OutputConfigBlock, OutputFormat, OutputFormatStructure, and JsonSchemaDefinition types match the AWS Converse API schema.
• Test coverage: 12 new unit tests covering model detection, schema normalization, outputConfig creation, request building, response handling, and streaming behavior. All tests are mock-only with no network calls.

Confidence Score: 4/5

• This PR is safe to merge with minor considerations around edge cases in schema normalization.
• The implementation is well-structured, follows existing patterns (consistent with Anthropic's native structured output approach), and includes comprehensive test coverage. The native/fallback branching is clean and the response handling correctly handles both paths. Two minor style issues were identified: the definitions keyword gap in schema normalization and the empty schema case for json_object type. Neither is likely to cause issues in typical usage but could affect edge cases. All provider-specific code stays within the llms/ directory, and tests make no real network calls.
• litellm/llms/bedrock/chat/converse_transformation.py — schema normalization and empty schema edge cases

Important Files Changed

Filename	Overview
litellm/llms/bedrock/chat/converse_transformation.py	Core implementation of native structured outputs via outputConfig.textFormat. Clean branching between native and tool-call fallback paths. Minor gaps: schema normalization misses definitions keyword; empty schema for json_object type may cause issues.
litellm/types/llms/bedrock.py	Adds well-structured TypedDict definitions (OutputConfigBlock, OutputFormat, OutputFormatStructure, JsonSchemaDefinition) matching the AWS Converse API schema. Clean addition to CommonRequestObject.
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	Comprehensive test suite (12 tests) covering model detection, schema normalization, outputConfig creation, request building, response handling, and streaming behavior. All tests are pure unit tests with no network calls.

Flowchart

flowchart TD
    A["response_format param received"] --> B{"Model supports native\nstructured outputs?"}
    B -->|Yes| C["_create_output_config_for_response_format()"]
    C --> D["Normalize schema:\nadditionalProperties: false"]
    D --> E["Set outputConfig.textFormat\nwith json_schema type"]
    E --> F["json_mode = True\nNo fake_stream needed"]
    B -->|No| G["_create_json_tool_call_for_response_format()"]
    G --> H["Inject synthetic tool +\ntool_choice forced"]
    H --> I["json_mode = True\nfake_stream = True if streaming"]
    F --> J["_prepare_request_params()"]
    I --> J
    J --> K["_transform_request_helper()"]
    K -->|Native path| L["outputConfig added as\ntop-level request field"]
    K -->|Fallback path| M["toolConfig contains\nsynthetic tool"]
    L --> N["Bedrock Converse API"]
    M --> N
    N -->|Response| O{"Tool call with\nRESPONSE_FORMAT name?"}
    O -->|Yes: fallback path| P["Extract JSON from\ntool arguments"]
    O -->|No tools / real tools| Q["Pass text content\nthrough directly"]
    P --> R["Return ModelResponse"]
    Q --> R

Loading

_{Last reviewed commit: 2129061}

[krrishdholakia]

@ndgigliotti unable to merge due to conflicts, can you resolve and bump me please?

@jquinter for monitoring

[ndgigliotti]

@krrishdholakia Conflicts resolved and pushed.

The 6 failing checks (test (llms), test (other), test (proxy-guardrails), test (proxy-unit-a), test (proxy-unit-b), test (root)) appear to be repo-wide — they seem to be failing on other open PRs as well (#21330, #21327, #21326, #21325, #21322). Not related to this PR's changes.

All checks relevant to this PR are passing: unit-test, test, lint, test (core-utils), test (integrations), test (proxy-core), test (proxy-misc), build-ui, validate-model-prices-json.

[ndgigliotti]

Resolved merge conflicts with the staging branch.