Fix Bedrock Converse API returning both json_tool_call and real tools when tools and response_format are used

[haggai-backline]

Summary

Fixed issue #18381 where Bedrock Converse API returns both json_tool_call (internal tool for structured output) and real tools when tools and response_format parameters are used together.

Changes

• Modified _transform_response method in litellm/llms/bedrock/chat/converse_transformation.py:

  • Added logic to detect when both json_tool_call and real tools are present in the response
  • Filter out json_tool_call when multiple tools are returned and json_mode is enabled
  • Preserve existing behavior when json_tool_call is the only tool (convert to content)
  • Only return real tools to the user, hiding the internal json_tool_call implementation detail

• Added comprehensive tests in tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py:

  • test_transform_response_with_structured_response_calling_tool: Tests scenario where only real tool is called
  • test_transform_response_with_both_json_tool_call_and_real_tool: Tests the bug scenario where both tools are returned
  • Both tests verify that json_tool_call is properly filtered out from the response

Relevant issues

Fixes #18381

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Problem

When using both tools and response_format parameters with Bedrock Converse API, LiteLLM internally adds a fake tool called json_tool_call to handle structured output. However, Bedrock sometimes returns both:

1. The json_tool_call (internal implementation detail)
2. The real tool that the user defined

This caused the response to contain the internal json_tool_call in the tool_calls array, which should be hidden from the user.

Solution

Modified the _transform_response method to intelligently filter tool calls based on the scenario:

1. Single tool (json_tool_call only): Convert to content (existing behavior)
2. Multiple tools with json_tool_call present: Filter out json_tool_call and only return real tools
3. Multiple tools without json_tool_call: Return all tools as-is (no change)

Code Changes

File: litellm/llms/bedrock/chat/converse_transformation.py

Lines 1473-1525: Refactored tool filtering logic to handle three scenarios:

• Detect if json_tool_call is present in the response
• If it's the only tool, convert to content (existing behavior)
• If there are multiple tools, filter out json_tool_call and return only real tools
• Only set tool_calls if there are tools remaining after filtering

Testing

Added two comprehensive test cases:

1. test_transform_response_with_structured_response_calling_tool (line 623):

   • Tests scenario where both tools are provided but only the real tool is called
   • Verifies that only the real tool appears in the response

2. test_transform_response_with_both_json_tool_call_and_real_tool (line 750):

   • Tests the exact bug scenario from issue [Bug]: Bedrock returns fake json_tool_call tool when using response_format + tools together, breaking OpenAI Agents SDK #18381
   • Simulates Bedrock returning both json_tool_call and real tool
   • Verifies that json_tool_call is filtered out and only real tool is returned

All 58 tests in test_converse_transformation.py are passing.

Edge Cases Handled

• ✅ Single json_tool_call only → Convert to content
• ✅ Multiple tools with json_tool_call → Filter out json_tool_call
• ✅ Multiple tools without json_tool_call → Return all tools
• ✅ No tools → No changes
• ✅ Empty tools list → No changes

Backward Compatibility

This change is fully backward compatible:

• Existing behavior for single json_tool_call is preserved
• Only affects the new edge case where both tools are returned
• No API changes or breaking changes to existing functionality

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 2, 2026 7:10am

[gstrauss-zscaler]

Hey, when will this fix will be merged?

[jquinter]

Hey! nice PR, thanks.

⚠️ Minor Observations

1. Redundant condition at line 1544:
   if filtered_tools and len(filtered_tools) > 0:
   Could be simplified to:
   if filtered_tools:
2. Import inside method at line 1409:
   import json
   The json import is already at the top of the file - this inner import is unnecessary (though harmless).
3. Variable shadowing potential - filtered_tools = tools initially assigns the reference. If tools is mutated elsewhere, it could cause issues. Consider filtered_tools = list(tools) for safety (though the current code doesn't mutate).

[haggai-backline]

Thanks for the review! I've addressed all of your feedback:

1. Simplified redundant condition - Changed if filtered_tools and len(filtered_tools) > 0 to if filtered_tools
2. Moved import json to top of file - Removed the inner import and added it to the module-level imports.
3. Safer copy to prevent variable shadowing - Changed filtered_tools = tools to filtered_tools = list(tools)

Regarding the CI failure: the mypy error in integrations/opentelemetry.py:1013 is a pre-existing issue unrelated to this PR.

[krrishdholakia]

@greptile for review

[krrishdholakia]

@haggai-backline can you cover the streaming case as well

[greptile-apps]

Greptile Overview

Greptile Summary

• Refactors Bedrock Converse response handling to isolate JSON-mode tool filtering into _filter_json_mode_tools.
• Fixes an edge case where Bedrock returns both internal json_tool_call (structured output) and a real tool call by filtering out the internal tool.
• Preserves prior behavior when json_tool_call is the only tool by converting its arguments into message.content (including unwrapping the {"properties": ...} wrapper).
• Adds regression tests covering (a) real tool-only calls under structured output and (b) mixed json_tool_call + real tool responses.

Confidence Score: 4/5

• Mostly safe to merge, but there is a concrete behavioral side effect in response transformation to address.
• The functional change is narrow and well-covered by new tests, but _transform_response mutates the caller-supplied optional_params by popping json_mode, which can break callers that reuse that dict after response handling.
• litellm/llms/bedrock/chat/converse_transformation.py

Important Files Changed

Filename	Overview
litellm/llms/bedrock/chat/converse_transformation.py	Refactors JSON-mode tool filtering into _filter_json_mode_tools and filters out internal json_tool_call when real tools are also present; however _transform_response now mutates optional_params by popping json_mode.
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	Adds regression tests for Bedrock responses containing both json_tool_call and a real tool; tests appear to validate filtering behavior.

Sequence Diagram

sequenceDiagram
  participant U as User
  participant L as LiteLLM
  participant B as Bedrock Converse API

  U->>L: completion(tools + response_format)
  L->>L: add internal RESPONSE_FORMAT_TOOL_NAME (json_tool_call)
  L->>B: /converse request (tools includes real + json_tool_call)
  B-->>L: response with content blocks
  L->>L: _translate_message_content()
  L->>L: tools[] includes json_tool_call and/or real tools
  L->>L: _filter_json_mode_tools(json_mode, tools)
  alt only json_tool_call returned
    L->>L: parse tool.arguments JSON
    L->>L: unwrap {properties: ...} if present
    L->>L: set message.content to JSON string
    L->>L: clear tool_calls
  else json_tool_call + real tools returned
    L->>L: remove json_tool_call from tool_calls
    L->>L: keep real tool_calls
  end
  L-->>U: ChatCompletionResponse (internal tool hidden)

Loading

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

litellm/llms/bedrock/chat/converse_transformation.py
Mutates caller optional_params

json_mode is pulled via optional_params.pop("json_mode", None) in _transform_response, which mutates the dict passed in by the caller. If the caller reuses optional_params after transform_response (e.g., for logging/retries/metrics), json_mode will be missing unexpectedly. Prefer optional_params.get("json_mode") (or pop from a copied dict) to avoid side effects from response transformation.

[krrishdholakia]

@shin-bot-litellm can you take over this PR to cover streaming as well?