Skip to content

Fix: malformed tool call transformation in bedrock #19198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sameerlite merged 2 commits into from
Jan 16, 2026
Merged

Fix: malformed tool call transformation in bedrock #19198

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bf99cea
Fix malformed tool call tranform
Sameerlite Jan 16, 2026
e315f6b
Merge pull request #19214 from BerriAI/main
Sameerlite Jan 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions litellm/litellm_core_utils/prompt_templates/factory.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3233,17 +3233,21 @@
id = tool["id"]
name = tool["function"].get("name", "")
arguments = tool["function"].get("arguments", "")
arguments_dict = json.loads(arguments) if arguments else {}
# Ensure arguments_dict is always a dict (Bedrock requires toolUse.input to be an object)
# When some providers return arguments: '""' (JSON-encoded empty string), json.loads returns ""
if not isinstance(arguments_dict, dict):
arguments_dict = {}
if not arguments or not arguments.strip():
arguments_dict = {}
arguments_input = {}
else:
arguments_dict = json.loads(arguments)
# Try to parse the arguments JSON
try:
arguments_input = json.loads(arguments)
except json.JSONDecodeError as e:
verbose_logger.warning(
f"Malformed JSON in tool call arguments for tool '{name}': {str(e)}. "
f"Storing as raw string to allow conversation to continue."
Comment on lines +3244 to +3245

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (password)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expressi

Copilot Autofix

AI 8 months ago

At a high level, the fix is to ensure that any logging or error/exception messages do not contain secret material: API keys, passwords, cloud credentials, or raw tool arguments. Instead, logs should contain only high‑level context (e.g., which tool failed, the model name, or that a key was missing) and, if necessary, redacted/masked versions of sensitive values. Also, helper structures like litellm_params that purposely store secrets for runtime behavior should not be blindly stringified or logged.

For the concrete sink in litellm/litellm_core_utils/prompt_templates/factory.py, we can safely change the log message so that it no longer interpolates potentially tainted data. The current warning logs the tool function name and exception detail:

verbose_logger.warning(
    f"Malformed JSON in tool call arguments for tool '{name}': {str(e)}. "
    f"Storing as raw string to allow conversation to continue."
)

To avoid leaking any part of the untrusted path (and still be useful for debugging), we can:

  • Log a generic message that does not include the tool name or parsed content.
  • Optionally include the exception type or a generic error tag, which is not secret.

This change preserves functionality (we still detect parsing errors, fall back to raw string arguments, and log that behavior) while eliminating the possibility of logging secret values coming from tool names/arguments.

No other files need explicit edits for this particular fix because the only explicit sink shown is in factory.py, and we are not allowed to alter external logging behavior in the snippets we haven’t seen. The various get_* and _get_openai_compatible_provider_info functions only construct values and don’t themselves log secrets in the displayed code.

Concretely:

  • In litellm/litellm_core_utils/prompt_templates/factory.py, in _convert_to_bedrock_tool_call_invoke, replace the verbose_logger.warning string with a generic message that omits {name} and str(e) details that come from tainted data.
  • No imports, method signatures, or call sites need to change.
Suggested changeset 1
litellm/litellm_core_utils/prompt_templates/factory.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/litellm_core_utils/prompt_templates/factory.py b/litellm/litellm_core_utils/prompt_templates/factory.py
--- a/litellm/litellm_core_utils/prompt_templates/factory.py
+++ b/litellm/litellm_core_utils/prompt_templates/factory.py
@@ -3239,10 +3239,12 @@
                     # Try to parse the arguments JSON
                     try:
                         arguments_input = json.loads(arguments)
-                    except json.JSONDecodeError as e:
+                    except json.JSONDecodeError:
+                        # Log a generic message without including tool arguments or names,
+                        # to avoid leaking potentially sensitive data.
                         verbose_logger.warning(
-                            f"Malformed JSON in tool call arguments for tool '{name}': {str(e)}. "
-                            f"Storing as raw string to allow conversation to continue."
+                            "Malformed JSON in tool call arguments. "
+                            "Storing original arguments string to allow conversation to continue."
                         )
                         arguments_input = arguments
                 
EOF
@@ -3239,10 +3239,12 @@
# Try to parse the arguments JSON
try:
arguments_input = json.loads(arguments)
except json.JSONDecodeError as e:
except json.JSONDecodeError:
# Log a generic message without including tool arguments or names,
# to avoid leaking potentially sensitive data.
verbose_logger.warning(
f"Malformed JSON in tool call arguments for tool '{name}': {str(e)}. "
f"Storing as raw string to allow conversation to continue."
"Malformed JSON in tool call arguments. "
"Storing original arguments string to allow conversation to continue."
)
arguments_input = arguments

Copilot is powered by AI and may make mistakes. Always verify output.
)
arguments_input = arguments

bedrock_tool = BedrockToolUseBlock(
input=arguments_dict, name=name, toolUseId=id
input=arguments_input, name=name, toolUseId=id
)
bedrock_content_block = BedrockContentBlock(toolUse=bedrock_tool)
_parts_list.append(bedrock_content_block)
Expand Down
9 changes: 8 additions & 1 deletion litellm/llms/bedrock/chat/converse_transformation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1395,9 +1395,16 @@ def _translate_message_content(self, content_blocks: List[ContentBlock]) -> Tupl
response_tool_name = get_bedrock_tool_name(
response_tool_name=_response_tool_name
)
tool_input = content["toolUse"]["input"]
if isinstance(tool_input, str):
arguments_str = tool_input
else:
# Otherwise, serialize it to JSON
arguments_str = json.dumps(tool_input)

_function_chunk = ChatCompletionToolCallFunctionChunk(
name=response_tool_name,
arguments=json.dumps(content["toolUse"]["input"]),
arguments=arguments_str,
)

_tool_response_chunk = ChatCompletionToolCallChunk(
Expand Down
2 changes: 1 addition & 1 deletion litellm/types/llms/bedrock.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ class ToolResultBlock(TypedDict, total=False):


class ToolUseBlock(TypedDict):
input: dict
input: Any # Per boto3 spec: document type can be dict, list, int, float, str, bool, or None
name: str
toolUseId: str

Expand Down
154 changes: 154 additions & 0 deletions tests/llm_translation/test_bedrock_completion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3954,3 +3954,157 @@ def test_bedrock_openai_error_handling():

assert exc_info.value.status_code == 422
print("✓ Error handling works correctly")


def test_bedrock_malformed_tool_json_handling():
"""
Test that Bedrock handles malformed JSON in tool call arguments gracefully.

This test covers the issue where:
1. LLM generates malformed JSON in tool call arguments
2. Subsequent requests with conversation history should not crash
3. The toolUse.input field should handle any JSON value type per boto3 spec

Related issue: https://github.com/BerriAI/litellm/issues/[issue_number]
"""
from litellm.litellm_core_utils.prompt_templates.factory import (
_convert_to_bedrock_tool_call_invoke,
)
from litellm.llms.bedrock.chat.converse_transformation import AmazonConverseConfig
from litellm.types.llms.bedrock import ContentBlock

# Test 1: Malformed JSON in tool call arguments
malformed_tool_calls = [
{
"id": "call_123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"location": "Paris", "invalid_json', # Malformed JSON
},
}
]

# Should not raise an exception, but store as raw string
result = _convert_to_bedrock_tool_call_invoke(malformed_tool_calls)
assert len(result) == 1
assert result[0]["toolUse"]["name"] == "get_weather"
# The malformed JSON should be stored as a string
assert isinstance(result[0]["toolUse"]["input"], str)
assert result[0]["toolUse"]["input"] == '{"location": "Paris", "invalid_json'
print("✓ Malformed JSON stored as raw string")

# Test 2: Valid JSON should still work normally
valid_tool_calls = [
{
"id": "call_456",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"location": "London"}',
},
}
]

result = _convert_to_bedrock_tool_call_invoke(valid_tool_calls)
assert len(result) == 1
assert result[0]["toolUse"]["name"] == "get_weather"
assert isinstance(result[0]["toolUse"]["input"], dict)
assert result[0]["toolUse"]["input"] == {"location": "London"}
print("✓ Valid JSON parsed correctly")

# Test 3: Empty arguments should create empty dict
empty_tool_calls = [
{
"id": "call_789",
"type": "function",
"function": {
"name": "no_args_function",
"arguments": "",
},
}
]

result = _convert_to_bedrock_tool_call_invoke(empty_tool_calls)
assert len(result) == 1
assert result[0]["toolUse"]["input"] == {}
print("✓ Empty arguments handled correctly")

# Test 4: Bedrock to OpenAI conversion handles string input
converse_config = AmazonConverseConfig()
content_blocks = [
ContentBlock(
toolUse={
"name": "get_weather",
"toolUseId": "call_123",
"input": '{"location": "Paris", "invalid_json', # String input (malformed)
}
)
]

content_str, tools, reasoning = converse_config._translate_message_content(
content_blocks
)
assert len(tools) == 1
assert tools[0]["function"]["name"] == "get_weather"
# Should return the string as-is
assert tools[0]["function"]["arguments"] == '{"location": "Paris", "invalid_json'
print("✓ Bedrock to OpenAI conversion handles string input")

# Test 5: Bedrock to OpenAI conversion handles dict input
content_blocks_dict = [
ContentBlock(
toolUse={
"name": "get_weather",
"toolUseId": "call_456",
"input": {"location": "London"}, # Dict input (normal case)
}
)
]

content_str, tools, reasoning = converse_config._translate_message_content(
content_blocks_dict
)
assert len(tools) == 1
assert tools[0]["function"]["name"] == "get_weather"
# Should serialize dict to JSON string
assert tools[0]["function"]["arguments"] == '{"location": "London"}'
print("✓ Bedrock to OpenAI conversion handles dict input")

# Test 6: Round-trip conversion with malformed JSON
# Test that we can convert OpenAI -> Bedrock -> OpenAI with malformed JSON
malformed_tool_calls_roundtrip = [
{
"id": "call_999",
"type": "function",
"function": {
"name": "test_function",
"arguments": '{"key": "value", "broken', # Malformed
},
}
]

# Step 1: OpenAI to Bedrock (should store as string)
bedrock_blocks = _convert_to_bedrock_tool_call_invoke(malformed_tool_calls_roundtrip)
assert isinstance(bedrock_blocks[0]["toolUse"]["input"], str)

# Step 2: Bedrock back to OpenAI (should preserve the string)
content_blocks_roundtrip = [
ContentBlock(
toolUse={
"name": bedrock_blocks[0]["toolUse"]["name"],
"toolUseId": bedrock_blocks[0]["toolUse"]["toolUseId"],
"input": bedrock_blocks[0]["toolUse"]["input"],
}
)
]

content_str, tools_roundtrip, reasoning = converse_config._translate_message_content(
content_blocks_roundtrip
)

# Should preserve the malformed JSON string through the round trip
assert tools_roundtrip[0]["function"]["arguments"] == '{"key": "value", "broken'
print("✓ Round-trip conversion preserves malformed JSON")

print("✓ All malformed JSON handling tests passed")
Loading