fix(anthropic): json response_format + user tools non-streaming

[Sameerlite]

Summary

Direct Anthropic non-streaming chat with response_format injects an internal json_tool_call tool. The previous path only unwrapped it when len(tool_calls) == 1, so responses that also included real user tools surfaced json_tool_call to the client.

Change

• Add _resolve_json_mode_non_streaming (same cases as Bedrock _filter_json_mode_tools): internal-only, mixed, or none.
• transform_parsed_response merges structured JSON into content and uses filtered tool_calls when mixed.

---

Note

Medium Risk
Touches Anthropic response parsing for json_mode and tool_calls, which can subtly change client-visible output formatting, but scope is limited and covered by a targeted regression test.

Overview
Fixes non-streaming Anthropic response_format handling when the provider injects an internal json_tool_call alongside real user tool calls.

Adds _resolve_json_mode_non_streaming to strip internal RESPONSE_FORMAT_TOOL_NAME tool calls, returning either a full replacement message (when only internal JSON tools are present) or merged JSON payload appended into content while preserving only user tool calls. transform_parsed_response now uses this to avoid leaking the internal tool call to clients and to keep structured JSON available in message.content.

Includes a regression test covering the mixed internal+user tool case.

^{Reviewed by Cursor Bugbot for commit f503c06. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR fixes a bug where non-streaming Anthropic calls using response_format alongside user-defined tools leaked the internal json_tool_call tool to the client, because the old path only unwrapped it when len(tool_calls) == 1. The new _resolve_json_mode_non_streaming method handles three cases — internal-only, mixed, and none — mirroring the existing Bedrock _filter_json_mode_tools pattern.

Confidence Score: 4/5

Safe to merge after addressing the silent content-drop edge case in the mixed-mode branch.

The core fix is correct and well-tested. One P2 finding (silent JSON drop when arguments is None in mixed mode) is a minor robustness gap with no guard or log. _transform_response_for_json_mode is now dead code but left in place. Keeping at 4 because the silent-drop condition, while unlikely, touches data integrity in the structured output path.

litellm/llms/anthropic/chat/transformation.py — mixed-mode branch in _resolve_json_mode_non_streaming

Important Files Changed

Filename	Overview
litellm/llms/anthropic/chat/transformation.py	Adds _resolve_json_mode_non_streaming to correctly strip internal json_tool_call entries in both pure and mixed-tool responses; replaces the old len==1 guard. Minor: silent content drop when arguments is None in the mixed-mode branch, and _transform_response_for_json_mode becomes dead code.
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py	Adds a focused unit test for the mixed internal+user tool case; uses only mocks, no real network calls, consistent with the test folder's rules.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[transform_parsed_response] --> B[extract_response_content\ntool_calls, text_content]
    B --> C[_resolve_json_mode_non_streaming]
    C --> D{json_mode=True\n& tool_calls?}
    D -- No --> E[return None, tool_calls, None\nno-op pass-through]
    D -- Yes --> F{any RESPONSE_FORMAT_TOOL_NAME\nin tool_calls?}
    F -- No --> E
    F -- Yes --> G{all tool_calls\nare internal json tools?}
    G -- Yes --> H[_convert_tool_response_to_message\nreturn json_message, empty, None]
    G -- No: mixed --> I[_convert_tool_response_to_message\nfor first json tool only]
    I --> J[return None, filtered_user_tools, extra_content]
    H --> K[json_mode_message != None\nset stop_reason=stop\n_message = json_mode_message]
    J --> L[merged_text = text_content + extra_content\n_message with filtered tool_calls]
    E --> M[litellm.Message with original\ntool_calls + text_content]

Loading

_{Reviews (3): Last reviewed commit: "Fix black formatting" | Re-trigger Greptile}

[gitguardian]

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
31539530	Triggered	Generic Password	9d58e6e	.github/workflows/_test-unit-services-base.yml	View secret
29203053	Triggered	Generic Password	9d58e6e	.circleci/config.yml	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[veria-ai]

Low: Response transformation refactor for Anthropic JSON mode

This PR refactors how non-streaming Anthropic responses handle the internal response_format tool call when mixed with user-defined tool calls. The changes are confined to the provider response transformation layer (litellm/llms/anthropic/chat/transformation.py) and operate on data returned from the Anthropic API. No security issues identified.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-24T03:39:40.104Z

[veria-ai]

Low: No security issues

This PR refactors Anthropic response transformation logic to handle JSON mode with mixed internal and user tool calls in non-streaming scenarios. The changes are confined to the SDK translation layer and involve only response data mapping between Anthropic and OpenAI formats. No auth, input validation, database, or file system code is affected.

---

Status: 0 open
Risk: 1/10

Posted by Veria AI · 2026-04-24T03:42:58.114Z

[codecov]

Codecov Report

❌ Patch coverage is 78.26087% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/anthropic/chat/transformation.py	78.26%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

[yuneng-berri]

@Sameerlite Can you see if the Greptile comment is applicable?

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f503c06. Configure here.}

[mateo-berri]

@Sameerlite Can you see if the Greptile comment is applicable?

The first comment is wrong -- it's still called in tests, but that test doesn't need to use it since it's just wrapper. I'll commit the cleanup

I think the second comment is a silly minor human readability thing. It's whatever

[mateo-berri]

LGTM. Thanks!