fix(redaction): redact assistant tool call arguments in spend logs by rs-03 · Pull Request #33109 · BerriAI/litellm

rs-03 · 2026-07-13T19:26:40Z

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

This is a redaction bug, so the observable difference is what ends up in the spend log for an assistant response that contains tool calls, with message redaction enabled (turn_off_message_logging: true, or store_prompts_in_spend_logs: false, or the per-request header)

Before this change, the message content is redacted but the tool call is left intact, so the arguments the model generated (which routinely echo user data) are written to the log in the clear:

"choices": [{"message": {
  "content": "redacted-by-litellm",
  "tool_calls": [{"id": "call_1", "type": "function",
    "function": {"name": "lookup_customer", "arguments": "{\"email\": \"jane.doe@example.com\"}"}}]
}}]

After this change the arguments are redacted while the tool name and call structure stay intact, so token counts and metrics are unaffected:

"choices": [{"message": {
  "content": "redacted-by-litellm",
  "tool_calls": [{"id": "call_1", "type": "function",
    "function": {"name": "lookup_customer", "arguments": "redacted-by-litellm"}}]
}}]

I verified this at the redaction layer rather than against a live proxy, since reproducing the spend-log entry end to end needs the proxy running with a real tool-calling model. The behavior is pinned by the added regression tests, which fail on the current code (arguments are not redacted) and pass with the fix, across the object response path, the dict response path, and the streaming delta path

Type

🐛 Bug Fix

Changes

_redact_choice_content and _redact_model_response_dict_choices in litellm/litellm_core_utils/redact_messages.py redacted content, reasoning_content, thinking_blocks and audio, but never tool_calls. Every redaction path funnels through these two helpers (the direct result, the streaming complete_streaming_response, and the standard_logging_object response), so assistant tool call arguments survived redaction everywhere

This adds tool_calls argument redaction to both helpers, for the message choice and the streaming delta, in both the object form (litellm.Choices / StreamingChoices) and the dict form. Only function.arguments is blanked; the tool name, id and overall structure are preserved, consistent with how content is replaced with a placeholder rather than dropped

QA runbook

The regression tests are TestPerformRedaction::test_redacts_tool_call_arguments (object ModelResponse path) and TestPerformRedaction::test_redacts_tool_call_arguments_in_dict_choices (dict message and delta paths) in tests/test_litellm/litellm_core_utils/test_redact_messages.py

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

When message redaction is enabled (turn_off_message_logging / store_prompts_in_spend_logs=false), message content, reasoning, thinking and audio were redacted but assistant tool_calls were left untouched, so the function arguments, which can echo sensitive conversation data, were still written to spend logs in the clear. Redact the arguments of every assistant tool call in both the object and dict redaction paths, for message and streaming delta choices, keeping the tool name and call structure so token counts and metrics stay intact. Fixes BerriAI#33107

greptile-apps · 2026-07-13T19:28:56Z

Greptile Summary

This PR fixes a redaction gap where assistant tool-call function.arguments survived message redaction even when turn_off_message_logging was enabled, leaving model-generated arguments (which often echo user data) in spend logs in the clear.

Adds _redact_tool_call_arguments (object form) and _redact_tool_call_arguments_dict (dict form) helpers, wired into _redact_choice_content for both litellm.Choices and litellm.utils.StreamingChoices, and into _redact_model_response_dict_choices for the dict "message" and dict "delta" branches — covering all paths through perform_redaction.
New tests confirm the fix for ModelResponse object choices, dict message choices, and dict streaming delta choices; the StreamingChoices object path (used by complete_streaming_response) is covered by production code but has no dedicated test.

Confidence Score: 4/5

The change is safe to merge — it adds argument redaction to two helper functions and two new unit tests, with no modifications to existing logic or test assertions.

The fix is correctly applied to all four choice paths and is consistent with how the rest of _redact_choice_content and _redact_model_response_dict_choices handle other sensitive fields. The only gap is the absence of a test that drives the StreamingChoices object path through complete_streaming_response, leaving that branch verified by code inspection alone rather than an automated regression guard.

The streaming path in redact_messages.py (lines 63–69, the StreamingChoices branch of _redact_choice_content) deserves a dedicated test in the test file.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/redact_messages.py	Adds `_redact_tool_call_arguments` and `_redact_tool_call_arguments_dict` helpers that blank `function.arguments` in all four choice paths (object Choices, object StreamingChoices, dict message, dict delta); logic is correct and consistent with existing redaction style.
tests/test_litellm/litellm_core_utils/test_redact_messages.py	Two new tests cover the ModelResponse object path and the dict message/delta paths; the StreamingChoices object path (complete_streaming_response with StreamingChoices containing tool_calls) is not exercised by the new tests.

_{Reviews (1): Last reviewed commit: "fix(redaction): redact assistant tool ca..." | Re-trigger Greptile}

greptile-apps · 2026-07-13T19:29:00Z

+    def test_redacts_tool_call_arguments(self):
+        tool_call = {
+            "id": "call_1",
+            "type": "function",
+            "function": {"name": "get_weather", "arguments": '{"city": "sensitive city"}'},
+        }
+        result = litellm.ModelResponse(
+            choices=[
+                litellm.Choices(
+                    message=litellm.Message(
+                        content="message content",
+                        role="assistant",
+                        tool_calls=[dict(tool_call)],
+                    )
+                )
+            ]
+        )
+
+        redacted = perform_redaction({}, result)
+
+        redacted_call = redacted.choices[0].message.tool_calls[0]
+        assert redacted_call.function.arguments == "redacted-by-litellm"
+        assert redacted_call.function.name == "get_weather"
+
+    def test_redacts_tool_call_arguments_in_dict_choices(self):
+        tool_call = {
+            "id": "call_1",
+            "type": "function",
+            "function": {"name": "get_weather", "arguments": '{"city": "sensitive city"}'},
+        }
+        result = {
+            "choices": [
+                {"message": {"content": "message content", "tool_calls": [dict(tool_call)]}},
+                {"delta": {"content": "delta content", "tool_calls": [dict(tool_call)]}},
+            ]
+        }
+
+        redacted = perform_redaction({}, result)
+
+        message_call = redacted["choices"][0]["message"]["tool_calls"][0]
+        assert message_call["function"]["arguments"] == "redacted-by-litellm"
+        assert message_call["function"]["name"] == "get_weather"
+
+        delta_call = redacted["choices"][1]["delta"]["tool_calls"][0]
+        assert delta_call["function"]["arguments"] == "redacted-by-litellm"


Missing test for StreamingChoices object-form tool calls

The PR exercises three of four redaction paths for tool calls: litellm.ModelResponse (object), dict "message", and dict "delta". The fourth path — complete_streaming_response with litellm.utils.StreamingChoices objects containing tool_calls — is covered by the new _redact_tool_call_arguments call added to _redact_choice_content's StreamingChoices branch, but no test drives it. A test passing model_call_details={"stream": True, "complete_streaming_response": ...} with StreamingChoices that have delta.tool_calls set would close this gap and guard against future regressions on the streaming path.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

codecov · 2026-07-13T19:32:49Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Add a regression test that drives perform_redaction over a complete_streaming_response whose StreamingChoices carry delta tool_calls, closing the one tool-call redaction path the earlier tests did not exercise directly.

rs-03 · 2026-07-13T19:34:57Z

Good catch, added est_redacts_tool_call_arguments_in_streaming_choices in 9ece9c9. It drives perform_redaction over a complete_streaming_response whose StreamingChoices carry delta.tool_calls, so the streaming object path is now covered directly. It fails on the pre-fix code and passes with the fix, same as the other three paths

Assert redaction skips non-dict and function-less tool_call entries without error while still redacting valid ones, guarding the tool_calls redaction against malformed provider responses.

veria-ai · 2026-07-13T19:42:51Z

                    choice["message"]["thinking_blocks"] = None
                if "audio" in choice["message"]:
                    choice["message"]["audio"] = None
+                _redact_tool_call_arguments_dict(choice["message"].get("tool_calls"), redacted_str)


Medium: Alternate function-call arguments bypass redaction

This only handles message.tool_calls[].function.arguments. An untrusted prompt can place sensitive data in legacy message.function_call.arguments or a Responses API output item’s top-level arguments; both survive perform_redaction() and are serialized into spend logs. Redact those representations in both the object and dictionary helpers as well.

veria-ai · 2026-07-13T19:42:56Z

PR overview

This pull request updates spend-log redaction logic so assistant tool call arguments are removed before messages are logged. The changes focus on the message redaction helpers in redact_messages.py.

There is still an open gap in the redaction coverage: alternate function-call argument formats can bypass the current handling and be written to spend logs. That means sensitive data supplied through legacy or Responses API-style fields may remain exposed in logging paths until those representations are redacted as well. No issues have been addressed yet, so the PR still needs a targeted fix before the redaction behavior is complete.

Open issues (1)

Medium: Alternate function-call arguments bypass redaction — litellm/litellm_core_utils/redact_messages.py:163

Fixed/addressed: 0 · PR risk: 6/10

The tool_calls fix left the legacy function_call field untouched, so a response using the older single function_call format still wrote its arguments to spend logs in the clear under message redaction. Redact function_call.arguments alongside tool_calls in every choice path, object and dict, message and streaming delta.

rs-03 · 2026-07-13T19:50:21Z

Good catch, addressed in 08cd10a. Extended the fix to redact the legacy message.function_call.arguments (and delta.function_call.arguments) alongside ool_calls, in both the object and dict paths, since a response using the older single function_call format had the same leak. Added est_redacts_legacy_function_call_arguments_object and est_redacts_legacy_function_call_arguments_dict, both mutation-checked against the pre-fix code

rs-03 · 2026-07-20T18:51:56Z

Friendly ping, this has been open about a week and is mergeable. Same unrelated systemic osv-scan / auth-and-jwt reds noted in the thread; the substantive checks are green. It is a scoped redaction fix so assistant tool-call arguments do not leak into spend logs when turn_off_message_logging is on, with mutation-verified regression tests. Glad to clarify or adjust anything. Thanks!

rs-03 · 2026-07-21T19:56:51Z

Closing this as redundant. #33111 landed the same assistant tool-call argument redaction and is already in the daily branches, including the Responses API function_call path, so this PR no longer adds anything on top. Thanks to those who reviewed. A separate follow-up to also redact the tool name (the selection), requested on #33107, would build on the merged code rather than this branch

rs-03 mentioned this pull request Jul 13, 2026

[Bug]: Assistant tool calls not redacted in spend logs #33107

Closed

1 task

greptile-apps Bot reviewed Jul 13, 2026

View reviewed changes

test: cover streaming StreamingChoices tool call redaction path

9ece9c9

Add a regression test that drives perform_redaction over a complete_streaming_response whose StreamingChoices carry delta tool_calls, closing the one tool-call redaction path the earlier tests did not exercise directly.

test: cover malformed tool_call entries in redaction

7c50ab7

Assert redaction skips non-dict and function-less tool_call entries without error while still redacting valid ones, guarding the tool_calls redaction against malformed provider responses.

veria-ai Bot reviewed Jul 13, 2026

View reviewed changes

rs-03 closed this Jul 21, 2026

Navigation Menu

Uh oh!

Uh oh!

fix(redaction): redact assistant tool call arguments in spend logs - #33109

fix(redaction): redact assistant tool call arguments in spend logs#33109
rs-03 wants to merge 4 commits into
BerriAI:litellm_oss_daily_2026_07_10from
rs-03:fix-redact-tool-call-args-in-spend-logs

rs-03 commented Jul 13, 2026

Uh oh!

greptile-apps Bot commented Jul 13, 2026

Important Files Changed

Uh oh!

greptile-apps Bot Jul 13, 2026

Uh oh!

codecov Bot commented Jul 13, 2026 •

edited

Loading

Uh oh!

rs-03 commented Jul 13, 2026

Uh oh!

veria-ai Bot Jul 13, 2026

Uh oh!

veria-ai Bot commented Jul 13, 2026

Uh oh!

rs-03 commented Jul 13, 2026

Uh oh!

rs-03 commented Jul 20, 2026

Uh oh!

rs-03 commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

rs-03 commented Jul 13, 2026

Relevant issues

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Type

Changes

QA runbook

Final Attestation

Uh oh!

greptile-apps Bot commented Jul 13, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

greptile-apps Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

rs-03 commented Jul 13, 2026

Uh oh!

veria-ai Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

veria-ai Bot commented Jul 13, 2026

PR overview

Open issues (1)

Uh oh!

rs-03 commented Jul 13, 2026

Uh oh!

rs-03 commented Jul 20, 2026

Uh oh!

rs-03 commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

codecov Bot commented Jul 13, 2026 •

edited

Loading