fix(panw_prisma_airs): scan tool call args as plain text, not a tool_event

[devin-ai-integration]

TLDR

Problem this solves:

• Tool calling with Prisma AIRS enabled returns HTTP 500
• AIRS rejects our tool events as an unsupported ecosystem
• fallback_on_error: allow cannot rescue a malformed request

How it solves it:

• Send the tool name and its arguments as ordinary prompt/response text
• Keep MCP gateway scans on the real tool_event schema
• Response-side scans now read response_masked_data correctly

Second problem, found after #37036 landed on staging:

• fix(guardrails): return the full PANW AIRS scan response on blocked requests #37036 withholds prompt_masked_data on response-side tool blocks
• That was correct while tool calls went out as a request-side tool_event
• Once this PR routes them by scan side, the same withholding drops a caller-facing audit field
• The two PRs merge with zero conflicts, so nothing flags it

Third problem, reported by Cursor Bugbot on fd9f6396e5:

• A non-string function.name failed slice validation, so the tool call was skipped entirely
• Its arguments never reached AIRS: no error, no log, no block
• Reachable by any caller with a valid key, since the OpenAI path forwards tool_calls verbatim

User Flow

Before: a developer whose app lets the model call tools cannot get a single answer once the guardrail is on, every request dies with a 500

1. They add the Prisma AIRS guardrail to their proxy with mode: [pre_call, post_call] and fallback_on_error: allow
2. They send POST https://litellm-domain/v1/chat/completions with a tools array holding get_weather and "tool_choice": "auto"
3. They get back HTTP 500 with "message": "Security scan failed - request blocked for safety", "code": "panw_prisma_airs_scan_failed" and "category": "http_400_error"
4. Plain prompts without tools still return 200, so tool calling is the only broken flow, and turning the guardrail off is the only workaround

After: the same request completes, and both the tool name and the arguments the model produced are still scanned

1. They add the Prisma AIRS guardrail to their proxy with mode: [pre_call, post_call] and fallback_on_error: allow
2. They send POST https://litellm-domain/v1/chat/completions with a tools array holding get_weather and "tool_choice": "auto"
3. They get back HTTP 200 with "finish_reason": "tool_calls" and the get_weather call carrying {"city": "San Francisco"}
4. A profile keyed on tool name still fires, and arguments carrying sensitive data are still masked or blocked, exactly like message text is

Behavior changes

• Tool calls are scanned as ordinary prompt/response text instead of a tool_event. MCP gateway invocations keep tool_event.
• The scan side now decides which masked-data key holds what. A response-side tool scan reports the model's generated arguments under response_masked_data; prompt_masked_data is the caller's own input on both sides.
• _build_error_detail's also_hide parameter, added by fix: send whisper timestamp_granularities as bracketed array field #36036/fix(guardrails): return the full PANW AIRS scan response on blocked requests #37036 for the old tool_event routing, is removed. It has no callers after this change. Model-generated content stays withheld through _CLIENT_HIDDEN_SCAN_FIELDS, which already covers response_masked_data.
• Net effect on a response-side tool block: model output is still withheld, and prompt_masked_data reaches the caller again, as LIT-5638 intends.
• _ToolCallFunctionSlice coerces name as well as arguments to scannable text. Previously only arguments had a coercing validator, so a wrong-typed name failed validation for the whole slice and the tool call was skipped unscanned.
• No change to status codes, block messages, block codes, the allow path, or MCP scanning.

Relevant issues

Interacts with #37036 (merged). This branch carries a merge of current litellm_internal_staging so that interaction is resolved here rather than left to whoever rebases.

Linear ticket

Resolves LIT-5279

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Part 1 — the 500 (original fix, at a477c3e841)

Real anthropic/claude-sonnet-4-6 upstream on a live proxy. AIRS itself stands in for the vendor service because we have no AIRS credentials, and it only enforces the one documented rule that causes the bug: tool_event.metadata.ecosystem must be mcp

Guardrail config used for both runs:

guardrails:
  - guardrail_name: Prisma_airs
    litellm_params:
      guardrail: panw_prisma_airs
      mode: [pre_call, post_call]
      api_key: <token>
      api_base: http://127.0.0.1:8899
      profile_name: <profile>
      default_on: true
      fallback_on_error: allow

Same command for both runs:

curl -s -o /dev/null -w "HTTP %{http_code}\n" http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"claude","messages":[{"role":"user","content":"What is the weather in San Francisco? Use the get_weather tool."}],"tools":[{"type":"function","function":{"name":"get_weather","description":"Get current weather for a city","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],"tool_choice":"auto"}'

Before, at 7a5b98e:

HTTP 500
{"error":{"message":"Security scan failed - request blocked for safety","code":"500",
"provider_specific_fields":{"error":{"message":"Security scan failed - request blocked for safety",
"type":"guardrail_scan_error","code":"panw_prisma_airs_scan_failed","guardrail":"Prisma_airs",
"category":"http_400_error"},"guardrail_name":"Prisma_airs","guardrail_mode":["pre_call","post_call"]}}}

and the AIRS side logged the rejection:

rejecting tool_event ecosystem='openai' with HTTP 400

After, at a477c3e:

HTTP 200
[{"index":0,"caller":{"type":"direct"},"function":{"arguments":"{\"city\": \"San Francisco\"}",
"name":"get_weather"},"id":"toolu_014VW9xR4oFLHGBozoHKn3zB","type":"function"}]

and the two scans AIRS received are plain text, the prompt on the way in and the tool call on the way out, name first then arguments:

content_keys=['prompt']   contents=[{"prompt": "What is the weather in San Francisco? Use the get_weather tool."}]
content_keys=['response'] contents=[{"response": "get_weather\n{\"city\": \"San Francisco\"}"}]

Part 2 — the #37036 interaction

Live A/B on a real proxy: separate Postgres per leg, separate ports, real Gemini upstream, POST /v1/messages with a tool the model actually calls. The AIRS stand-in returns both masked keys with deliberately distinguishable values so the client body shows which one was withheld:

prompt_masked_data   = {"data": "CALLER-INPUT my ssn is XXX-XX-XXXX"}
response_masked_data = {"data": "MODEL-TOOLARGS {\"to_account\": \"XXXXXXXXXX\", \"amount\": 5000}"}

First, the merge itself is silent:

$ git merge-tree $(git merge-base origin/litellm_internal_staging f6aff5f089) \
      origin/litellm_internal_staging f6aff5f089 | grep -c '<<<<<<<'
0

The scan that reaches AIRS confirms the routing change — response-side, plain text, no tool_event:

is_response=True  tool_event=False  action=block

Response-side tool-call block, error.provider_specific_fields.error:

tree	prompt_masked_data (caller input)	response_masked_data (model output)
staging + this PR, as auto-merged	absent — audit field dropped	absent
same tree, also_hide removed (this commit)	present, CALLER-INPUT	absent

Model output stays withheld either way; the only difference is whether the caller gets their own masked input back. MODEL-TOOLARGS never appears in any body on either leg.

Part 3 — the wrong-typed tool name

_get_tool_call_function turns any ValidationError into (None, None), and _scan_tool_calls_for_guardrail reads that as an unscannable tool call and skips it. Since name was typed str with no coercing validator, a client could suppress the scan on a tool call by sending a non-string.

The tool call a caller posts to /v1/chat/completions, which the OpenAI path forwards verbatim:

{"id": "call_1", "type": "function",
 "function": {"name": 123,
              "arguments": "{\"to_account\": \"ATTACKER-001\", \"ssn\": \"123-45-6789\"}"}}

Counting calls to _call_panw_api and checking whether the arguments reached it:

function.name	before	after
"transfer_funds"	scanned	scanned
123	skipped, args never sent	scanned
{"x": 1}	skipped, args never sent	scanned
None	scanned	scanned

None was always fine, so the trigger is specifically a wrong type — the shape a ValidationError-to-(None, None) fallback produces. The fix widens the existing _coerce_arguments validator to cover name, rather than adding a second near-identical one.

Unit

211 passed. Two regression guards, each verified to fail against the defect it covers.

TestPanwAirsToolCallBlockMaskedDataRouting drives _scan_tool_calls_for_guardrail end to end rather than calling _build_error_detail directly. Restoring the auto-merged combination fails it:

FAILED TestPanwAirsToolCallBlockMaskedDataRouting::test_response_side_block_still_returns_caller_input
1 failed, 2 passed

test_non_string_tool_name_does_not_suppress_the_scan is parametrized over 123, {"x": 1}, ["a"] and True. Reverting the validator to arguments-only fails all four:

FAILED ...test_non_string_tool_name_does_not_suppress_the_scan[True]
FAILED ...test_non_string_tool_name_does_not_suppress_the_scan[bad_name1]
FAILED ...test_non_string_tool_name_does_not_suppress_the_scan[bad_name2]
4 failed

Type

🐛 Bug Fix

Caveats (if any)

• Masked verdicts are expected to keep the name line, so only the arguments slice is written back
• Custom tool calls and calls with no function payload are skipped rather than scanned
• MCP gateway invocations keep their existing tool_event reporting
• This branch contains a merge of litellm_internal_staging, so the diff is larger than the original change. Rebase instead if you prefer a linear history; the fix is one call site plus the removed parameter.
• Not verified: the AIRS service itself, for want of a tenant. Part 1's stand-in enforces only the documented ecosystem rule, and Part 2's accepts any payload, so neither run proves what a real AIRS returns. Both exercise everything downstream of response.json(), which is all either change touches.
• On /v1/messages the block detail is delivered as a Python-repr string rather than structured JSON ("400: {'error': {...}}"). Pre-existing, reproduces before both PRs, and worth its own ticket.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/0dd1270c67bc4a8393052f161740fc32

---

Note

^{Cursor Bugbot is generating a summary for commit 887be41. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

The PR changes Prisma AIRS tool-call scanning from unsupported OpenAI tool_event payloads to side-aware plain-text scans while preserving MCP tool_event behavior.

• Scans function names and arguments as newline-separated prompt or response text.
• Extracts supported tool-call shapes through a validated, typed helper.
• Routes masked tool arguments according to scan side and preserves caller-facing audit data.
• Adds regression coverage for masking, blocking, malformed values, fallback behavior, and MCP isolation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/panw_prisma_airs/panw_prisma_airs.py	Reworks tool-call extraction and AIRS scanning to use side-aware plain text, with typed helper parameters addressing the previous review thread.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_panw_prisma_airs.py	Updates and expands mocked regression coverage for plain-text tool-call scanning, masked-data routing, malformed inputs, and MCP behavior.

_{Reviews (5): Last reviewed commit: "fix(panw_prisma_airs): a wrong-typed too..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 92.15686% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ardrail_hooks/panw_prisma_airs/panw_prisma_airs.py	92.15%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ yucheng-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[shivamrawat1]

@greptile review again

[shivamrawat1]

@BugBot

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_panw_airs_tool_call_scan (887be41) with litellm_internal_staging (abddd64)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (9a96b93) during the generation of this report, so abddd64 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

bugbot run

[yucheng-berri]

Ran a base-vs-head live diff of this PR on a real proxy (real Postgres, real Gemini upstream, identical requests to both sides). The core fix is solid and I could reproduce it cleanly — the LIT-5279 repro goes 500 on base → 200 on head, and the MCP path correctly keeps ecosystem: "mcp". Good, targeted diagnosis.

Three things I'd like a closer look at before merge. I'm not asking for immediate fixes — a couple of these may be intentional, and one of them I genuinely can't settle without a PANW tenant.

1. Tool names no longer reach AIRS, and empty-arg tool calls aren't scanned at all

The new caller does if not args_text or not args_text.strip(): continue, and only the arguments string goes out — tool_invoked is gone from the OpenAI-format path.

Assistant tool call {"name":"BLOCKME_exfiltrate_all_secrets","arguments":""}, same request to both sides:

base  tool_events[0].metadata = { "ecosystem":"openai",
        "tool_invoked":"BLOCKME_exfiltrate_all_secrets" }   <- name reaches AIRS
head  (no AIRS request emitted for this tool call at all)   HTTP 200

And for a tool call that is scanned:

base  contents[0].tool_event.metadata.tool_invoked = "get_weather"
head  contents[0].response = "{\"city\": \"San Francisco\"}"   <- name absent

I understand the old tool_event path was hard-failing, so nothing was being enforced in practice either — this isn't a regression from a working state. But the outcome is that an AIRS profile keyed on tool name silently stops firing, with no error and no config flag to restore it. Would sending name and arguments together as prompt text (e.g. prefixing the tool name) keep the fix while preserving name visibility?

Two related things to weigh:

• test_tool_call_empty_args_block_by_name_policy became test_tool_call_empty_args_not_scanned — same handler, same tool still named dangerous_tool, same AIRS block verdict, but the assertion flipped from "raises HTTPException 400" to mock_api.assert_not_called(). That reads as encoding the new gap rather than the intended contract, and it's why CI stays green.
• litellm-docs/docs/proxy/guardrails/panw_prisma_airs.md lines 10, 283 and 285 all promise tool-name scanning ("scan tool name and arguments", "tool_event payloads containing tool name… always use request mode"). No docs PR accompanies this one.

2. Two shapes that base tolerated now raise

The old extractor used hasattr guards and fell through to None; the new one ends in a bare return tool_call.function.arguments. Custom (non-function) tool calls carry .custom and have no .function, and ChatCompletionMessageToolCall accepts function=None verbatim. Both shapes are constructed by litellm's own Message / Delta / stream_chunk_builder paths:

Message(role="assistant", tool_calls=[{"id":"c1","type":"custom",
                                       "custom":{"name":"x","input":"y"}}])

head  AttributeError: 'ChatCompletionMessageCustomToolCall'
                      object has no attribute 'function'
head  function=None -> AttributeError: 'NoneType' object has no attribute 'arguments'
base  handled without raising

To be straight about reachability: I ran this directly against each tree's handler, and I could not find a live HTTP route that emits a custom tool call today — so treat the crash as confirmed but the route as latent. A hasattr/None guard would be cheap insurance either way. Note _set_tool_call_arguments still uses the old hasattr(tool_call, "function") shape check, so the read and write paths now disagree about what a tool call is.

3. Two behaviors from the original guardrail PR are reversed

Both came from #22999 and were labelled as fixes there. Running the base test file against this branch, 8 previously-pinned behaviors fail while this branch's own 232 tests pass:

• TestPanwAirsToolEventIsResponseFix — docstring reads "Bug A fix: tool_event scans must not set is_response metadata." This PR changes the hardcoded is_response=False to pass-through, so post-call tool arguments now reach AIRS as response content and get evaluated in the other policy direction.
• TestPanwAirsResponseToolCallMasking — pinned that response-side tool scans read prompt_masked_data. This PR reads response_masked_data. With mask_response_content=True, a tool call AIRS wants masked would now be blocked with a 400 instead of masked, because the key it looks for isn't there.

Here's the part I can't settle: which masked-data key real AIRS returns for a response-typed content scan. If it returns response_masked_data, your change is correct and the old test encoded a wrong assumption — in which case the right move is deleting that test with a note, not leaving the reversal implicit. There's no PANW tenant in my environment, so this needs either a tenant or an answer from PANW. Worth resolving before merge rather than after, since it decides whether #3 is a fix or a regression.

---

Caveat on my rig: no PANW tenant available, so the AIRS endpoint was a local stand-in replaying the documented AIRS schema and enforcing its documented validation — it rejects any tool_event whose ecosystem isn't "mcp", which is what reproduces the LIT-5279 failure at base. Everything downstream of response.json() ran for real.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 380c449. Configure here.}

[devin-ai-integration]

a477c3e sends name and args newline-joined, scans name-only calls again, and parses shapes via Pydantic so custom or missing functions skip safely. Masked-key question still needs your call.

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

bugbot run

[yucheng-berri]

Your a477c3e841 covers the tool-name and empty-args points properly, including the two things I was most worried about — the "\n" prefix strip in _masked_tool_call_arguments so masked text can't round-trip a corrupted arguments back onto the tool call, and restoring test_tool_call_empty_args_block_by_name_policy to assert the block rather than the skip. The pydantic slice is a cleaner shape than the hasattr ladder I was going to propose.

First, an apology. I force-pushed over a477c3e841 at 18:33 while you were pushing it — my git fetch and git push --force-with-lease were in the same command, so the lease was computed against the ref my own fetch had just advanced and it didn't protect you. I noticed immediately and restored your commit as branch head; it is intact and unmodified at a477c3e841, and my work is now a normal follow-up commit on top of it. Nothing of yours was lost, and I won't chain fetch and push like that again.

What I added on top: f6aff5f089

One gap left, and it's a scan bypass rather than a crash.

_ToolCallFunctionSlice types arguments as str | None, so model_validate raises ValidationError on anything else and _get_tool_call_function returns (None, None). The caller then reads that as "no function payload" and continues. But the OpenAI request path forwards client-supplied tool_calls verbatim (litellm/llms/openai/chat/guardrail_translation/handler.py:236-241 appends the raw dict with no validation), so a client can post arguments as already-parsed JSON and the entire tool call — name included — is skipped without ever reaching AIRS.

Verified against your commit before changing anything:

tc = {"id":"c1","type":"function",
      "function":{"name":"exfiltrate","arguments":{"ssn":"123-45-6789"}}}
H._get_tool_call_function(tc)   ->  (None, None)      # never scanned

That is worse than the pre-PR behaviour, which crashed with a 400 on args_text.strip() — ugly, but fail-closed. Silently skipping is the one outcome a scanner shouldn't have.

The fix is a mode="before" validator that coerces dict/list to JSON instead of rejecting it. After it:

dict arguments     -> ('exfiltrate', '{"ssn": "123-45-6789"}')   # scanned
str arguments      -> ('f', '{"a":1}')
empty arguments    -> ('dangerous_tool', '')                      # your name-only path
custom tool        -> (None, None)                                # still skipped
null function      -> (None, None)                                # still skipped

Test test_parsed_dict_arguments_are_still_scanned asserts both the SSN and the tool name reach _call_panw_api. It fails if the coercion is removed; the suite is 190 passed.

Live check on the combined result

Proxy on this branch, real Gemini upstream, AIRS endpoint replaying the documented schema and enforcing the real ecosystem validation:

• The LIT-5279 repro still goes 500 on the merge base, 200 here — the core fix is unaffected.
• The dict-arguments request now reaches AIRS with {"ssn": "123-45-6789"} scanned, where the merge base returned 400 Invalid request format: 'dict' object has no attribute 'strip' from inside the guardrail.

One thing worth knowing about that last case: the request still fails, but now at the provider (Unable to convert openai tool calls ... the JSON object must be str, bytes or bytearray, not dict) rather than in the guardrail. Dict-valued arguments is genuinely invalid for the Gemini transform, so that failure is correct and pre-existing — the change just stops the guardrail from being the thing that reports it, and makes sure the content gets scanned on the way through.

Still open, and not something either of us should decide alone

_get_masked_text still reads response_masked_data for response-side tool scans, where the pre-PR code read prompt_masked_data and TestPanwAirsResponseToolCallMasking (from the original guardrail PR) pinned that. Which key real AIRS returns for a contents=[{"response": ...}] scan decides whether that reversal is a fix or a regression, and there's no PANW tenant in this environment to settle it. I'd leave it as-is and get an answer from PANW rather than guess in either direction. Flagging it here so it doesn't merge as an unexamined behaviour change.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a477c3e. Configure here.}

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

bugbot run

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 887be41. Configure here.}