fix(guardrails): walk Responses-API text taxonomy in shared content helpers

[yucheng-berri]

Relevant issues

Linear ticket

Resolves LIT-4294

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy repro on both /v1/responses and /v1/chat/completions. The Responses request body is the shape a chat-to-Responses bridge (e.g. Google ADK) POSTs; the guardrail is a custom pre_call hook that logs what _content_utils extracted and calls walk_user_text to redact AKIAEXAMPLE

Before (unfixed, HEAD 0f1e29b)

curl -sS http://localhost:4000/v1/responses \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "input": [
      {"type": "message", "role": "user",
       "content": [{"type": "input_text", "text": "AKIAEXAMPLE"}]},
      {"type": "function_call", "call_id": "c1", "name": "get_weather", "arguments": "{}"},
      {"type": "function_call_output", "call_id": "c1",
       "output": [{"type": "input_text", "text": "another AKIAEXAMPLE inside tool result"}]}
    ]
  }'

proxy log:

LIT4294 pre_call call_type=aresponses fragments_seen=0 inspection_msgs=0
LIT4294 fragments=[]
LIT4294 inspection=[]
LIT4294 walk_user_text visited=0

outbound POST body to OpenAI carries the unredacted keys; every text guardrail is a no-op on this path. /v1/chat/completions on the same proxy sees fragments_seen=1 and redacts correctly, so the bypass is Responses-API-only

After (this PR, HEAD 18d3a6f)

Same curl:

LIT4294 pre_call call_type=aresponses fragments_seen=2 inspection_msgs=2
LIT4294 fragments=['AKIAEXAMPLE', 'another AKIAEXAMPLE inside tool result']
LIT4294 inspection=[{'role': 'user', 'content': 'AKIAEXAMPLE'},
                    {'role': 'tool', 'content': 'another AKIAEXAMPLE inside tool result'}]
LIT4294 walk_user_text visited=2

outbound POST body to https://api.openai.com/v1/responses:

{'model': 'gpt-4o-mini', 'input': [
   {'type': 'message', 'role': 'user',
    'content': [{'type': 'input_text', 'text': '[REDACTED]'}]},
   {'type': 'function_call', 'call_id': 'c1', 'name': 'get_weather', 'arguments': '{}'},
   {'type': 'function_call_output', 'call_id': 'c1',
    'output': [{'type': 'input_text', 'text': 'another [REDACTED] inside tool result'}]}]}

Both user text and tool-output text are redacted end-to-end; function_call metadata is preserved. The inspection payload keeps role fidelity (function_call_output surfaces as role: "tool" for downstream guardrails to consume). AIM's schema-validating POST to /fw/v1/analyze collapses tool to user locally right before the POST, keeping AIM schema-safe without imposing that coercion on other guardrails

Type

🐛 Bug Fix

Changes

Two fixes in litellm/proxy/guardrails/_content_utils.py and one AIM-local fix in litellm/proxy/guardrails/guardrail_hooks/aim/aim.py.

1. _iter_text_parts_in_content and the list branch of walk_user_text now recognise {text, input_text, output_text} as text-carrying part types via a new TEXT_PART_TYPES frozenset. The Responses API uses input_text on request messages and output_text on assistant messages; the previous single-value check made every text guardrail a no-op on /v1/responses
2. _coerce_input_to_messages and the list branch of walk_user_text now walk the actual Responses input taxonomy (message items, function_call, function_call_output, bare content-part dicts, bare strings) instead of the all(item has role) gate. That gate returned False for any tool-calling turn (function_call and function_call_output have no role), which caused the whole list to collapse to a single-message blob whose content was never walked. function_call items are dropped from the inspection view since they carry no free-form text (arguments walking is LIT-4304 scope); function_call_output maps to role: tool by default, preserving the caller-supplied role when present. Role fidelity is otherwise preserved throughout: message items, bare parts, and content items pass through their original role
3. AIM's schema-validating POST to /fw/v1/analyze needs {system, user, assistant}-only roles because the flatten drops tool_call_id and name. That coercion lives in a private AimGuardrail._build_aim_inspection_messages helper right before the POST rather than in the shared helper, so other guardrails (Lakera, Cato, Lasso, Repello, IBM, Azure, secret detection) see the caller's original role fidelity on their inspection payloads

Regression tests: tests/test_litellm/proxy/guardrails/test_content_utils.py covers the Responses taxonomy walk, input_text/output_text extraction and redaction, function_call_output.output list + string form, role preservation on bare parts and message items, and the function_call_output → role: tool default. tests/test_litellm/proxy/guardrails/guardrail_hooks/test_aim.py covers the AIM-local role coercion. 41 tests pass; every new test fails on unfixed code

QA verification

Two layers: live proxy hitting real OpenAI, and unit-level checks directly against the helpers.

Live proxy (cases 1-9, 16): custom pre_call guardrail hook logs BEFORE/AFTER snapshots of data["messages"] and data["input"], the iter_message_text fragment list, and the build_inspection_messages role list. Proxy launched with PYTHONPATH pinned at the PR worktree so the exercised code is the PR source, not an installed package. One curl per case; each AFTER snapshot changed only leaf text values, every structural field (role, type, call_id, tool_call_id, tool_calls, name, arguments, annotations) byte-for-byte identical to BEFORE. OpenAI responses confirmed no AKIAEXAMPLE marker reached the model.

Unit-level (cases 9-15): import script that calls build_inspection_messages, iter_message_text, walk_user_text, apply_redacted_messages_back, and AimGuardrail._build_aim_inspection_messages directly with hand-constructed payloads. Uses copy.deepcopy between assertions. Covers shapes OpenAI's API rejects (bare input_text/output_text parts, developer role, exact AIM-payload role values).

Regression gate (case 17): ran the suite on the PR tip → 41 pass. Restored staging HEAD for _content_utils.py and aim.py while keeping the PR's tests checked out; the 4 AIM tests and 8 content-utils tests failed. Those 12 are exactly the behaviors this PR fixes.

Before-baseline: second proxy on staging HEAD, same curl → fragments=[], walk_visited=0, OpenAI response echoed the raw AKIAEXAMPLE back. Confirmed the bug on unfixed code before verifying the fix.

Case-by-case verdict on 18d3a6fe5b:

#	Description	Verdict
1	chat/completions plain string	PASS
2	chat/completions multimodal text+image	PASS
3	chat/completions tool-call conversation	PASS
4	Responses raw string input	PASS
5	Responses message with input_text	PASS
6	Responses message with output_text	PASS
7	Responses mixed tool-calling (main repro)	PASS
8	Responses function_call_output string output	PASS
9	Bare input_text/output_text parts	PASS
10	Shared inspection role fidelity	PASS
11	AIM pre-call schema-safe inspection	PASS
12	AIM post-call / output inspection	PASS
13	Non-AIM guardrail role fidelity	PASS
14	Mask/write-back regression	PASS
15	Empty / non-text / unsupported items	PASS
16	Outbound request structural integrity	PASS
17	Unit/regression tests	PASS (41/41 on 18d3a6fe5b; 12 fail on staging HEAD)

Caveat

The apply_redacted_messages_back helper writes the flat [{role, content}] inspection payload back over data["messages"], which drops sibling assistant.tool_calls and any tool_call_id on a chat-completions tool-message masking path. That's a pre-existing structural flatten limitation, not introduced by this PR; verified case 14 does not corrupt the shape further. Fix belongs to a separate ticket.

Known follow-up scope

The reviewer surfaced three related silent-bypass paths on the Responses-API surface that this PR deliberately leaves for follow-up commits so the change stays scoped to the customer's original repro. Tickets filed:

• LIT-4302: custom_tool_call_output items share the output shape with function_call_output and still bypass the helper.
• LIT-4303: reasoning.summary_text items replayed as input (ADK bridge, LangGraph-style loops) carry chain-of-thought text that guardrails do not currently walk.
• LIT-4304: function_call.arguments and its chat-completions sibling message.tool_calls[i].function.arguments are JSON strings that can carry user-supplied secrets; the shared helper does not walk them today. Pre-existing gap on both APIs, best fixed together.

---

Note

Medium Risk
Changes shared proxy guardrail parsing/redaction on the security-critical pre-call path; behavior shifts for all hooks using _content_utils, though scope is limited to Responses API shapes and extensive tests were added.

Overview
Fixes a Responses API bypass where text guardrails saw zero fragments on /v1/responses (including tool-calling turns), while chat completions kept working.

Shared _content_utils now treats text / input_text / output_text as text parts and walks mixed input lists item-by-item (messages, bare strings/parts, function_call_output.output) instead of collapsing the list when some items lack role. walk_user_text and build_inspection_messages follow the same rules so scan, redact, and remote inspection payloads stay aligned; function_call_output defaults to role: tool in the shared flatten.

AIM alone maps non-{system,user,assistant} roles to user before /fw/v1/analyze, avoiding 422s without changing other hooks’ role fidelity.

Regression coverage added in test_content_utils.py and test_aim.py.

^{Reviewed by Cursor Bugbot for commit 18d3a6f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR fixes a Responses API bypass where shared guardrail content helpers silently skipped almost all /v1/responses bodies, making text guardrails (secret detection, PII, remote analyzers) a no-op on that path while /v1/chat/completions continued to work correctly.

• Text-part type coverage: _iter_text_parts_in_content and the walk_user_text list branch now recognise input_text and output_text alongside text via a new TEXT_PART_TYPES frozenset.
• Responses input taxonomy: _coerce_input_to_messages and walk_user_text now walk the full Responses input list item-by-item instead of the old all(item has role) gate.
• Role fidelity + AIM coercion: build_inspection_messages preserves synthesised roles faithfully; AIM collapses unsupported roles to user in its own local helper.

Confidence Score: 5/5

The change is safe to merge: the new taxonomy walking is consistent between the read path and the write path, all previously-supported shapes remain covered, and targeted regression tests confirm both extraction and in-place redaction on the new Responses API paths.

All three modified helpers are updated consistently, the AIM-local role coercion is correctly isolated so the shared helper remains role-faithful for other guardrails, no existing tests are weakened, and the three CI failures are confirmed unrelated to this PR.

No files require special attention; the core change in _content_utils.py is well-scoped and the AIM integration follows naturally.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/_content_utils.py	Core fix: adds TEXT_PART_TYPES frozenset, rewrites _coerce_input_to_messages and the walk_user_text list branch to properly walk the Responses API taxonomy.
litellm/proxy/guardrails/guardrail_hooks/aim/aim.py	Moves AIM-specific role coercion into _build_aim_inspection_messages, keeping the shared build_inspection_messages role-faithful.
tests/test_litellm/proxy/guardrails/test_content_utils.py	Adds targeted regression tests for input_text/output_text extraction, mixed tool-call taxonomy walking, and function_call_output redaction.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_aim.py	New unit tests for _build_aim_inspection_messages covering role coercion and safe-role pass-through.

_{Reviews (4): Last reviewed commit: "refactor(guardrails): preserve role fide..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will degrade performance by 12.1%

❌ 1 regressed benchmark
✅ 29 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_completion_simple_message	4.2 ms	4.8 ms	-12.1%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing litellm_lit-4294-guardrails-responses (18d3a6f) with litellm_internal_staging (9d74548)}

[yucheng-berri]

@greptileai please re-review — pushed two follow-up commits since the last score:

• 7e42d65: map function_call_output straight to role: user in _coerce_input_to_messages so a schema-invalid bare tool message is never materialised in memory, even in isolation. The _INSPECTION_SAFE_ROLES coercion in build_inspection_messages stays as defense-in-depth against caller-supplied non-standard roles (e.g. developer on chat completions where flattening drops the required tool_call_id).
• 20dc5a3: three test additions (bare-string function_call_output.output, coercion of caller-supplied developer role, dropped em-dash in a new docstring).

Both changes are mutation-verified to fail on the pre-patch base branch. 35 tests pass locally, ruff format + ruff check clean.

[yucheng-berri]

bugbot run

[ryan-crabbe-berri]

so basically if its not a role we recognize then we say its user role instead, do i have that right?

can you give an example of this happening? ive never seen this but i could imagine it being the case

[yucheng-berri]

{"role":"tool","tool_call_id":"c1","content":"sunny"} gets flattened to {"role":"tool","content":"sunny"}, which AIM rejects because tool_call_id is missing, so we coerce it to {"role":"user","content":"sunny"} to keep it valid

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 7e42d65. Configure here.}

[yucheng-berri]

@greptileai please re-review — pushed a5bfcc4 which changes approach after QA + reviewer feedback:

The generic unsupported-role coercion in build_inspection_messages (previous commits) turned out to be too broad. Combined with the pre-existing apply_redacted_messages_back flatten-and-write-back in Lakera/AIM/Cato, it converted a loud OpenAI 400 on chat-completions tool-message masking into silent semantic corruption of the outbound request. Reproduced live via a Lakera-shaped custom guardrail.

Shift:

• Removed the generic coercion + _INSPECTION_SAFE_ROLES from _content_utils.py. Shared helper now returns caller-supplied roles unchanged.
• Added AIM-specific AimGuardrail._build_aim_inspection_messages that coerces unsupported roles to user right before the POST to /fw/v1/analyze, since AIM validates against the OpenAI chat schema and the flatten drops tool_call_id/name.
• Moved the caller-supplied tool/developer role tests to a new test_aim.py. Kept test_build_inspection_messages_function_call_output_becomes_user in test_content_utils.py since that's about synthesising a role for a role-less Responses item, not caller coercion.

QA Path F re-run: shared helper now preserves role: tool on the inspection payload; Lakera/Cato-shaped mask+write-back path fails loud (OpenAI 400) instead of silently corrupting, matching pre-PR base behavior. The pre-existing apply_redacted_messages_back structural flatten remains as separate follow-up.

37 tests pass locally, ruff format + check clean.

[yucheng-berri]

@greptileai please re-review — pushed 18d3a6f to tighten the role-handling logic in the shared helper. Preserve role fidelity whenever the source item provides one, only synthesise when the source shape genuinely has none:

• Bare content-part dicts with an explicit role keep it; role-less ones default to user.
• Responses message items pass their role through unchanged.
• function_call_output defaults to role: tool (semantic equivalent of chat-completions tool messages) instead of role: user, so Responses and chat completions produce symmetric inspection payloads. A caller-supplied role still wins.

AIM's local coercion in _build_aim_inspection_messages still collapses tool/function/developer to user for its schema-validating POST, so nothing regresses there.

Also ran a 42-check acceptance matrix against HEAD covering (a) every old supported case (chat completions strings, multimodal text parts, raw input, list-of-messages input, empty strings, image parts), (b) every new Responses case (input_text/output_text extraction and redaction, tool-call taxonomy, function_call skip, function_call_output list + string form), (c) role fidelity (bare parts, message items, function_call_output default, explicit role preservation), (d) AIM-local coercion for unsupported roles, and (e) structural invariants (all metadata preserved, tool_calls / tool_call_id / assistant sibling preserved, no top-level messages field added). All 42 pass.

41 unit tests pass locally (34 shared + 7 AIM-specific). Ruff format + check clean. The 3 failing CI checks are the current Chainguard cgr.dev registry outage affecting every open PR (test-server-root-path × 2 + build_docker_database_image); unrelated to this PR.

[yucheng-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 18d3a6f. Configure here.}

[cursor]

Cato analyze missing role coercion

Medium Severity

build_inspection_messages now emits role: "tool" for Responses function_call_output and no longer collapses non-chat roles, while Cato still POSTs that payload to /fw/v1/analyze without the AIM-local coercion. That can reproduce the bare-tool 422 AIM avoids on tool-calling /v1/responses requests.

Additional Locations (1)

• litellm/proxy/guardrails/_content_utils.py#L226-L228

 

^{Reviewed by Cursor Bugbot for commit 18d3a6f. Configure here.}

[yucheng-berri]

this is not a regression, Cato skipped check before.