Skip to content

chore(release): backport #32542 to stable/1.90.x and cut 1.90.4 - #32937

Merged
yuneng-berri merged 3 commits into
stable/1.90.xfrom
litellm_backport_1_90_x_bp-32542-90x
Jul 11, 2026
Merged

chore(release): backport #32542 to stable/1.90.x and cut 1.90.4#32937
yuneng-berri merged 3 commits into
stable/1.90.xfrom
litellm_backport_1_90_x_bp-32542-90x

Conversation

@yuneng-berri

Copy link
Copy Markdown
Contributor

Relevant issues

Backports #32542 onto stable/1.90.x and cuts 1.90.4. On the /v1/responses path the shared guardrail content helpers in litellm/proxy/guardrails/_content_utils.py extracted no text, so every guardrail built on those helpers (AIM, Lakera v2, Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret detection) inspected an empty payload and passed the request through without scanning or redacting. Chat completions were unaffected; the gap was specific to the Responses input taxonomy. This restores parity so those helpers walk the Responses input shapes (text / input_text / output_text parts, and message / function_call / function_call_output items) the same way they walk chat messages

Linear ticket

Resolves LIT-4294

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

What is included

Adaptation notes

The pick is content-identical to the staging commit; the only divergence is formatting. stable/1.90.x gates format-check on Black 26.3.1 at 88 columns (cd litellm && black --check .), while staging authored the change with ruff format at 120 columns. The two touched source files were reflowed with the line's own Black so they pass lint-black; no logic changed. The test files are outside the line's Black scope and were taken as-is

Known noise on this line

The targeted guardrail suite is green on this line both before and after the pick, so there is no pre-existing red to discount. tests/test_litellm/proxy/guardrails/test_content_utils.py was 27 passed / 0 failed at the line tip before the pick; after the pick the two touched test files run 41 passed / 0 failed

Screenshots / Proof of Fix

Live proxy on stable/1.90.x, /v1/responses routed to a real Anthropic model through the responses bridge. A custom pre_call guardrail logs what the shared helpers extracted (iter_message_text fragments, build_inspection_messages roles) and calls walk_user_text to redact the marker AKIAEXAMPLE, mirroring the reproducer in #32542. Same request body in both runs

Request:

curl -sS http://localhost:4001/v1/responses \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{
    "model": "claude-repro",
    "input": [
      {"type": "message", "role": "user",
       "content": [{"type": "input_text", "text": "AKIAEXAMPLE"}]},
      {"type": "function_call", "call_id": "c1", "name": "get_weather", "arguments": "{}"},
      {"type": "function_call_output", "call_id": "c1",
       "output": [{"type": "input_text", "text": "another AKIAEXAMPLE inside tool result"}]}
    ]
  }'

Before (line tip 2d28d8fadf, unpicked), the helpers see nothing and nothing is redacted:

LIT4294 pre_call call_type=aresponses fragments_seen=0 inspection_msgs=0
LIT4294 fragments=[]
LIT4294 inspection=[]
LIT4294 walk_user_text visited=0
LIT4294 redacted_input=[{'type': 'message', 'role': 'user', 'content': [{'type': 'input_text', 'text': 'AKIAEXAMPLE'}]}, {'type': 'function_call', 'call_id': 'c1', 'name': 'get_weather', 'arguments': '{}'}, {'type': 'function_call_output', 'call_id': 'c1', 'output': [{'type': 'input_text', 'text': 'another AKIAEXAMPLE inside tool result'}]}]

After (pick 342d8897b6), both the user text and the tool-output text are redacted and role fidelity is preserved (function_call_output -> role: tool), while function_call metadata (call_id, name, arguments) is untouched:

LIT4294 pre_call call_type=aresponses fragments_seen=2 inspection_msgs=2
LIT4294 fragments=['AKIAEXAMPLE', 'another AKIAEXAMPLE inside tool result']
LIT4294 inspection=[{'role': 'user', 'content': 'AKIAEXAMPLE'}, {'role': 'tool', 'content': 'another AKIAEXAMPLE inside tool result'}]
LIT4294 walk_user_text visited=2
LIT4294 redacted_input=[{'type': 'message', 'role': 'user', 'content': [{'type': 'input_text', 'text': '[REDACTED]'}]}, {'type': 'function_call', 'call_id': 'c1', 'name': 'get_weather', 'arguments': '{}'}, {'type': 'function_call_output', 'call_id': 'c1', 'output': [{'type': 'input_text', 'text': 'another [REDACTED] inside tool result'}]}]

Both runs returned HTTP 200 from the real Anthropic call with the redacted input. Targeted tests on the line: 41 passed / 0 failed, versus a 27 passed / 0 failed baseline (the AIM test file is new in this pick). A deep adversarial pass over the shared-helper callers (AIM, Cato, IBM, Lakera v2, Lasso, Repello, Azure Content Safety) found no existing caller broken by the behavior change

Type

🐛 Bug Fix

Changes

Cherry-pick of #32542 onto stable/1.90.x, reflowed to the line's Black formatter, plus the 1.90.4 version bump and lock refresh. No schema, dependency, or configuration changes

yucheng-berri and others added 3 commits July 11, 2026 12:07
…elpers (#32542)

* fix(guardrails): walk Responses-API text taxonomy in shared content helpers

Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently
drops all text on the /v1/responses path. AIM turns it into a loud 422 (
{"error":"No messages in the request"}); every other guardrail (Lakera v2,
Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret
detection) scans an empty payload and lets the request through unscanned.

Three defects, all in _content_utils.py:

1. _iter_text_parts_in_content recognised only part.type == "text", but the
   Responses API uses input_text (request) and output_text (assistant).
2. _coerce_input_to_messages gated on "every item has a role key"; any
   Responses input list containing a function_call or function_call_output
   item failed the check and was wrapped as one opaque blob.
3. build_inspection_messages forwarded any role through, including a bare
   tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze
   reject with a schema error.

Fix walks the actual Responses item taxonomy (message, function_call,
function_call_output, bare content parts and strings), recognises
{text, input_text, output_text} everywhere, and coerces any role outside
{system, user, assistant} to user in the outbound inspection payload.

* style: ruff-format changed guardrail files

* test(guardrails): cover function_call_output string form; drop em-dash in new docstring

* fix(guardrails): map function_call_output straight to user role

Avoids ever materialising a schema-invalid bare tool message. The
downstream role-safety coercion in build_inspection_messages still
guards genuinely caller-supplied non-standard roles (developer,
function, custom values); add a regression test covering that path
so the coercion has real coverage after this simplification.

* test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages

* docs(test): soften AIM-specific claims in LIT-4294 test docstrings

Ryan's review flagged that several test docstrings assert AIM's
/fw/v1/analyze validates + rejects specific schema violations. That
behavior is customer-reported in the LIT-4294 writeup, not directly
verified by us. Rephrase to attribute the AIM 422 to the customer's
writeup and describe the underlying constraint as the OpenAI chat
schema; any downstream API that validates against that schema rejects
the same shape.

* refactor(guardrails): move unsupported-role coercion into AIM only

The generic coercion in build_inspection_messages collapsed any role
outside {system, user, assistant} to user for every caller of the
helper. Combined with the pre-existing apply_redacted_messages_back
write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400
on chat-completions tool-message masking into a silent semantic
corruption of the outbound request (role tool with tool_call_id got
rewritten to bare role user, dropping the assistant + tool_calls
sibling).

AIM specifically requires the coercion because its /fw/v1/analyze
validates the payload against the OpenAI chat schema; other guardrails
either do not validate roles or do their own reconstruction. Move the
coercion to AimGuardrail._build_aim_inspection_messages so the shared
helper keeps caller roles intact and no new cross-guardrail role
corruption is introduced. The pre-existing apply_redacted_messages_back
structural flatten remains as separate follow-up work.

function_call_output items still synthesise role user in the shared
helper because they have no natural role field, which is a different
concern from coercing a caller-supplied role.

* refactor(guardrails): preserve role fidelity in shared _content_utils

Shared inspection helpers should extract text and preserve semantic
role signals; role coercion for third-party schema safety stays inside
the guardrail that needs it (AIM).

Three shared-helper changes:
- Bare content-part dicts (input_text/output_text) with an explicit role
  keep it; only role-less parts default to user.
- Responses message items already had their role preserved; the
  behavior is now covered by an explicit test.
- function_call_output items default to role tool (semantic equivalent
  of the chat-completions tool message shape) instead of role user, so
  Responses and chat completions produce symmetric inspection payloads.
  A caller-supplied role on the item is still preserved.

AIM's schema-safe coercion in _build_aim_inspection_messages already
handles the resulting role tool: it collapses to user before the POST
to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the
bare tool message (no tool_call_id can survive the flatten). Added a
regression test in test_aim.py covering that path.

(cherry picked from commit e84a19a)
@greptile-apps

greptile-apps Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This backport fixes a gap in the shared guardrail content helpers (_content_utils.py) where the /v1/responses input taxonomy was not walked, causing every text-inspection guardrail (AIM, Lakera v2, Cato, Lasso, Repello, IBM, Azure Content Safety, secret detection) to scan an empty payload on that path. The cherry-pick from staging is content-identical; the two touched source files were reflowed with Black 26.3.1 to satisfy the stable branch's format gate — no logic changed.

  • _content_utils.py: adds TEXT_PART_TYPES (text, input_text, output_text), rewrites _coerce_input_to_messages to walk Responses input items individually (message, function_call_output, bare text parts), and extends walk_user_text to redact the same shapes in place.
  • aim.py: introduces _build_aim_inspection_messages which coerces any role outside {system, user, assistant} to "user" before POSTing to AIM's /fw/v1/analyze, preventing 422 errors from bare tool roles in the flattened inspection payload.
  • Tests: 14 new mock-only unit tests across two files; two existing assertions reflowed to single lines (cosmetic only); no test coverage weakened.

Confidence Score: 5/5

Safe to merge — the change is a well-scoped bug fix that restores parity between the chat completions and Responses-API code paths in the guardrail helpers. No schema, dependency, or interface changes.

The two modified source files have straightforward, well-isolated logic changes backed by 14 new tests that directly exercise each fixed path. The formatting-only reflow on the stable branch's Black gate introduces no behavioural divergence from the staging commit. The AIM role-coercion wrapper is additive and does not alter the upstream LLM request. No pre-existing tests were weakened, and the targeted guardrail test suite went from 27 to 41 passed.

No files require special attention.

Important Files Changed

Filename Overview
litellm/proxy/guardrails/_content_utils.py Core fix: adds TEXT_PART_TYPES frozenset, rewrites _coerce_input_to_messages to walk Responses-API items individually, and updates walk_user_text to handle message/function_call_output shapes. Logic is correct and well-tested.
litellm/proxy/guardrails/guardrail_hooks/aim/aim.py Adds _build_aim_inspection_messages static method that coerces non-standard roles (e.g. "tool") to "user" before POSTing to AIM's /fw/v1/analyze endpoint. Both call sites updated correctly.
tests/test_litellm/proxy/guardrails/test_content_utils.py Adds 10 new regression tests covering input_text/output_text parts, function_call_output traversal, and role-preservation in the shared helpers. Two existing assertions are reflowed to single lines (cosmetic). No test coverage is weakened.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_aim.py New file with 4 mock-only unit tests for _build_aim_inspection_messages covering tool-role coercion, non-standard roles, function_call_output, and safe-role pass-through. No real network calls.
pyproject.toml Version bump from 1.90.3 to 1.90.4 in both [project] and [tool.commitizen] sections.

Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.90.4" | Re-trigger Greptile

@yuneng-berri
yuneng-berri enabled auto-merge July 11, 2026 20:28
@yuneng-berri
yuneng-berri merged commit ae0cfc0 into stable/1.90.x Jul 11, 2026
38 of 52 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_backport_1_90_x_bp-32542-90x branch July 11, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants