chore(release): backport #32542 to stable/1.90.x and cut 1.90.4 - #32937
Conversation
…elpers (#32542) * fix(guardrails): walk Responses-API text taxonomy in shared content helpers Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently drops all text on the /v1/responses path. AIM turns it into a loud 422 ( {"error":"No messages in the request"}); every other guardrail (Lakera v2, Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret detection) scans an empty payload and lets the request through unscanned. Three defects, all in _content_utils.py: 1. _iter_text_parts_in_content recognised only part.type == "text", but the Responses API uses input_text (request) and output_text (assistant). 2. _coerce_input_to_messages gated on "every item has a role key"; any Responses input list containing a function_call or function_call_output item failed the check and was wrapped as one opaque blob. 3. build_inspection_messages forwarded any role through, including a bare tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze reject with a schema error. Fix walks the actual Responses item taxonomy (message, function_call, function_call_output, bare content parts and strings), recognises {text, input_text, output_text} everywhere, and coerces any role outside {system, user, assistant} to user in the outbound inspection payload. * style: ruff-format changed guardrail files * test(guardrails): cover function_call_output string form; drop em-dash in new docstring * fix(guardrails): map function_call_output straight to user role Avoids ever materialising a schema-invalid bare tool message. The downstream role-safety coercion in build_inspection_messages still guards genuinely caller-supplied non-standard roles (developer, function, custom values); add a regression test covering that path so the coercion has real coverage after this simplification. * test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages * docs(test): soften AIM-specific claims in LIT-4294 test docstrings Ryan's review flagged that several test docstrings assert AIM's /fw/v1/analyze validates + rejects specific schema violations. That behavior is customer-reported in the LIT-4294 writeup, not directly verified by us. Rephrase to attribute the AIM 422 to the customer's writeup and describe the underlying constraint as the OpenAI chat schema; any downstream API that validates against that schema rejects the same shape. * refactor(guardrails): move unsupported-role coercion into AIM only The generic coercion in build_inspection_messages collapsed any role outside {system, user, assistant} to user for every caller of the helper. Combined with the pre-existing apply_redacted_messages_back write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400 on chat-completions tool-message masking into a silent semantic corruption of the outbound request (role tool with tool_call_id got rewritten to bare role user, dropping the assistant + tool_calls sibling). AIM specifically requires the coercion because its /fw/v1/analyze validates the payload against the OpenAI chat schema; other guardrails either do not validate roles or do their own reconstruction. Move the coercion to AimGuardrail._build_aim_inspection_messages so the shared helper keeps caller roles intact and no new cross-guardrail role corruption is introduced. The pre-existing apply_redacted_messages_back structural flatten remains as separate follow-up work. function_call_output items still synthesise role user in the shared helper because they have no natural role field, which is a different concern from coercing a caller-supplied role. * refactor(guardrails): preserve role fidelity in shared _content_utils Shared inspection helpers should extract text and preserve semantic role signals; role coercion for third-party schema safety stays inside the guardrail that needs it (AIM). Three shared-helper changes: - Bare content-part dicts (input_text/output_text) with an explicit role keep it; only role-less parts default to user. - Responses message items already had their role preserved; the behavior is now covered by an explicit test. - function_call_output items default to role tool (semantic equivalent of the chat-completions tool message shape) instead of role user, so Responses and chat completions produce symmetric inspection payloads. A caller-supplied role on the item is still preserved. AIM's schema-safe coercion in _build_aim_inspection_messages already handles the resulting role tool: it collapses to user before the POST to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the bare tool message (no tool_call_id can survive the flatten). Added a regression test in test_aim.py covering that path. (cherry picked from commit e84a19a)
Greptile SummaryThis backport fixes a gap in the shared guardrail content helpers (
Confidence Score: 5/5Safe to merge — the change is a well-scoped bug fix that restores parity between the chat completions and Responses-API code paths in the guardrail helpers. No schema, dependency, or interface changes. The two modified source files have straightforward, well-isolated logic changes backed by 14 new tests that directly exercise each fixed path. The formatting-only reflow on the stable branch's Black gate introduces no behavioural divergence from the staging commit. The AIM role-coercion wrapper is additive and does not alter the upstream LLM request. No pre-existing tests were weakened, and the targeted guardrail test suite went from 27 to 41 passed. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/_content_utils.py | Core fix: adds TEXT_PART_TYPES frozenset, rewrites _coerce_input_to_messages to walk Responses-API items individually, and updates walk_user_text to handle message/function_call_output shapes. Logic is correct and well-tested. |
| litellm/proxy/guardrails/guardrail_hooks/aim/aim.py | Adds _build_aim_inspection_messages static method that coerces non-standard roles (e.g. "tool") to "user" before POSTing to AIM's /fw/v1/analyze endpoint. Both call sites updated correctly. |
| tests/test_litellm/proxy/guardrails/test_content_utils.py | Adds 10 new regression tests covering input_text/output_text parts, function_call_output traversal, and role-preservation in the shared helpers. Two existing assertions are reflowed to single lines (cosmetic). No test coverage is weakened. |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_aim.py | New file with 4 mock-only unit tests for _build_aim_inspection_messages covering tool-role coercion, non-standard roles, function_call_output, and safe-role pass-through. No real network calls. |
| pyproject.toml | Version bump from 1.90.3 to 1.90.4 in both [project] and [tool.commitizen] sections. |
Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.90.4" | Re-trigger Greptile
Relevant issues
Backports #32542 onto
stable/1.90.xand cuts1.90.4. On the/v1/responsespath the shared guardrail content helpers inlitellm/proxy/guardrails/_content_utils.pyextracted no text, so every guardrail built on those helpers (AIM, Lakera v2, Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret detection) inspected an empty payload and passed the request through without scanning or redacting. Chat completions were unaffected; the gap was specific to the Responses input taxonomy. This restores parity so those helpers walk the Responsesinputshapes (text/input_text/output_textparts, andmessage/function_call/function_call_outputitems) the same way they walk chat messagesLinear ticket
Resolves LIT-4294
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
What is included
e84a19acd5), cherry-picked with-x1.90.3->1.90.4uv.lockfor1.90.4Adaptation notes
The pick is content-identical to the staging commit; the only divergence is formatting.
stable/1.90.xgatesformat-checkon Black 26.3.1 at 88 columns (cd litellm && black --check .), while staging authored the change withruff formatat 120 columns. The two touched source files were reflowed with the line's own Black so they passlint-black; no logic changed. The test files are outside the line's Black scope and were taken as-isKnown noise on this line
The targeted guardrail suite is green on this line both before and after the pick, so there is no pre-existing red to discount.
tests/test_litellm/proxy/guardrails/test_content_utils.pywas 27 passed / 0 failed at the line tip before the pick; after the pick the two touched test files run 41 passed / 0 failedScreenshots / Proof of Fix
Live proxy on
stable/1.90.x,/v1/responsesrouted to a real Anthropic model through the responses bridge. A custompre_callguardrail logs what the shared helpers extracted (iter_message_textfragments,build_inspection_messagesroles) and callswalk_user_textto redact the markerAKIAEXAMPLE, mirroring the reproducer in #32542. Same request body in both runsRequest:
Before (line tip
2d28d8fadf, unpicked), the helpers see nothing and nothing is redacted:After (pick
342d8897b6), both the user text and the tool-output text are redacted and role fidelity is preserved (function_call_output->role: tool), whilefunction_callmetadata (call_id,name,arguments) is untouched:Both runs returned HTTP 200 from the real Anthropic call with the redacted input. Targeted tests on the line: 41 passed / 0 failed, versus a 27 passed / 0 failed baseline (the AIM test file is new in this pick). A
deepadversarial pass over the shared-helper callers (AIM, Cato, IBM, Lakera v2, Lasso, Repello, Azure Content Safety) found no existing caller broken by the behavior changeType
🐛 Bug Fix
Changes
Cherry-pick of #32542 onto
stable/1.90.x, reflowed to the line's Black formatter, plus the1.90.4version bump and lock refresh. No schema, dependency, or configuration changes