chore(release): backport #32542, #32655 to stable/1.91.x and cut 1.91.3 - #32948
Conversation
…elpers (#32542) * fix(guardrails): walk Responses-API text taxonomy in shared content helpers Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently drops all text on the /v1/responses path. AIM turns it into a loud 422 ( {"error":"No messages in the request"}); every other guardrail (Lakera v2, Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret detection) scans an empty payload and lets the request through unscanned. Three defects, all in _content_utils.py: 1. _iter_text_parts_in_content recognised only part.type == "text", but the Responses API uses input_text (request) and output_text (assistant). 2. _coerce_input_to_messages gated on "every item has a role key"; any Responses input list containing a function_call or function_call_output item failed the check and was wrapped as one opaque blob. 3. build_inspection_messages forwarded any role through, including a bare tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze reject with a schema error. Fix walks the actual Responses item taxonomy (message, function_call, function_call_output, bare content parts and strings), recognises {text, input_text, output_text} everywhere, and coerces any role outside {system, user, assistant} to user in the outbound inspection payload. * style: ruff-format changed guardrail files * test(guardrails): cover function_call_output string form; drop em-dash in new docstring * fix(guardrails): map function_call_output straight to user role Avoids ever materialising a schema-invalid bare tool message. The downstream role-safety coercion in build_inspection_messages still guards genuinely caller-supplied non-standard roles (developer, function, custom values); add a regression test covering that path so the coercion has real coverage after this simplification. * test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages * docs(test): soften AIM-specific claims in LIT-4294 test docstrings Ryan's review flagged that several test docstrings assert AIM's /fw/v1/analyze validates + rejects specific schema violations. That behavior is customer-reported in the LIT-4294 writeup, not directly verified by us. Rephrase to attribute the AIM 422 to the customer's writeup and describe the underlying constraint as the OpenAI chat schema; any downstream API that validates against that schema rejects the same shape. * refactor(guardrails): move unsupported-role coercion into AIM only The generic coercion in build_inspection_messages collapsed any role outside {system, user, assistant} to user for every caller of the helper. Combined with the pre-existing apply_redacted_messages_back write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400 on chat-completions tool-message masking into a silent semantic corruption of the outbound request (role tool with tool_call_id got rewritten to bare role user, dropping the assistant + tool_calls sibling). AIM specifically requires the coercion because its /fw/v1/analyze validates the payload against the OpenAI chat schema; other guardrails either do not validate roles or do their own reconstruction. Move the coercion to AimGuardrail._build_aim_inspection_messages so the shared helper keeps caller roles intact and no new cross-guardrail role corruption is introduced. The pre-existing apply_redacted_messages_back structural flatten remains as separate follow-up work. function_call_output items still synthesise role user in the shared helper because they have no natural role field, which is a different concern from coercing a caller-supplied role. * refactor(guardrails): preserve role fidelity in shared _content_utils Shared inspection helpers should extract text and preserve semantic role signals; role coercion for third-party schema safety stays inside the guardrail that needs it (AIM). Three shared-helper changes: - Bare content-part dicts (input_text/output_text) with an explicit role keep it; only role-less parts default to user. - Responses message items already had their role preserved; the behavior is now covered by an explicit test. - function_call_output items default to role tool (semantic equivalent of the chat-completions tool message shape) instead of role user, so Responses and chat completions produce symmetric inspection payloads. A caller-supplied role on the item is still preserved. AIM's schema-safe coercion in _build_aim_inspection_messages already handles the resulting role tool: it collapses to user before the POST to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the bare tool message (no tool_call_id can survive the flatten). Added a regression test in test_aim.py covering that path. (cherry picked from commit e84a19a)
…d LLM calls (#32655) * feat(otel): emit the gen_ai.client.operation.exception event on failed LLM calls The GenAI semantic conventions record failures of a GenAI client operation as a log-based event named gen_ai.client.operation.exception, carrying the exception.type / exception.message / exception.stacktrace trio at severity WARN and correlated to the failed span. OTel v2 never emitted it: a failed LLM call produced only the deprecated error.* span attributes, a generic exception span event without a stacktrace, and the stacktrace under the vendor key litellm.provider.error.stack_trace. Build the logs pipeline (LoggerProvider + console/OTLP log exporters mirroring the metrics plumbing) and record the event behind the enable_events flag, which until now was defined but consumed nowhere. An operator-configured LoggerProvider global is reused so the events ride their existing logs pipeline; an explicit NoOpLoggerProvider global is honored as an opt-out and builds no recorder at all. The existing span-side error surface (error.type, error.message, the exception span event, and the litellm.provider.error.* detail keys) is untouched for backwards compatibility. * fix(otel): always ride the semconv-required exception pair on the GenAI event Filtering the event attributes on truthiness conflated "absent" with "empty", so an empty exception.type or exception.message would have been dropped, leaving an event with neither semconv-required field. Build the attributes so the pair is unconditional and only the recommended stacktrace is omitted when the payload carries none. * docs(otel): document the events plumbing module in the package README * test(otel): cover the log exporter selection and logs endpoint normalization The new logs plumbing had no coverage for exporter-kind selection, the console fallback for an unrecognized kind, the /v1/logs signal-path rewriting that lets one OTEL_ENDPOINT serve every signal, or the simple-vs-batch processor split. (cherry picked from commit 99b4c5e)
|
|
Greptile SummaryThis backport cut (
Confidence Score: 4/5Safe to merge; changes are well-scoped cherry-picks with comprehensive unit-test coverage and no modifications to auth, schema, or dependency versions. Both cherry-picks apply cleanly and the import adaptation described in the PR notes (dropping the unused No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/_content_utils.py | Extends TEXT_PART_TYPES to cover Responses-API part types and rewrites _coerce_input_to_messages / walk_user_text to walk the full Responses taxonomy (message items, function_call, function_call_output); logic is correct and well-tested |
| litellm/proxy/guardrails/guardrail_hooks/aim/aim.py | Adds _build_aim_inspection_messages to coerce non-standard roles to 'user' before the AIM POST; uses direct m['role'] access which is safe given build_inspection_messages always returns role-keyed dicts |
| litellm/integrations/otel/plumbing/events.py | New file; implements GenAIEventRecorder emitting gen_ai.client.operation.exception log events at WARN severity with the required exception.* trio; straightforward and correct |
| litellm/integrations/otel/plumbing/providers.py | Adds log exporter/provider factories (_otlp_logs_endpoint, build_log_exporter, build_logger_provider, resolve_logger_provider, get_event_logger) mirroring the existing metrics pattern; uses opentelemetry.sdk._logs private API which is pinned at 1.28.0 |
| litellm/integrations/otel/emitter.py | Wires optional GenAIEventRecorder into SpanEmitter; event emission is correctly guarded on both event_recorder presence and SpanRole.LLM_CALL |
| litellm/integrations/otel/logger.py | Strengthens logger_provider type from Any to LoggerProvider and adds _init_events helper that gates event recorder construction on enable_events config; clean DI pattern |
| litellm/integrations/otel/model/semconv.py | Adds STACKTRACE constant to ExceptionEvent and new GenAIEvent class with OPERATION_EXCEPTION constant; pinned constant values match the semconv spec |
| tests/test_litellm/proxy/guardrails/test_content_utils.py | Adds comprehensive new tests for Responses-API content-part types and function_call_output taxonomy; existing test assertions unchanged (only whitespace reformatting) |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_aim.py | New test file; covers role coercion in _build_aim_inspection_messages for tool, developer, and function_call_output inputs; unit tests only, no network calls |
| tests/test_litellm/integrations/otel/test_otel_v2_components.py | New tests for log exporter variants, logger-provider selection, and operation exception event emission; all use in-memory exporters, no network calls |
| tests/test_litellm/integrations/otel/test_otel_v2_logger.py | New integration tests verify enable_events wires through the full failure-callback path and that events are off by default; clean additions |
| pyproject.toml | Version bump 1.91.2 → 1.91.3 in both [project] and [tool.commitizen] blocks |
Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.91.3" | Re-trigger Greptile
Relevant issues
Backports two already-merged
litellm_internal_stagingchanges ontostable/1.91.xand cuts1.91.3. The tip1.91.2is already published, so this line needs the patch bump#32542 fixes a Responses-API bypass in the shared guardrail content helpers: every guardrail that reads
litellm/proxy/guardrails/_content_utils.pysaw zero text on the/v1/responsespath, so AIM turned it into a 422 and every other content guardrail scanned an empty payload and let the request through unscanned. #32655 adds the GenAI-semconvgen_ai.client.operation.exceptionlog event so a failed LLM call is recorded withexception.type/exception.message/exception.stacktraceat WARN, which OTel v2 never emitted beforeLinear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewWhat is included
In staging merge order, then the release bump:
e84a19acd5)gen_ai.client.operation.exceptionevent on failed LLM calls (cherry-picked from99b4c5ed3e)bump: version 1.91.2 -> 1.91.3chore: refresh uv.lock for 1.91.3Adaptation notes
#32542 applied verbatim (patch-id identical to the staging commit)
#32655 needed one import resolution in
litellm/integrations/otel/logger.py. Staging importsfrom opentelemetry.context import Context, attach, get_current; this line droppedContextbecause it predates staging's identity-baggage and MCP-list-tools work that referenced it, so the pick keeps the line'sfrom opentelemetry.context import attach, get_currentand adds the fix's own newfrom opentelemetry.sdk._logs import LoggerProviderverbatim. KeepingContextwould be an unused-import failure on this line. Nothing else diverges; the OTel dependency pins are identical between the line and staging (opentelemetry-sdk==1.28.0,opentelemetry-api==1.28.0), andOpenTelemetryV2Config.enable_eventsalready exists on the lineKnown noise on this line
None. The pre-pick baseline of the touched test files was 131 passed, 0 failures, so any red on those files is introduced by this PR and should be treated as a real signal
Screenshots / Proof of Fix
Verified end-to-end against a live proxy running this branch's source, hitting real provider APIs. The extraction proof uses a small custom
pre_callguardrail that logs what_content_utilsextracted (iter_message_textfragments,build_inspection_messagesroles,walk_user_textvisit count); the OTel proof runs theotelcallback withLITELLM_OTEL_V2=true,LITELLM_OTEL_INTEGRATION_ENABLE_EVENTS=true, and the default console log exporter, then fails a call with a deliberately-bogus OpenAI key so OpenAI returns a genuine 401 that maps toAuthenticationError(nothing mocked)#32542 -
/v1/responsesbefore vs after (same curl)Before (pre-pick, the bug - every text guardrail is a no-op on this path):
After (this branch - the Responses taxonomy is walked, and
function_call_outputsurfaces asrole: "tool"):/v1/chat/completionswas the control and reportedfragments_seen=1both before and after, so chat scanning is unchanged#32655 - failed call before vs after (same curl)
Both before and after, the call returns a real
401 AuthenticationError. Before, grepping the proxy log for the event returns nothing; after, exactly one record appears:The span-side
error.type/error.message/litellm.provider.error.*keys are still present alongside it, so the existing error surface is untouchedSanity and auth path (this branch)
A real
anthropic/claude-haiku-4-5completion returnspong;/key/generateplus a scoped completion with the generated key both succeedTests
Targeted suite on this line: 158 passed, 0 failures (
test_content_utils.py,test_aim.py,test_otel_v2_components.py,test_otel_v2_logger.py), up from the 131-pass / 0-fail baseline by the tests these PRs add. As a regression check for existing callers of the modified helpers (build_inspection_messages,iter_message_text,walk_user_text, theSpanEmitterconstructor), the fulltests/test_litellm/proxy/guardrails/andtests/test_litellm/integrations/otel/trees pass 2371, 0 failures on this branchType
🆕 New Feature
🐛 Bug Fix
Changes
Two cherry-picks onto
stable/1.91.xplus the1.91.3version bump and lock refresh. No dependency, schema, migration, auth-default, CI, or UI changes; the diff is code and tests only