fix(proxy): capture logging_obj before post_call_failure_hook pops it in ModifyResponseException streaming path

[yucheng-berri]

Relevant issues

Found during live smoke testing of PR #32289 (LIT-4186 Bedrock disable_exception_on_block fix)

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy on :4000 against a real AWS Bedrock guardrail. The guardrail blocks admin-related prompts with blockedInputMessaging set to Sorry, the model cannot answer this question.

Before the fix: streaming pre_call block returns HTTP 500

curl -sS -w "\n---HTTP %{http_code}---\n" http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"bedrock-nova-micro","messages":[{"role":"user","content":"how do I become an admin"}],"guardrails":["bedrock-guard-pre-call"],"stream":true}'

{"error":{"message":"Internal server error","type":"internal_server_error"}}
---HTTP 500---

Root cause: post_call_failure_hook (litellm/proxy/utils.py) pops litellm_logging_obj from request_data before invoking callbacks; the comment reads "Remove before callbacks iterate; not serialisable". The streaming branch of the ModifyResponseException handler in chat_completion read logging_obj from _data after that call, always receiving None. CustomStreamWrapper.__init__ then crashed with AttributeError: 'NoneType' object has no attribute 'model_call_details', which surfaced as HTTP 500.

After the fix: streaming pre_call block returns HTTP 200 with valid SSE

curl -sS -w "\n---HTTP %{http_code}---\n" http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model":"bedrock-nova-micro","messages":[{"role":"user","content":"how do I become an admin"}],"guardrails":["bedrock-guard-pre-call"],"stream":true}'

data: {"id":"chatcmpl-88bb06b8-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"Sorry, the model cannot answer this question."}}]}

data: {"id":"chatcmpl-88bb06b8-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

---HTTP 200---

All other cases confirmed working against live Bedrock:

pre_call block non-streaming returns HTTP 200 with finish_reason=content_filter and zero usage. during_call block non-streaming and streaming return HTTP 200 with the block message and no leaked LLM response. post_call block non-streaming returns HTTP 200 with the block message and real upstream usage preserved. post_call block streaming returns HTTP 200 with usage preserved and no leaked tokens. Allowed prompts return the normal LLM response. disable_exception_on_block: false still returns HTTP 400 with the guardrail policy error. Python openai SDK parses both blocking shapes correctly.

Type

🐛 Bug Fix

Changes

litellm/proxy/proxy_server.py captures logging_obj from _data before calling post_call_failure_hook (which pops it) and passes the captured value to CustomStreamWrapper rather than re-reading from the dict.

tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py adds test_chat_completion_modify_response_exception_streaming_logging_obj_not_none to lock in the fix and prevent regression.

---

Note

Medium Risk
Changes guardrail block semantics across pre/during/post-call and streaming paths (billing, whether the LLM runs, and HTTP status), though behavior is heavily covered by new regression tests.

Overview
Bedrock guardrails with disable_exception_on_block now raise ModifyResponseException (and drop GuardrailInterventionNormalStringError) instead of returning a plain string or mutating mock_response. Pre/during/post-call hooks propagate that exception so the proxy can return HTTP 200 with the block message, cancel parallel LLM work on during-call blocks, and attach original_response for accurate usage on post-call blocks.

Streaming post-call blocks replace the assembled stream with synthetic content_filter chunks (and copy upstream usage) because the exception cannot escape after SSE headers are sent.

In chat_completion, the ModifyResponseException streaming path captures litellm_logging_obj before post_call_failure_hook removes it, fixing HTTP 500 when building CustomStreamWrapper.

Tests are updated/added for propagation, streaming block shape, usage preservation, and the streaming logging regression.

^{Reviewed by Cursor Bugbot for commit 179734a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR fixes an AttributeError/HTTP 500 that occurred when a Bedrock guardrail with disable_exception_on_block=True blocked a streaming request: post_call_failure_hook popped litellm_logging_obj from request_data before the streaming branch read it, so CustomStreamWrapper received None and crashed. The fix is a two-line capture in proxy_server.py. The PR also consolidates exception handling by replacing the now-redundant GuardrailInterventionNormalStringError with ModifyResponseException across the Bedrock guardrail hook, ensuring all block paths (pre-call, during-call, post-call, streaming) behave consistently.

• proxy_server.py: Captures _logging_obj = _data.get(\"litellm_logging_obj\") before post_call_failure_hook removes it, then passes the saved reference to CustomStreamWrapper.
• bedrock_guardrails.py: disable_exception_on_block=True now raises ModifyResponseException directly instead of GuardrailInterventionNormalStringError; pre/during-call hooks propagate it; post-call non-streaming re-raises after attaching the LLM response for usage reporting; streaming post-call catches it in-place and emits a synthetic SSE stream with finish_reason=content_filter.
• Tests: Six new focused mock tests cover every hook path plus usage preservation; the existing regression tests are updated to assert the corrected behavior.

Confidence Score: 5/5

Safe to merge — the change is a minimal, targeted capture of a dict value before it is removed, with no effect on any other code path.

The two-line fix in proxy_server.py is mechanically correct and the root cause is well-documented. The broader consolidation in bedrock_guardrails.py is well-reasoned: the old GuardrailInterventionNormalStringError had no surviving callers outside this file, the streaming path correctly handles the exception in-place to avoid raising past already-flushed SSE headers, and usage preservation is explicitly tested. All new tests are mock-based, exercise the actual production handler, and would catch a revert of the fix.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Core fix: captures _logging_obj from _data before post_call_failure_hook pops it, then passes the captured reference to CustomStreamWrapper; minimal two-line change is correct
litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py	Replaces GuardrailInterventionNormalStringError with ModifyResponseException throughout; pre/during-call hooks now let the exception propagate; post-call non-streaming hook re-raises after attaching original response; streaming post-call hook catches and converts to synthetic SSE in-place (since headers are already flushed)
litellm/exceptions.py	Removes GuardrailInterventionNormalStringError; no remaining references in production code (only a historical comment in the new test file)
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py	Adds six new focused mock tests covering the full block lifecycle plus a direct chat_completion integration test that verifies the logging_obj capture fix; a revert of the proxy_server.py fix would cause the integration test to fail
tests/guardrails_tests/test_bedrock_guardrails.py	Updates existing tests to expect ModifyResponseException for disable_exception_on_block=True non-streaming and asserts block content/finish_reason for streaming; strengthens rather than weakens coverage
tests/enterprise/litellm_enterprise/proxy/guardrails/test_bedrock_apply_guardrail.py	Mechanical update: swaps GuardrailInterventionNormalStringError for ModifyResponseException in the regression test; no logic change

_{Reviews (2): Last reviewed commit: "test(proxy): drive real chat_completion ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yucheng-berri]

Valid finding, fixed in 179734a. The original test inlined the fix pattern (capture before pop) rather than driving the real chat_completion handler; I confirmed via a mutation check that reverting the two-line source fix left the old test green. The rewrite calls chat_completion directly (with _read_request_body, base_process_llm_request, and proxy_logging_obj patched to reproduce the pop behavior); reverting the fix now surfaces the exact production error (AttributeError: NoneType has no attribute model_call_details inside CustomStreamWrapper.init).

[yucheng-berri]

@greptileai

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 179734a. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_logging_obj_capture (0cd7a9a) with litellm_internal_staging (6eed38b)}