fix(proxy): run post_call guardrails on /v1/messages streaming via unified guardrail translation

[mateo-berri]

TLDR

Problem this solves:

• post_call guardrails never scan /v1/messages streaming responses
• blocked output streams to the client in full
• the guardrail then logs a success entry, so the audit trail is wrong

How it solves it:

• /v1/messages streams route apply_guardrail guardrails through the unified guardrail's Anthropic translation
• rerouted guardrails withhold chunks until moderation passes, so blocked content never reaches the client
• guardrails that mask response content keep their own stream hook, which handles raw Anthropic bytes safely

Relevant issues

Fixes #35257

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Config used for both runs: a litellm_content_filter guardrail with mode: post_call, default_on: true, and blocked word zebra, plus anthropic/claude-sonnet-5 in the model list. All calls hit the real Anthropic and OpenAI APIs

Before, at commit 71b825a (base litellm_internal_staging, proxy on port 48731). The blocked word streams through in full and the stream completes normally

$ curl -s -N http://localhost:48731/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model":"anthropic-sonnet-5","max_tokens":50,"stream":true,"messages":[{"role":"user","content":"Reply with exactly: the zebra runs"}]}'
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"the"}}
event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" zebra runs"}   }
event: content_block_stop
...
event: message_stop
data: {"type":"message_stop"               }

The spend log for that request records the post_call guardrail as "guardrail_status": "success" with risk_score 0 even though the blocked keyword was delivered

After, at commit 23f3e10 (this branch's head, proxy on port 45951). Nothing is delivered before moderation; the client receives only the block

$ curl -s -N http://localhost:45951/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model":"anthropic-sonnet-5","max_tokens":50,"stream":true,"messages":[{"role":"user","content":"Reply with exactly: the zebra runs"}]}'
data: {"error": {"message": "Content blocked: keyword 'zebra' detected", "type": "None", "param": "None", "code": "400", "provider_specific_fields": {"error": "Content blocked: keyword 'zebra' detected", "keyword": "zebra", "description": null, "guardrail_name": "output-filter", "guardrail_mode": "post_call"}}}

data: [DONE]

Clean requests still stream end to end at the same commit

$ curl -s -N http://localhost:45951/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model":"anthropic-sonnet-5","max_tokens":50,"stream":true,"messages":[{"role":"user","content":"Say hello in one word"}]}'
event: message_start
...
event: message_stop
data: {"type":"message_stop"           }

Unchanged behavior checks at 23f3e10: non-streaming /v1/messages with the blocked word still returns HTTP 400, and /v1/chat/completions streaming still withholds the blocked word through the guardrail's own iterator hook (zero occurrences of the keyword in the delivered stream)

The non Anthropic provider path through /v1/messages behaves the same at 23f3e10: {"model":"gpt-4o-mini", ...} streaming with the blocked word delivers only the block error frame, and a clean request streams the adapter's Anthropic format events end to end

Checkpoint	Route	Result
before fix (71b825a)	/v1/messages stream	FAIL, blocked word delivered
before fix (71b825a)	audit log	FAIL, success recorded
after fix (23f3e10)	/v1/messages stream	PASS, only block error delivered
after fix (23f3e10)	/v1/messages clean stream	PASS, full stream
after fix (23f3e10)	/v1/messages non-streaming	PASS, HTTP 400
after fix (23f3e10)	/chat/completions stream	PASS, word withheld
after fix (23f3e10)	/v1/messages stream, openai model	PASS, only block error delivered

Type

🐛 Bug Fix

Changes

ProxyLogging.async_post_call_streaming_iterator_hook (litellm/proxy/utils.py) now checks whether the request route streams a non OpenAI wire format (today /v1/messages, which streams raw Anthropic SSE bytes). On such routes, a guardrail that defines both its own async_post_call_streaming_iterator_hook and apply_guardrail is dispatched through unified_guardrail, whose AnthropicMessagesHandler translation already parses Anthropic SSE, instead of through its own hook, which only understands OpenAI ModelResponseStream chunks and passed raw bytes through unscanned (litellm_content_filter) or crashed assembling them (bedrock style hooks). This mirrors how _execute_guardrail_hook already prefers the unified path for non streaming hooks, which is why non streaming /v1/messages blocking worked all along

Because those guardrails' own iterator hooks withheld content until it was scanned, the rerouted invocation defaults streaming_buffer_until_moderated to true, so chunks are withheld until end of stream moderation passes. This trades time to first byte for the guarantee that blocked content never reaches the client, matching the contract the guardrail's own hook provided on /chat/completions. A guardrail can opt back into sampling based scanning by explicitly setting streaming_buffer_until_moderated: false. Guardrails that were already dispatched through the unified path keep their existing default

unified_guardrail.async_post_call_streaming_iterator_hook now accepts the guardrail as an explicit guardrail_to_apply parameter instead of only reading it from a shared request_data key. With the shared key, chaining two unified routed guardrails made the last assignment win and earlier guardrails silently became passthrough

Two dispatch details landed from review feedback in 23f3e10. The reroute predicate uses uses_apply_guardrail_interface() instead of a leaf class __dict__ check, so an apply_guardrail implementation inherited from a vendor base class is recognized and rerouted rather than left on the raw byte path unscanned. And guardrails with mask_response_content enabled are excluded from the reroute and keep their own iterator hook: the unified streaming path cannot re-emit rewritten text on raw Anthropic SSE (its block_only mode drops rewrites and buffered replay would release the unredacted originals), while such guardrails' own hooks already handle that stream shape safely, for example PANW Prisma AIRS parses the raw bytes itself and blocks instead of masking there. Without the exclusion, a masking guardrail's rewrite would be dropped and the original unmasked content delivered

OpenAI format streaming routes are untouched: guardrails with their own iterator hooks keep running them there, preserving incremental masking

Tests are in tests/test_litellm/proxy/test_proxy_logging_hook_detection.py. The new anthropic stream test fails on the base commit by delivering the blocked word without raising, the route detection test pins the dispatch predicate including unmapped routes, the chat completions test proves the own hook path still masks (which the unified block_only path never does, so it fails if dispatch over reroutes), and the explicit parameter test fails on the base signature. The review fix tests fail on the previous head: the inherited apply_guardrail test delivers the blocked word when the predicate only checks the leaf class, and the masking guardrail test observes the own hook being bypassed when mask_response_content is not excluded

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR updates post-call streaming guardrail dispatch for Anthropic Messages streams

• Routes inherited apply_guardrail implementations through unified Anthropic translation
• Passes each guardrail explicitly and buffers rerouted streams until moderation by default
• Preserves native iterator hooks for masking guardrails and OpenAI-format streams
• Adds regression coverage for blocking, masking, route detection, inherited implementations, and explicit guardrail dispatch

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains; the previously reported inherited-guardrail dispatch issue is fixed because the new interface check recognizes implementations inherited from intermediate subclasses, and the regression test exercises that case

Important Files Changed

Filename	Overview
litellm/proxy/utils.py	Detects non-OpenAI stream routes and reroutes applicable guardrails through unified translation while preserving masking guardrails' native hooks
litellm/proxy/guardrails/guardrail_hooks/unified_guardrail/unified_guardrail.py	Accepts an explicit guardrail instance and a caller-selected default for buffering until moderation
tests/test_litellm/proxy/test_proxy_logging_hook_detection.py	Adds focused regression tests covering Anthropic stream blocking, inherited apply-guardrail detection, explicit dispatch, and masking-path preservation

_{Reviews (2): Last reviewed commit: "fix(proxy): recognize inherited apply_gu..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_messages_streaming_post_call_guardrails (23f3e10) with litellm_internal_staging (4eecf7a)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (fb79a4e) during the generation of this report, so 4eecf7a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai