fix(proxy): Bedrock guardrail spend logs - hook mode, match redaction, streaming request_data by milan-berri · Pull Request #25854 · BerriAI/litellm

milan-berri · 2026-04-16T11:43:26Z

Relevant issues

Fixes:

LIT-2407: incorrect guardrail_mode / spend-log labels for Bedrock when proxy hooks are during_call / post_call but Bedrock uses INPUT/OUTPUT
LIT-2405: sensitive match / regex values in spend/compliance metadata,

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit (CI will confirm; locally: test_bedrock_guardrails.py, test_custom_guardrail.py, test_core_helpers.py, test_litellm/proxy/test_proxy_utils.py, and targeted proxy_unit_tests/test_proxy_utils.py guardrail/during tests were run green.)
My PR's scope is as isolated as possible, it only solves 1 specific problem (bundles related proxy logging fixes; can split if maintainers prefer.)
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Screenshots / Proof of Fix

Hook mode: Spend / evaluation UI should show PRE-CALL / DURING-CALL / POST-CALL aligned with proxy hooks, not inferred only from Bedrock INPUT/OUTPUT.

Match redaction: Compliance exports / metadata should not contain raw Bedrock match spans; "[REDACTED]" where applicable.

Type

🐛 Bug Fix
✅ Test

Changes

Problem

Wrong guardrail_mode in spend logs: Bedrock ApplyGuardrail INPUT/OUTPUT was used to infer logging hook phase, so during_call (and some post_call) runs could appear as PRE-CALL in spend/UI.
Sensitive match / regex in logs: Raw match-like fields could appear in standard guardrail logging (spend/compliance) and related paths.

Fix

logging_event_type on make_bedrock_api_request — When provided, drives event_type for add_standard_logging_guardrail_information_to_request_data. If omitted, keep legacy mapping from source.
BedrockGuardrail.use_native_during_call_hook — during_call uses native async_moderation_hook so spend logs record during_call instead of unified apply_guardrail always looking like pre_call. ProxyLogging / _execute_guardrail_hook respect the flag (CustomGuardrail default False).
Centralized redaction — redact_nested_match_and_regex_keys in litellm_core_utils/core_helpers.py; applied in CustomGuardrail.add_standard_logging_guardrail_information_to_request_data. Bedrock passes raw JSON into that path (single redaction pass). Bedrock _redact_pii_matches / HTTP detail["assessments"] helpers delegate to the same logic.

Tests

tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py: hook / logging_event_type / redaction / streaming request_data assertions.
tests/test_litellm/integrations/test_custom_guardrail.py: standard logging redacts match / regex.
tests/test_litellm/litellm_core_utils/test_core_helpers.py: redact_nested_match_and_regex_keys unit tests.

Files (high level)

litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py
litellm/proxy/utils.py
litellm/integrations/custom_guardrail.py
litellm/litellm_core_utils/core_helpers.py
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py
tests/test_litellm/integrations/test_custom_guardrail.py
tests/test_litellm/litellm_core_utils/test_core_helpers.py

…post) Bedrock ApplyGuardrail uses INPUT/OUTPUT for the API body; spend logs must use the proxy hook (pre_call, during_call, post_call). Pass logging_event_type from each hook into make_bedrock_api_request. During-call was still logged as pre_call because unified guardrails call apply_guardrail with input_type=request. BedrockGuardrail now sets use_native_during_call_hook so during_call runs async_moderation_hook instead. Includes a small test asserting the Bedrock class flag. Made-with: Cursor

…outing - Assert make_bedrock_api_request forwards logging_event_type to standard logging and legacy INPUT maps to pre_call when omitted. - Assert during_call_hook invokes Bedrock async_moderation_hook when native path is used. Made-with: Cursor

…k via patch.object Made-with: Cursor

Made-with: Cursor

- Add redact_nested_match_and_regex_keys in core_helpers for nested match/regex. - Apply in CustomGuardrail standard logging; Bedrock forwards raw JSON to avoid double redaction. - Delegate Bedrock HTTP detail assessments and _redact_pii_matches to the same helper. - Extend unit tests (core_helpers, CustomGuardrail, Bedrock spend-log mock). Made-with: Cursor

…logs Greptile: async_post_call_streaming_iterator_hook omitted request_data on OUTPUT make_bedrock_api_request calls, so standard_logging attached to a throwaway dict. - Pass request_data for parallel OUTPUT task and OUTPUT-only branch. - Add unit tests asserting OUTPUT (and parallel INPUT) receive the same request_data. Made-with: Cursor

vercel · 2026-04-16T11:43:32Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 16, 2026 1:00pm

greptile-apps · 2026-04-16T11:52:39Z

Greptile Summary

This PR fixes three Bedrock guardrail logging bugs: (1) incorrect guardrail_mode labels in spend logs when the proxy hook was during_call (Bedrock's INPUT/OUTPUT was incorrectly used to infer the proxy hook phase), (2) raw PII match and regex strings being persisted in spend/compliance metadata, and (3) missing request_data for streaming OUTPUT guardrail logs. The fixes are centralized — a new redact_nested_match_and_regex_keys utility handles all redaction, a new logging_event_type parameter on make_bedrock_api_request propagates the proxy hook phase explicitly, and use_native_during_call_hook = True on BedrockGuardrail routes during_call through async_moderation_hook (which carries the correct phase) instead of the unified apply_guardrail path.

Confidence Score: 5/5

Safe to merge — all three bug fixes are correctly implemented, tests are comprehensive, and no regressions were found in the guardrail logging or redaction paths.

All findings are P2 or lower. The three bugs are fixed with correct logic: explicit logging_event_type propagation replaces the fragile INPUT/OUTPUT inference, use_native_during_call_hook is a clean non-breaking opt-in (default False on CustomGuardrail, only True on BedrockGuardrail), and redact_nested_match_and_regex_keys uses a deep copy so originals are never mutated. Tests cover every new code path including edge cases (malformed payloads, streaming request_data, end-to-end redaction in HTTPException detail).

No files require special attention.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/core_helpers.py	Adds `redact_nested_match_and_regex_keys` — a deep-copy tree-walk that replaces every `match`/`regex` key with "[REDACTED]". Two separate try/except blocks guard against deepcopy and walk failures; correctly returns the original payload on error.
litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py	Adds `use_native_during_call_hook = True` ClassVar, `logging_event_type` parameter to `make_bedrock_api_request`, simplifies `_redact_pii_matches` to delegate to the new utility, and adds `_redact_assessment_match_fields` for customer-visible HTTPException payloads. All call sites pass explicit `logging_event_type` matching the proxy hook phase.
litellm/integrations/custom_guardrail.py	Adds `use_native_during_call_hook: ClassVar[bool] = False` base default and wires `redact_nested_match_and_regex_keys` into `add_standard_logging_guardrail_information_to_request_data` as the single authoritative redaction point for all standard guardrail logging.
litellm/proxy/utils.py	Both `_execute_guardrail_hook` and `during_call_hook` now respect `use_native_during_call_hook`; when True the guardrail's own `async_moderation_hook` is invoked directly, bypassing the unified `apply_guardrail` path that always logged INPUT as pre_call.
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_bedrock_guardrails.py	Comprehensive new unit tests covering: `logging_event_type` forwarding to standard logging, `use_native_during_call_hook` ClassVar, `during_call_hook` invokes `async_moderation_hook`, streaming hook passes `request_data` to OUTPUT calls, and `_get_http_exception_for_blocked_guardrail` emits redacted assessment matches.
tests/test_litellm/integrations/test_custom_guardrail.py	New `TestCustomGuardrailSpendLogMatchRedaction` class verifies that `add_standard_logging_guardrail_information_to_request_data` redacts both `match` and `regex` keys while preserving the original payload.
tests/test_litellm/litellm_core_utils/test_core_helpers.py	New `TestRedactNestedMatchAndRegexKeys` class covers recursive redaction, passthrough for None/string inputs, and deep-copy isolation. Existing finish-reason tests are untouched.

_{Reviews (2): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

greptile-apps · 2026-04-16T11:52:46Z

 def _redact_pii_matches(response_json: dict) -> dict:
-    try:
-        # Create a deep copy to avoid modifying the original response
-        redacted_response = copy.deepcopy(response_json)
-
-        # Get assessments from the response
-        assessments = redacted_response.get("assessments", [])
-        if not assessments:
-            return redacted_response
-
-        for assessment in assessments:
-            # Redact PII entities in sensitive information policy
-            sensitive_info_policy = assessment.get("sensitiveInformationPolicy")
-            if sensitive_info_policy:
-                pii_entities = sensitive_info_policy.get("piiEntities", [])
-                for pii_entity in pii_entities:
-                    if "match" in pii_entity:
-                        pii_entity["match"] = "[REDACTED]"
-
-                # Redact regex matches
-                regexes = sensitive_info_policy.get("regexes", [])
-                for regex_match in regexes:
-                    if "match" in regex_match:
-                        regex_match["match"] = "[REDACTED]"
+    """
+    Redact match-like fields from a Bedrock ApplyGuardrail JSON payload.

-            # Redact custom word matches in word policy
-            word_policy = assessment.get("wordPolicy")
-            if word_policy:
-                custom_words = word_policy.get("customWords", [])
-                for custom_word in custom_words:
-                    if "match" in custom_word:
-                        custom_word["match"] = "[REDACTED]"
-
-                managed_words = word_policy.get("managedWordLists", [])
-                for managed_word in managed_words:
-                    if "match" in managed_word:
-                        managed_word["match"] = "[REDACTED]"
-
-        return redacted_response
+    Delegates to :func:`redact_nested_match_and_regex_keys` (same rules as spend
+    logging). Kept as a Bedrock-module entry point for existing unit tests.
+    """
+    try:
+        redacted = redact_nested_match_and_regex_keys(response_json)
+        return redacted if isinstance(redacted, dict) else response_json
    except Exception as e:
        # We do not want to fail in any case so this is just a warning
        verbose_proxy_logger.warning("Guardrail log redaction failed: %s", str(e))
        return response_json


Redundant outer try/except in _redact_pii_matches

redact_nested_match_and_regex_keys already handles all exceptions internally (two separate try/except blocks guarantee it never propagates), so the outer try/except in _redact_pii_matches and its verbose_proxy_logger.warning call can never be reached. The wrapper can be simplified:

def _redact_pii_matches(response_json: dict) -> dict: """ Redact match-like fields from a Bedrock ApplyGuardrail JSON payload. Delegates to :func:`redact_nested_match_and_regex_keys` (same rules as spend logging). Kept as a Bedrock-module entry point for existing unit tests. """ redacted = redact_nested_match_and_regex_keys(response_json) return redacted if isinstance(redacted, dict) else response_json

…l_spend_logging Resolve Bedrock guardrail conflicts: keep redact_nested_match_and_regex_keys in _redact_pii_matches (staging null-safety loops superseded by tree walk). Retain streaming request_data tests. Address Greptile P2: clarify malformed-response test comment; simplify _redact_pii_matches (drop redundant try/except). Made-with: Cursor

codecov · 2026-04-16T13:02:56Z

Codecov Report

❌ Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/litellm_core_utils/core_helpers.py	91.66%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Restore guardrail spend/UI event_type wiring, request_data on streaming OUTPUT paths, and centralized match redaction after the upstream revert. Made-with: Cursor

milan-berri added 6 commits April 16, 2026 00:31

test(proxy): group Bedrock spend tests and patch async_moderation_hoo…

36afe10

…k via patch.object Made-with: Cursor

style(test): isort imports in test_bedrock_guardrails

2e5982a

Made-with: Cursor

vercel Bot deployed to Preview April 16, 2026 11:45 View deployment

greptile-apps Bot reviewed Apr 16, 2026

View reviewed changes

vercel Bot deployed to Preview April 16, 2026 13:00 View deployment

yuneng-berri changed the base branch from litellm_internal_staging to litellm_yj_apr21 April 21, 2026 19:48

yuneng-berri merged commit 09ff936 into BerriAI:litellm_yj_apr21 Apr 21, 2026
43 of 44 checks passed

Navigation Menu

Uh oh!

Uh oh!

fix(proxy): Bedrock guardrail spend logs - hook mode, match redaction, streaming request_data - #25854

fix(proxy): Bedrock guardrail spend logs - hook mode, match redaction, streaming request_data#25854
yuneng-berri merged 7 commits into
BerriAI:litellm_yj_apr21from
milan-berri:litellm_bedrock_guardrail_spend_logging

milan-berri commented Apr 16, 2026

Uh oh!

vercel Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Apr 16, 2026 •

edited

Loading

Important Files Changed

Uh oh!

greptile-apps Bot Apr 16, 2026

Uh oh!

codecov Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

milan-berri commented Apr 16, 2026

Relevant issues

Pre-Submission checklist

CI (LiteLLM team)

Screenshots / Proof of Fix

Type

Changes

Problem

Fix

Tests

Files (high level)

Uh oh!

vercel Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

greptile-apps Bot Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Apr 16, 2026 •

edited

Loading

greptile-apps Bot commented Apr 16, 2026 •

edited

Loading

codecov Bot commented Apr 16, 2026 •

edited

Loading