Add Lakera v2 post-call hook and tests (fixed PII masking) - #21783
Conversation
…er, test location, mask order - PII masking path: return ModelResponse instead of dict so deployment hook accepts it - Avoid mutating request data: deep copy original_messages and messages in _mask_pii_in_messages - Add guardrail header in PII-only return path - Add test in tests/test_litellm/ (test_lakera_ai_v2.py) per PR checklist - Sort PII payload spans by (start,end) descending so multiple spans in one message mask correctly Co-authored-by: Cursor <cursoragent@cursor.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile SummaryThis PR adds a post-call guardrail hook (
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/lakera_ai_v2.py | Adds async_post_call_success_hook with PII masking, blocking/monitoring, and proper ModelResponse return. Fixes mutation bug in _mask_pii_in_messages with copy.deepcopy, adds reverse-sorted mask application. Addresses previous review feedback for index mismatch and on_flagged consistency. |
| tests/guardrails_tests/test_lakera_v2.py | Adds three well-structured post-call hook tests: block on flagged content, allow clean content, and PII masking with ModelResponse validation. All tests use mocks appropriately. |
| tests/test_litellm/proxy/guardrails/guardrail_hooks/test_lakera_ai_v2.py | New unit test file in the required tests/test_litellm/ directory, validates that PII masking returns ModelResponse (not dict) for parent hook compatibility. Mock-only, no network calls. |
Sequence Diagram
sequenceDiagram
participant Proxy as LiteLLM Proxy
participant Hook as async_post_call_success_deployment_hook
participant Lakera as LakeraAIGuardrail.async_post_call_success_hook
participant API as Lakera API v2/guard
Proxy->>Hook: response from LLM
Hook->>Hook: should_run_guardrail(post_call)
Hook->>Lakera: async_post_call_success_hook(data, response)
Lakera->>Lakera: Extract assistant messages from response
Lakera->>Lakera: Build post_call_messages (user + assistant)
Lakera->>API: call_v2_guard(post_call_messages)
API-->>Lakera: LakeraAIResponse (flagged, payload, breakdown)
alt Not flagged
Lakera-->>Hook: return original response
else Flagged: PII only
Lakera->>Lakera: _mask_pii_in_messages()
Lakera->>Lakera: Write masked content back to response_dict
Lakera-->>Hook: return ModelResponse(**response_dict)
else Flagged: on_flagged == "block"
Lakera--xHook: raise HTTPException(400)
else Flagged: on_flagged == "monitor"
Lakera->>Lakera: Log warning
Lakera-->>Hook: return original response
end
Hook->>Hook: _is_valid_response_type(result)
Hook-->>Proxy: return result or original response
Last reviewed commit: 33257c3
…nd inconsistent on_flagged access pattern
|
|
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
|
@eurogig can you please fix the linting error? |
Review1. Does this PR fix the issue it describes? 2. Has this issue already been solved elsewhere? 3. Are there other PRs addressing the same problem? 4. Are there other issues this potentially closes? ✅ LGTM — all checklist items passed, Greptile-reviewed. |
* Add post-call hook for Lakera guardrail and mask PII in responses * Add post-call hook for Lakera and mask PII in responses * Fix post-call hook: pass event_type to call_v2_guard * Address Greptile review: return ModelResponse, fix mutation, add header, test location, mask order - PII masking path: return ModelResponse instead of dict so deployment hook accepts it - Avoid mutating request data: deep copy original_messages and messages in _mask_pii_in_messages - Add guardrail header in PII-only return path - Add test in tests/test_litellm/ (test_lakera_ai_v2.py) per PR checklist - Sort PII payload spans by (start,end) descending so multiple spans in one message mask correctly Co-authored-by: Cursor <cursoragent@cursor.com> * Updated ponteital for index mismatch when choices have null content and inconsistent on_flagged access pattern * Update litellm/proxy/guardrails/guardrail_hooks/lakera_ai_v2.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update to explicitly state supported endpoints - chat completions * Fix minor lint error on masked_entity_count --------- Co-authored-by: Steve <steve.giguere@lakera.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…1783) * Add post-call hook for Lakera guardrail and mask PII in responses * Add post-call hook for Lakera and mask PII in responses * Fix post-call hook: pass event_type to call_v2_guard * Address Greptile review: return ModelResponse, fix mutation, add header, test location, mask order - PII masking path: return ModelResponse instead of dict so deployment hook accepts it - Avoid mutating request data: deep copy original_messages and messages in _mask_pii_in_messages - Add guardrail header in PII-only return path - Add test in tests/test_litellm/ (test_lakera_ai_v2.py) per PR checklist - Sort PII payload spans by (start,end) descending so multiple spans in one message mask correctly Co-authored-by: Cursor <cursoragent@cursor.com> * Updated ponteital for index mismatch when choices have null content and inconsistent on_flagged access pattern * Update litellm/proxy/guardrails/guardrail_hooks/lakera_ai_v2.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update to explicitly state supported endpoints - chat completions * Fix minor lint error on masked_entity_count --------- Co-authored-by: Steve <steve.giguere@lakera.ai> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Relevant issues
[Feature]: Add missing post_call hook to Lakera guardrails #18016
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
Changes
Add Lakera v2 post-call hook and tests (fixed PII masking found by Greptile)