Fix replicate to foreign server without PDPA#21978
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes two issues in Singapore PDPA guardrail policy YAML templates to resolve failing test
Issues found:
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/policy_templates/sg_pdpa_data_transfer.yaml | Adds bare "PDPA" as an identifier_word to fix conditional matching for inputs like "replicate to foreign server without PDPA". The addition is functionally correct but broadens the trigger scope since "PDPA" alone will now match as an identifier in any sentence. |
| litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/policy_templates/sg_pdpa_profiling_automated_decisions.yaml | Changes exception from "human oversight" to "with human oversight" to prevent the exception from incorrectly matching the block word "without human oversight" (since exceptions use substring matching). This is a valid bug fix. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Input Text] --> B{Check Global Exceptions}
B -->|Exception found| C[ALLOW]
B -->|No exception| D{Check Category Exceptions}
D -->|Exception found| C
D -->|No exception| E{Split into Sentences}
E --> F{Sentence contains identifier_word?}
F -->|No| G[Next sentence]
F -->|Yes e.g. 'PDPA'| H{Sentence also contains block_word?}
H -->|No| G
H -->|Yes e.g. 'replicate to'| I[BLOCK - Conditional Match]
G -->|More sentences| F
G -->|No more| J{Check always_block_keywords}
J -->|Match| K[BLOCK - Always Block]
J -->|No match| C
style I fill:#ff6b6b,color:#fff
style K fill:#ff6b6b,color:#fff
style C fill:#51cf66,color:#fff
Last reviewed commit: 044639f
| - "PDPA data" | ||
| - "Singapore employee data" | ||
| # PDPA alone implies Singapore regulatory context | ||
| - "PDPA" |
There was a problem hiding this comment.
Broad identifier may cause false positives
Adding bare "PDPA" as an identifier word means any sentence mentioning "PDPA" alongside any additional_block_words entry will trigger a block. Since identifier matching uses case-insensitive substring matching (if identifier in sentence_lower), this could block legitimate requests like:
- "How do I export data under PDPA?" (matches identifier
"pdpa"+ block word"export") - "Is PDPA relevant when I copy to another department?" (matches
"pdpa"+"copy to")
These seem like informational queries that should be allowed. The existing exceptions ("explain", "what is", "how to comply") won't cover all such cases.
Consider whether a more specific phrase like "PDPA data" (already present at line 17) or "under PDPA" would be sufficient, or add more exceptions to guard against false positives on informational queries.
| - "human in the loop" | ||
| - "human review" | ||
| - "human oversight" | ||
| - "with human oversight" |
There was a problem hiding this comment.
Narrowed exception may miss valid phrasing
Changing from "human oversight" to "with human oversight" correctly prevents the exception from unintentionally matching the block word "without human oversight" (since exceptions use substring matching). However, this also means phrases like "ensure human oversight is maintained" or "human oversight required" will no longer trigger the exception.
Consider additionally listing other common phrasings that convey the same intent, such as:
"human oversight required""ensure human oversight""under human oversight"
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Fix replicate to foreign server without PDPA
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes