Add claims agent guardrails (5 categories + policy template)

[ghost]

Summary

• Add 5 new content filter guardrail categories for healthcare claims agent chatbots covering the highest-signal failure cases: fraud coaching, PHI disclosure without auth, prior-auth gaming, system override injection, and medical advice
• Add claims_agent_safety.yaml policy template that combines all 5 into a single deployable policy
• Add 243-case eval dataset (5 JSONL files, ~50 cases each with balanced BLOCK/ALLOW split) and 5 test classes in test_eval.py

Guardrails added

Category	Failure Cases Covered	Test Cases	Precision	Recall
claims_fraud_coaching	Fraud coaching, exaggeration, document forgery	50	100%	100%
claims_phi_disclosure	PHI disclosure without auth, bulk data extraction	50	100%	100%
claims_prior_auth_gaming	Prior-auth gaming, code manipulation	49	100%	100%
claims_system_override	System override injection, employer/third-party access	45	100%	100%
claims_medical_advice	Medical advice (claims-context-aware)	49	100%	100%

Each uses 3-layer detection: conditional matching (identifier + block word in same sentence), always-block keywords, and regex phrase patterns — with exceptions to prevent false positives on legitimate claims operations.

Test plan

• All 5 claims eval suites pass at 100% precision/recall/F1
• All 7 existing eval suites still pass at 100% (no regressions)
• Run: pytest litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/test_eval.py -k "Claims" -v -s

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 25, 2026 7:11pm

[greptile-apps]

Greptile Summary

Adds 5 new content filter guardrail categories for healthcare claims agent chatbots (claims_fraud_coaching, claims_phi_disclosure, claims_prior_auth_gaming, claims_system_override, claims_medical_advice), a policy template entry in policy_templates.json, a 243-case eval dataset with 100% precision/recall across all categories, and a 315-prompt UI compliance dataset for claims assistant testing.

• 5 YAML category files added to categories/ using the established 3-layer detection pattern (conditional matching, always-block keywords, regex phrase patterns) with well-designed exception lists to minimize false positives on legitimate claims operations
• 5 JSONL eval files (243 total test cases) with balanced BLOCK/ALLOW splits; all pass at 100% precision/recall/F1 with sub-millisecond latency
• 5 test classes added to test_eval.py following the existing pattern exactly
• Policy template added to policy_templates.json pointing directly to the 5 individual category files (the previously proposed combined claims_agent_safety.yaml template was removed in a subsequent commit)
• UI compliance prompts added in claimsCompliancePrompts.ts with 315 prompts across 17 categories (comment inaccurately says 250)
• Existing benchmark results re-run with no regressions; newly added results for 6 previously un-recorded categories (age, disability, gender, military, religion discrimination)
• No changes to the core content filter engine or critical request path

Confidence Score: 4/5

• This PR is safe to merge — it adds new YAML config files, test data, and UI code with no changes to the content filter engine or critical request path.
• Score of 4 reflects that: (1) all changes are additive — new YAML categories, eval data, test classes, and UI prompts with no modifications to core logic; (2) all 5 new eval suites pass at 100% precision/recall and existing suites show no regressions; (3) the only concern is that some always-block keywords (e.g., "without restrictions", "claims records") in claims_system_override.yaml may be broad enough to cause false positives on edge cases outside the eval dataset, but this is a tuning concern rather than a correctness bug.
• claims_system_override.yaml has some broad always-block keywords worth monitoring in production. claimsCompliancePrompts.ts has a minor inaccuracy in the prompt count comment.

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/categories/claims_fraud_coaching.yaml	New guardrail category for detecting claims fraud coaching. Well-structured with 3-layer detection (conditional, always-block, regex), plus exceptions for fraud prevention/detection contexts. 50/50 eval split with 100% precision/recall.
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/categories/claims_medical_advice.yaml	New guardrail for blocking medical advice in claims context. Allows legitimate claims-related medical term usage (code lookups, coverage) while blocking diagnosis/treatment recommendations. Comprehensive exception list for insurance terminology.
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/categories/claims_phi_disclosure.yaml	New guardrail for detecting unauthorized PHI disclosure and bulk data extraction. Allows self-service ("my own", "my claim") and aggregate data. Well-designed exception list for HIPAA educational content.
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/categories/claims_prior_auth_gaming.yaml	New guardrail for detecting prior authorization gaming (code manipulation, medical necessity misrepresentation). Good exceptions for legitimate prior-auth inquiries (status checks, appeals, documentation).
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/categories/claims_system_override.yaml	New guardrail for system override injection and role impersonation. Some always-block keywords like "without restrictions" and "claims records" are broad and could cause false positives outside the eval dataset, though the eval suite passes 100%.
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/guardrail_benchmarks/test_eval.py	Adds 5 new test classes following the existing pattern exactly. Each class loads its JSONL, creates a content filter, and runs the confusion matrix. Consistent with existing test structure.
policy_templates.json	Adds the "claims-agent-safety" policy template entry with 5 guardrail definitions. Points directly to category YAML files (not combined template). Correct structure matching existing patterns.
ui/litellm-dashboard/src/data/claimsCompliancePrompts.ts	Adds 315 compliance prompts across 17 categories for claims assistant testing. Comment says "250 prompts" but actual count is 315. Pattern follows existing codeExecutionCompliancePrompts.ts structure.
ui/litellm-dashboard/src/data/compliancePrompts.ts	Adds import and spread of claimsCompliancePrompts, plus new "Claims Assistant" framework metadata entry. Minimal, correct changes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User Input Text] --> B{Exception Match?}
    B -->|Yes| C[ALLOW]
    B -->|No| D{Conditional Match?<br/>identifier + block word<br/>in same sentence}
    D -->|Yes| E[BLOCK]
    D -->|No| F{Phrase Pattern<br/>Regex Match?}
    F -->|Yes| E
    F -->|No| G{Always-Block<br/>Keyword Match?}
    G -->|Yes| E
    G -->|No| H{Regular Keyword<br/>Match?}
    H -->|Yes| I{Category Exception?}
    I -->|Yes| C
    I -->|No| E
    H -->|No| C

    subgraph "New Claims Categories"
        J[claims_fraud_coaching]
        K[claims_phi_disclosure]
        L[claims_prior_auth_gaming]
        M[claims_system_override]
        N[claims_medical_advice]
    end

    D -.->|each category checked| J
    D -.->|each category checked| K
    D -.->|each category checked| L
    D -.->|each category checked| M
    D -.->|each category checked| N

Loading

_{Last reviewed commit: 4e9936a}

[greptile-apps]

_{12 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Duplicate exception entry

"file an appeal" is listed twice in the exceptions list (lines 123 and 124). While this won't cause a runtime error (duplicate strings in a list are harmless), it looks like a copy-paste artifact.

Suggested change

	- "file an appeal"
	- "file an appeal"
	- "my prior authorization was denied"
	- "file an appeal"
	- "my prior authorization was denied"

[greptile-apps]

Combined template is missing ~50% of entries from individual category files

The combined claims_agent_safety.yaml policy template is documented as combining all 5 categories into "a single deployable policy", but it is significantly stripped down compared to the sum of the individual files:

• always_block_keywords: 64 entries missing (e.g., "fraudulent workers comp", "stage a car accident", "gets automatically approved", "auto-approved", "skip prior authorization", "all claimants", "all member records", "ignore the coverage", "disable the auto-deny", "diagnose what's wrong", "need surgery", "indicate cancer", and 52 more)
• exceptions: 51 entries missing (e.g., fraud prevention variants like "anti-fraud", "fraud investigation", PHI self-service like "my own medical", "send my claim documents", prior-auth terms like "urgent", "emergency services", "retroactive")
• additional_block_words: 38 entries missing (e.g., "ADHD", "ibuprofen", "herniated disc", "pre-authorization", "preauthorization", "billing details", "pregnancy-related")
• identifier_words: ~28 entries missing (all -ing gerund forms like "exaggerating", "fabricating", "forging", etc., plus "access", "duplicate claim", "alter")
• phrase_patterns: 6 of 11 patterns missing (including both prior-auth gaming regex patterns)

The missing always_block_keywords means the combined template will fail to catch many explicitly harmful phrases that the individual files would block. The missing exceptions means the combined template will produce more false positives on legitimate queries (e.g., HIPAA education, fraud reporting, prior-auth process inquiries). If this policy template is intended as a convenience wrapper with the same detection behavior, it needs to be expanded. If it's intentionally a lighter-weight subset, that should be documented clearly.

[ghost]

@greptile please re-review this

[greptile-apps]

_{26 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Inaccurate prompt count in comment

The file header says "250 prompts" but the actual total from the Array.from({ length: N }) calls sums to 315 prompts (30+20+20+20+25+30+25+20+20+20+10+10+20+10+15+10+10). Consider updating the comment to match.

Suggested change

	* Claims Assistant Security + UX Dataset (250 prompts)
	* Claims Assistant Security + UX Dataset (315 prompts)