Skip to content

[Docs] Add docs guide for using policies #20914

Merged
ishaan-jaff merged 20 commits intomainfrom
litellm_docs_policies
Feb 11, 2026
Merged

[Docs] Add docs guide for using policies #20914
ishaan-jaff merged 20 commits intomainfrom
litellm_docs_policies

Conversation

@ishaan-jaff
Copy link
Contributor

[Docs] Add docs guide for using policies

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

ishaan-jaff and others added 18 commits February 10, 2026 16:28
- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Feb 11, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 11, 2026 2:54am

Request Review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 11, 2026

Greptile Overview

Greptile Summary

This PR adds tag-based policy scoping, a policy resolve/debug API, an attachment impact estimation ("blast radius") API, a Policy Simulator UI tab, and comprehensive documentation with UI screenshots.

  • Tag-based scoping: Policies can now target keys/teams by metadata.tags or request-body tags, with wildcard support (health-*). Tags use opt-in semantics — empty tags means "don't filter by tags" (unlike teams/keys/models which default to match-all).
  • /policies/resolve endpoint: Debug which policies and guardrails apply for a given team/key/model/tags context.
  • /policies/attachments/estimate-impact endpoint: Preview how many keys/teams an attachment would affect before creating it.
  • Policy Simulator UI: New tab in the Policies panel to simulate policy resolution with team/key/model/tag inputs.
  • Impact preview: Both inline (in attachment creation form) and popover (in attachment table) blast-radius previews.
  • x-litellm-policy-sources header: New response header showing why each policy matched (e.g., hipaa=tag:healthcare; baseline=scope:*).
  • Refactor: Module-level singleton constants in policy_endpoints.py replaced with lazy getter calls; _initialized flag set after DB sync in registry.
  • Schema: tags column added to LiteLLM_PolicyAttachmentTable across all three schema.prisma files.
  • Tests: Unit tests added for tag matching, wildcard patterns, combined AND logic, and match attribution — all mock-based with no network calls.

Confidence Score: 4/5

  • This PR is safe to merge — the new endpoints are admin-only, tag matching logic is well-tested, and critical path changes are minimal.
  • Score of 4 reflects: well-structured code with good test coverage for the core matching logic, consistent schema changes across all three prisma files, no changes to critical request path beyond adding tag context and attribution metadata. Deducted one point because the estimate-impact endpoint fetches up to 1000 rows unfiltered for tag-based lookups (acceptable for admin endpoint but worth noting), and the new resolve endpoint has no dedicated integration/API tests.
  • Pay attention to litellm/proxy/policy_engine/policy_resolve_endpoints.py — the estimate-impact endpoint does broad DB queries (up to 1000 rows with where={}) for tag-based lookups.

Important Files Changed

Filename Overview
litellm/proxy/policy_engine/policy_resolve_endpoints.py New file: two admin endpoints — /policies/resolve (debug policy matching) and /policies/attachments/estimate-impact (blast radius preview). Tag-based impact fetches up to 1000 keys/teams unfiltered then filters in Python.
litellm/proxy/policy_engine/attachment_registry.py Added tags field propagation throughout attachment CRUD and get_attached_policies_with_reasons() method for match attribution. Clean refactor of get_attached_policies to delegate to the new method.
litellm/proxy/policy_engine/policy_matcher.py Added tag-based scope matching with opt-in semantics (empty tags = don't check, specified tags = ALL must match). Correct AND logic with other scope dimensions.
litellm/proxy/litellm_pre_call_utils.py Updated add_guardrails_from_policy_engine to extract request tags, pass them in PolicyMatchContext, and track policy attribution for the x-litellm-policy-sources header.
litellm/proxy/common_utils/callback_utils.py Added add_policy_sources_to_metadata helper and x-litellm-policy-sources response header generation using ; delimiter.
litellm/types/proxy/policy_engine/policy_types.py Added tags field to PolicyScope and PolicyAttachment with opt-in semantics (empty = don't check, unlike teams/keys/models which default to match-all).
litellm/types/proxy/policy_engine/resolver_types.py Added tags to PolicyMatchContext, PolicyScopeResponse, PolicyAttachmentCreateRequest, PolicyAttachmentDBResponse. New types: PolicyResolveRequest, PolicyMatchDetail, PolicyResolveResponse, AttachmentImpactResponse.
tests/test_litellm/proxy/policy_engine/test_attachment_registry.py Added comprehensive unit tests for tag-based matching (exact, wildcard, combined with teams) and match attribution. All mock-based, no network calls.
tests/test_litellm/proxy/policy_engine/test_policy_matcher.py Added unit tests for PolicyMatcher.scope_matches with tags: exact, wildcard, no-match, combined AND logic. All mock-based.
ui/litellm-dashboard/src/components/policies/policy_test_panel.tsx New Policy Simulator UI panel: form with team/key/model/tags selectors, calls /policies/resolve, displays matched policies and effective guardrails. Uses Tremor Button which is deprecated per AGENTS.md except for tables.
ui/litellm-dashboard/src/components/policies/add_attachment_form.tsx Added tags field to attachment creation form, impact estimation preview, and refactored form data building into shared buildAttachmentData utility. Fixed response parsing for teams/keys/models lists.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Proxy as LiteLLM Proxy
    participant PolicyEngine as Policy Engine
    participant DB as Database

    Note over Client,DB: Policy Resolve (Debug) Flow
    Client->>Proxy: POST /policies/resolve {team, key, model, tags}
    Proxy->>PolicyEngine: get_attached_policies_with_reasons(context)
    PolicyEngine->>PolicyEngine: scope_matches() + tag matching
    PolicyEngine-->>Proxy: matched policies + reasons
    Proxy->>PolicyEngine: resolve_policy_guardrails(each policy)
    Proxy-->>Client: {effective_guardrails, matched_policies}

    Note over Client,DB: Impact Estimation Flow
    Client->>Proxy: POST /policies/attachments/estimate-impact
    Proxy->>DB: find_many(keys, teams) [up to 1000]
    DB-->>Proxy: key/team rows
    Proxy->>PolicyEngine: filter by tag/alias patterns
    Proxy-->>Client: {affected_keys_count, affected_teams_count, samples}

    Note over Client,DB: Request-time Policy Matching (updated)
    Client->>Proxy: POST /chat/completions {model, metadata.tags}
    Proxy->>PolicyEngine: get_attached_policies_with_reasons(context+tags)
    PolicyEngine-->>Proxy: policies + match reasons
    Proxy->>Proxy: add_policy_sources_to_metadata()
    Proxy-->>Client: response + x-litellm-policy-sources header
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

29 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines +369 to +372
keys = await prisma_client.db.litellm_verificationtoken.find_many( # type: ignore
where={}, order={"created_at": "desc"},
take=MAX_POLICY_ESTIMATE_IMPACT_ROWS,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tag-based impact estimation fetches up to 1000 keys with where={} (no filter), then filters in Python. This is fine for an admin endpoint, but if the number of keys grows large, consider adding a DB-side filter on metadata containing the tag to reduce the fetch size.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@ishaan-jaff ishaan-jaff merged commit 3407006 into main Feb 11, 2026
7 of 11 checks passed
ghost pushed a commit that referenced this pull request Feb 11, 2026
…e OpenAI (#20883)

* feat: add opus 4.5 and 4.6 to use outout_format param

* generate poetry lock with 2.3.2 poetry

* restore poetry lock

* e2e tests, key delete, update tpm rpm, and regenerate

* Split e2e ui testing for browser

* new login with sso button in login page

* option to hide usage indicator

* fix(cloudzero): update CBF field mappings per LIT-1907 (#20906)

* fix(cloudzero): update CBF field mappings per LIT-1907

Phase 1 field updates for CloudZero integration:

ADD/UPDATE:
- resource/account: Send concat(api_key_alias, '|', api_key_prefix)
- resource/service: Send model_group instead of service_type
- resource/usage_family: Send provider instead of hardcoded 'llm-usage'
- action/operation: NEW - Send team_id
- resource/id: Send model name instead of CZRN
- resource/tag:organization_alias: Add if exists
- resource/tag:project_alias: Add if exists
- resource/tag:user_alias: Add if exists

REMOVE:
- resource/tag:total_tokens: Removed
- resource/tag:team_id: Removed (team_id now in action/operation)

Fixes LIT-1907

* Update litellm/integrations/cloudzero/transform.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: define api_key_alias variable, update CBFRecord docstring

- Fix F821 lint error: api_key_alias was used but not defined
- Update CBFRecord docstring to reflect LIT-1907 field mappings
- Remove unused Optional import

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Add banner notifying of breaking change

* Add semgrep & Fix OOMs (#20912)

* [Feat] Policies - Allow connecting Policies to Tags, Simulating Policies, Viewing how many keys, teams it applies on  (#20904)

* init schema with TAGS

* ui: add policy test

* resolvePoliciesCall

* add_policy_sources_to_metadata + headers

* types Policy

* preview Impact

* def _describe_match_reason(

* match based on TAGs

* TestTagBasedAttachments

* test fixes

* add policy_resolve_router

* add_guardrails_from_policy_engine

* TestMatchAttribution

* refactor

* fix

* fix: address Greptile review feedback on policy resolve endpoints

- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: eliminate duplicate DB queries and fix header delimiter ambiguity

- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: type error & better error handling (#20689)

* [Docs] Add docs guide for using policies  (#20914)

* init schema with TAGS

* ui: add policy test

* resolvePoliciesCall

* add_policy_sources_to_metadata + headers

* types Policy

* preview Impact

* def _describe_match_reason(

* match based on TAGs

* TestTagBasedAttachments

* test fixes

* add policy_resolve_router

* add_guardrails_from_policy_engine

* TestMatchAttribution

* refactor

* fix

* fix: address Greptile review feedback on policy resolve endpoints

- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: eliminate duplicate DB queries and fix header delimiter ambiguity

- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs v1 guide with UI imgs

* docs fix

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add dashscope/qwen3-max model with tiered pricing (#20919)

Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix linting

* docs: add Greptile review requirement to PR template (#20762)

* fix(azure): preserve content_policy_violation error details from Azure OpenAI

Closes #20811

Azure OpenAI returns rich error payloads for content policy violations
(inner_error with ResponsibleAIPolicyViolation, content_filter_results,
revised_prompt). Previously these details were lost when:

1. The top-level error code was not "content_policy_violation" but the
   inner_error.code was "ResponsibleAIPolicyViolation" -- the structured
   check only examined the top-level code.

2. The DALL-E image generation polling path stringified the error JSON
   into the message field instead of setting the structured body, making
   it impossible for exception_type() to extract error details.

3. The string-based fallback detector used "invalid_request_error" as a
   content-policy indicator, which is too broad and could misclassify
   regular bad-request errors.

Changes:
- exception_mapping_utils.py: Check inner_error.code for
  ResponsibleAIPolicyViolation when top-level code is not
  content_policy_violation. Replace overly broad "invalid_request_error"
  string match with specific Azure safety-system messages.
- azure.py: Set structured body on AzureOpenAIError in both async and
  sync DALL-E polling paths so exception_type() can inspect error details.
- test_azure_exception_mapping.py: Add regression tests covering the
  exact error payloads from issue #20811.
- Fix pre-existing lint: duplicate PerplexityResponsesConfig dict key,
  unused RouteChecks top-level import.

---------

Co-authored-by: Kelvin Tran <kelvin-tran@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com>
Co-authored-by: ken <122603020@qq.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant