Skip to content

feat: guardrail tracing UI - policy, detection method, match details#21349

Merged
ishaan-jaff merged 14 commits intomainfrom
litellm_feat/guardrail-tracing-ui
Feb 17, 2026
Merged

feat: guardrail tracing UI - policy, detection method, match details#21349
ishaan-jaff merged 14 commits intomainfrom
litellm_feat/guardrail-tracing-ui

Conversation

@ishaan-jaff
Copy link
Copy Markdown
Contributor

@ishaan-jaff ishaan-jaff commented Feb 16, 2026

Summary

  • Adds GuardrailTracingDetail TypedDict with fields: guardrail_id, policy_template, detection_method, confidence_score, classification, match_details, patterns_checked, alert_recipients
  • Content filter populates these fields from detection results
  • UI shows policy template badges, detection method chips, match details table, classification panel, and execution timeline when multiple guardrails run
  • All fields optional — no impact on existing guardrail providers
Screenshot 2026-02-16 at 6 13 30 PM

Test plan

  • 4 unit tests for base class tracing fields (set, default absent, multiple policies, classification)
  • 5 e2e tests for content filter tracing (pattern detection, fallback ID, category keywords, blocked request, no detections)
  • 74 total tests passing

@vercel
Copy link
Copy Markdown

vercel bot commented Feb 16, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 17, 2026 2:39am

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Feb 16, 2026

Greptile Summary

This PR adds guardrail tracing metadata to the logging pipeline and renders it in the dashboard UI. A new GuardrailTracingDetail TypedDict carries optional fields (guardrail_id, policy_template, detection_method, confidence_score, classification, match_details, patterns_checked, alert_recipients) through CustomGuardrail.add_standard_logging_guardrail_information_to_request_data() into StandardLoggingGuardrailInformation. The content filter guardrail populates these fields from its detection results, and the GuardrailViewer UI component renders them as policy badges, detection method chips, match details tables, classification panels, and an execution timeline.

  • Adds GuardrailTracingDetail TypedDict and extends StandardLoggingGuardrailInformation with 8 new optional fields in litellm/types/utils.py
  • CustomGuardrail base class accepts tracing_detail param and unpacks it into the SLG dict — backward compatible via **(tracing_detail or {})
  • Content filter guardrail populates tracing fields from its detection results with four new helper methods
  • UI adds PolicyDetectionRow, MatchDetailsTable, ClassificationDetails, and ExecutionTimeline components
  • 9 new tests (4 unit + 5 E2E) covering the tracing fields pipeline, all using local mocks with no network calls

Confidence Score: 4/5

  • This PR is safe to merge — all new fields are optional and backward-compatible, with no impact on the critical request path.
  • Score of 4 reflects a well-structured, additive feature with proper tests. All new TypedDict fields use total=False (optional), the base class change is backward-compatible via default None, and the content filter changes are self-contained. The UI components have proper null guards. Minor deduction for the PolicyDetectionRow showing when only patterns_checked is 0 and the field duplication between GuardrailTracingDetail and StandardLoggingGuardrailInformation.
  • litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/content_filter.py has the most new logic with tracing field population helpers. ui/litellm-dashboard/src/components/view_logs/GuardrailViewer/GuardrailViewer.tsx adds substantial UI rendering code.

Important Files Changed

Filename Overview
litellm/types/utils.py Adds new optional tracing fields to StandardLoggingGuardrailInformation (guardrail_id, policy_template, detection_method, etc.) and a new GuardrailTracingDetail TypedDict. All fields are optional with total=False, so no impact on existing consumers.
litellm/types/guardrails.py Adds optional policy_template field to the Guardrail TypedDict. Minimal change, no impact on existing code.
litellm/integrations/custom_guardrail.py Adds tracing_detail parameter to add_standard_logging_guardrail_information_to_request_data() and unpacks it via **(tracing_detail or {}) into the SLG dict. Clean, backward-compatible change.
litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/content_filter.py Adds tracing detail population: _build_match_details, _get_detection_methods, _get_patterns_checked_count, _get_policy_templates helper methods. Passes GuardrailTracingDetail to the logging method. Minor issue: two datetime.now() calls in _log_guardrail_information (pre-existing).
ui/litellm-dashboard/src/components/view_logs/GuardrailViewer/GuardrailViewer.tsx Adds five new UI components: PolicyDetectionRow, MatchDetailsTable, ClassificationDetails, ExecutionTimeline, and policy template badges in the header. Well-structured with proper null guards. Minor concern: hasData shows row even when only patterns_checked is 0.
tests/test_litellm/integrations/test_custom_guardrail.py Adds 4 unit tests for tracing fields: set, default absent, multiple policies, classification. All tests use mocks, no network calls.
tests/test_litellm/proxy/guardrails/guardrail_hooks/content_filter/test_content_filter.py Adds 5 E2E tests for content filter tracing: pattern detection, fallback ID, category keywords, blocked request, no detections. All tests use local guardrail instances, no network calls.

Sequence Diagram

sequenceDiagram
    participant CF as ContentFilterGuardrail
    participant CG as CustomGuardrail (base)
    participant SLG as StandardLoggingGuardrailInformation
    participant UI as GuardrailViewer (UI)

    CF->>CF: apply_guardrail()
    CF->>CF: _filter_single_text() → detections
    CF->>CF: _build_match_details(detections)
    CF->>CF: _get_detection_methods(detections)
    CF->>CF: _get_patterns_checked_count()
    CF->>CF: _get_policy_templates()
    CF->>CF: Build GuardrailTracingDetail
    CF->>CG: add_standard_logging_guardrail_information_to_request_data(tracing_detail)
    CG->>SLG: Create SLG with **(tracing_detail or {})
    CG->>CG: Append SLG to request_data["metadata"]
    Note over SLG: Fields: guardrail_id, policy_template,<br/>detection_method, match_details,<br/>patterns_checked, confidence_score, etc.
    SLG-->>UI: Logged data rendered in dashboard
    UI->>UI: PolicyDetectionRow (badges, chips)
    UI->>UI: MatchDetailsTable (per-detection table)
    UI->>UI: ClassificationDetails (LLM-judge panel)
    UI->>UI: ExecutionTimeline (multi-guardrail view)
Loading

Last reviewed commit: d648e0f

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines +2648 to +2663
class GuardrailTracingDetail(TypedDict, total=False):
"""
Typed fields for guardrail tracing metadata.

Passed to add_standard_logging_guardrail_information_to_request_data()
to enrich the StandardLoggingGuardrailInformation with provider-specific details.
"""

guardrail_id: Optional[str]
policy_template: Optional[str]
detection_method: Optional[str]
confidence_score: Optional[float]
classification: Optional[dict]
match_details: Optional[List[dict]]
patterns_checked: Optional[int]
alert_recipients: Optional[List[str]]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated field definitions across TypedDicts

GuardrailTracingDetail duplicates all 8 field definitions from StandardLoggingGuardrailInformation (lines 2623-2645). If a field is added, renamed, or has its type changed on one but not the other, the **(tracing_detail or {}) unpacking in custom_guardrail.py will silently drop the mismatched field.

Consider deriving one from the other or extracting a shared base, e.g.:

class _GuardrailTracingFields(TypedDict, total=False):
    guardrail_id: Optional[str]
    policy_template: Optional[str]
    detection_method: Optional[str]
    confidence_score: Optional[float]
    classification: Optional[dict]
    match_details: Optional[List[dict]]
    patterns_checked: Optional[int]
    alert_recipients: Optional[List[str]]

# Then use _GuardrailTracingFields as a base or as GuardrailTracingDetail directly
GuardrailTracingDetail = _GuardrailTracingFields

Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.
…l-tracing-ui

# Conflicts:
#	litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/content_filter.py
#	litellm/types/utils.py
#	ui/litellm-dashboard/src/components/view_logs/GuardrailViewer/GuardrailViewer.tsx
@ishaan-jaff ishaan-jaff merged commit 0f96b5a into main Feb 17, 2026
33 of 81 checks passed
ghost pushed a commit that referenced this pull request Feb 17, 2026
…ION_PARAMS (#21360)

* allow filtering by user in global usage

* add server root path test to github actions

* Update .github/workflows/test_server_root_path.yml

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* address greptile review feedback (greploop iteration 1)

- Fix HTTPException swallowed by broad except block in get_user_daily_activity
  and get_user_daily_activity_aggregated: re-raise HTTPException before the
  generic handler so 403 status codes propagate correctly
- Add status_code assertions in non-admin access tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* address greptile review feedback (greploop iteration 2)

- Default user_id to caller's own ID for non-admins instead of 403 when
  omitted, preserving backward compatibility for API consumers
- Apply same fix to aggregated endpoint
- Update test to verify defaulting behavior instead of expecting 403
- Add useEffect to sync selectedUserId when auth state settles in
  UsagePageView to handle async auth initialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fixing syntax

* remove artifacts

* feat: guardrail tracing UI - policy, detection method, match details (#21349)

* feat: add GuardrailTracingDetail TypedDict and tracing fields to StandardLoggingGuardrailInformation

* feat: add policy_template field to Guardrail config TypedDict

* feat: accept GuardrailTracingDetail in base guardrail logging method

* feat: populate tracing fields in content filter guardrail

* test: add tracing fields tests for custom guardrail base class

* test: add tracing fields e2e tests for content filter guardrail

* feat: add guardrail tracing UI - policy badges, match details, timeline

* feat: redesign GuardrailViewer to Guardrails & Policy Compliance layout

Two-column layout with request lifecycle timeline on the left
and compact evaluation detail cards on the right. Header shows
guardrail count, pass/fail status, total overhead, policy info,
and an export button.

* feat: add clickable guardrail link in metrics + show policy names

* feat: add risk_score field to StandardLoggingGuardrailInformation

* feat: compute risk_score in content filter guardrail

* feat: display backend risk_score badge on evaluation cards

* fix: fallback to frontend risk score when backend doesn't provide one

* passing in masster key for api calls

* Fix: Add blog as incident report

* Fix: Add blog as incident report

* remove timeline

* feat(models): add github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast (#21316)

Add missing GitHub Copilot model entries for gpt-5.3-codex (GA) and
claude-opus-4.6-fast (Public Preview) to both the root and backup
model pricing JSON files.

* only tests for /ui

* bump: version 1.81.12 → 1.81.13

* Fixing mapped tests

* fixing no_config test

* fixing container tests

* fixing test_basic_openai_responses_api

* Adding bedrock thinking budget tokens to docs

* fixing regen key tests

* fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS

Add store, prompt_cache_key, prompt_cache_retention, safety_identifier, and verbosity
to OPENAI_CHAT_COMPLETION_PARAMS list.

These params were already in DEFAULT_CHAT_COMPLETION_PARAM_VALUES but missing from
the OPENAI_CHAT_COMPLETION_PARAMS list, causing them to be dropped when passed to
OpenAI-compatible providers.

---------

Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
Co-authored-by: Cesar Garcia <128240629+Chesars@users.noreply.github.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant