Skip to content

fix(guardrails): show YAML-defined guardrails in Guardrail Monitor - #26830

Closed
yuneng-berri wants to merge 3 commits into
litellm_internal_stagingfrom
litellm_guardrailMonitorYamlFix
Closed

fix(guardrails): show YAML-defined guardrails in Guardrail Monitor#26830
yuneng-berri wants to merge 3 commits into
litellm_internal_stagingfrom
litellm_guardrailMonitorYamlFix

Conversation

@yuneng-berri

Copy link
Copy Markdown
Collaborator

Summary

Fixes LIT-2529 — the dashboard's Guardrail Monitor never showed details for guardrails defined in config.yaml.

The three endpoints serving the monitor (/guardrails/usage/overview, /guardrails/usage/detail/{id}, /guardrails/usage/logs) only read from the litellm_guardrailstable Prisma table. YAML-defined guardrails live in IN_MEMORY_GUARDRAIL_HANDLER and never get persisted, so:

  • detail 404'd for any YAML guardrail
  • overview rendered YAML guardrails as provider="Custom" / type="Guardrail" via the orphan-metric fallback (and not at all when they had no metrics)
  • logs silently missed the logical-name alias for YAML guardrails

This PR mirrors the union/fallback pattern already used by list_guardrails_v2 and get_guardrail_info:

  • usage/overview — union DB rows with IN_MEMORY_GUARDRAIL_HANDLER.list_in_memory_guardrails(), deduped by guardrail_id, before building rows.
  • usage/detail/{id} — fall back to IN_MEMORY_GUARDRAIL_HANDLER.get_guardrail_by_id() when the DB lookup misses; only 404 if both miss.
  • usage/logs — same fallback so the YAML guardrail's logical name gets added to effective_guardrail_ids.
  • Added small _get_guardrail_field / _to_dict helpers that uniformly handle Prisma rows, plain dicts, TypedDicts, and LitellmParams Pydantic instances.

Test plan

  • New unit tests in tests/test_litellm/proxy/guardrails/test_usage_endpoints.py (7 tests) — written red first, all pass after the fix
  • pytest tests/test_litellm/proxy/guardrails/1358 passed, no regressions in test_guardrail_endpoints.py / test_guardrail_registry.py / hooks
  • In-process verification with the real IN_MEMORY_GUARDRAIL_HANDLER singleton seeded as if from a YAML load (mocked prisma_client) — overview row, detail response, and logs query all return the YAML guardrail's real metadata
  • black + ruff check clean
  • (Reviewer) Spin up a local proxy with the OTEL test config (which defines YAML guardrails) and confirm the Guardrail Monitor in the Admin UI shows rows + detail panels for them

Reopened from #26648 — the previous branch was missing the litellm_ prefix so CircleCI did not run.

yuneng-berri and others added 2 commits April 27, 2026 15:22
The /guardrails/usage/{overview,detail,logs} endpoints only consulted the
litellm_guardrailstable Prisma table, so guardrails defined in config.yaml
(stored only in IN_MEMORY_GUARDRAIL_HANDLER) were invisible: detail 404'd,
overview rendered them as "Custom"/"Guardrail" via the orphan-metric path or
not at all, and logs missed the logical-name alias.

Mirror the existing list_guardrails_v2 / get_guardrail_info pattern: union DB
rows with IN_MEMORY_GUARDRAIL_HANDLER.list_in_memory_guardrails() (deduped by
guardrail_id) for overview, fall back to get_guardrail_by_id() for detail and
logs. Adds tests covering DB-only, YAML-only, both, and DB-takes-precedence.

Fixes LIT-2529.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address PR review feedback. InMemoryGuardrailHandler.initialize_guardrail was
constructing the parsed Guardrail without guardrail_info, so even after the
endpoint fallback fix, /guardrails/usage/{overview,detail} couldn't render
type or description for YAML-defined guardrails — both fields would silently
default to "Guardrail" / None at runtime.

Pass guardrail_info through into IN_MEMORY_GUARDRAILS, and add a regression
test that exercises the real handler (not a mock) end-to-end.

Also refactors _get_guardrail_attrs to use the new _get_guardrail_field helper
for consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@yuneng-berri

Copy link
Copy Markdown
Collaborator Author

@greptile

@greptile-apps

greptile-apps Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Fixes the Guardrail Monitor never showing YAML-defined guardrails by unioning IN_MEMORY_GUARDRAIL_HANDLER entries with DB rows in all three usage endpoints (overview, detail, logs), with dedup by guardrail_id so DB takes precedence. The _get_guardrail_field / _to_dict helpers cleanly handle the Prisma-row-vs-TypedDict-vs-Pydantic-model split, and the one-line registry change ensures guardrail_info is actually persisted into the in-memory store.

Confidence Score: 4/5

Safe to merge; logic is correct and well-tested with only a minor style suggestion.

No P0 or P1 issues found. The union/dedup/fallback logic is correct — the registry mutates the input dict before constructing the stored TypedDict, so guardrail_id is always non-None. The only finding is a P2 style suggestion on _to_dict being narrowly typed to LitellmParams.

No files require special attention; usage_endpoints.py has the minor _to_dict typing suggestion.

Important Files Changed

Filename Overview
litellm/proxy/guardrails/guardrail_registry.py One-line addition: guardrail_info is now passed when constructing the in-memory Guardrail TypedDict, so detail/overview endpoints can read type and description for YAML guardrails. Change is correct — guardrail["guardrail_id"] is mutated before this line, so the stored TypedDict always has a non-None ID.
litellm/proxy/guardrails/usage_endpoints.py Overview, detail, and logs endpoints now union DB guardrails with IN_MEMORY_GUARDRAIL_HANDLER entries, with dedup and correct fallback ordering. Helper functions _get_guardrail_field / _to_dict cleanly unify access across Prisma rows and TypedDicts/Pydantic models. Minor: _to_dict is narrowly typed to LitellmParams specifically.
tests/test_litellm/proxy/guardrails/test_usage_endpoints.py 7 new unit tests covering all three fixed endpoints, including DB-precedence dedup, orphan-metric suppression, logical-name alias in logs query, and a real-handler integration test. All tests use mocked Prisma/in-memory handlers with no real network calls — consistent with the tests/test_litellm/ no-network rule.

Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile

Comment on lines +154 to +160
def _to_dict(value: Any) -> Dict[str, Any]:
"""Coerce a LitellmParams / dict / None into a plain dict."""
if isinstance(value, LitellmParams):
return value.model_dump(exclude_none=True)
if isinstance(value, dict):
return value
return {}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 _to_dict hardcodes LitellmParams type

_to_dict special-cases LitellmParams specifically, which means any other Pydantic model stored in these fields (e.g. a custom subclass or a future guardrail_info model) silently returns {} rather than its actual data. Using BaseModel from Pydantic as the isinstance check would be more resilient.

Suggested change
def _to_dict(value: Any) -> Dict[str, Any]:
"""Coerce a LitellmParams / dict / None into a plain dict."""
if isinstance(value, LitellmParams):
return value.model_dump(exclude_none=True)
if isinstance(value, dict):
return value
return {}
def _to_dict(value: Any) -> Dict[str, Any]:
"""Coerce a Pydantic BaseModel / dict / None into a plain dict."""
from pydantic import BaseModel as _BaseModel
if isinstance(value, _BaseModel):
return value.model_dump(exclude_none=True)
if isinstance(value, dict):
return value
return {}

@yuneng-berri

Copy link
Copy Markdown
Collaborator Author

Superseded by #32853, which reimplements this fix from scratch against current staging (the code here had drifted to the GuardrailsRepository accessor since this branch was cut). Closing in favor of that PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant