Skip to content
Closed
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,17 @@ LiteLLM is a unified interface for 100+ LLMs that:
- Test provider-specific functionality thoroughly
- Consider adding load tests for performance-critical changes

4. **Code Style**: Follow [Google's Python Style Guide](https://google.github.io/styleguide/pyguide.html) for Python; the project also uses Black, Ruff, and MyPy.

### Performance and Database

- **Hot paths:** Do not add duplicate DB queries in proxy/MCP auth or other high-frequency request paths. Reuse already-loaded auth/context (key, team, end-user objects) instead of re-querying.
- **List endpoints:** Avoid N+1 queries when implementing list endpoints (e.g. agents, MCP servers). Use batch loads, `include`/joins, or a single query with needed relations so DB round-trips stay constant.
- **Pre-merge:** For proxy and MCP code, verify that new DB usage in request-handling paths does not introduce duplicate or N+1 queries.

### Database Migrations
- **Do not auto-create or write `migration.sql` files.** If schema changes are needed or a migration is missing, surface the error to the user and instruct them to run `ci_cd/run_migration.py` to generate new schemas/migrations.

### MAKING CODE CHANGES FOR THE UI (IGNORE FOR BACKEND)

1. **Tremor is DEPRECATED, do not use Tremor components in new features/changes**
Expand Down Expand Up @@ -174,8 +185,11 @@ When opening issues or pull requests, follow these templates:
3. **Rate Limits**: Respect provider rate limits in tests
4. **Memory Usage**: Be mindful of memory usage in streaming scenarios
5. **Dependencies**: Keep dependencies minimal and well-justified
6. **UI/Backend Contract Mismatch**: When adding a new entity type to the UI, always check whether the backend endpoint accepts a single value or an array. Match the UI control accordingly (single-select vs. multi-select) to avoid silently dropping user selections
7. **Missing Tests for New Entity Types**: When adding a new entity type (e.g., in `EntityUsage`, `UsageViewSelect`), always add corresponding tests in the existing test files and update any icon/component mocks
6. **Duplicate DB queries in hot paths**: Adding redundant DB calls in MCP auth, proxy auth, or other high-frequency paths causes performance degradation at scale.
7. **N+1 queries in list endpoints**: Looping over a list and performing a separate DB query per item (e.g. loading agents or related entities) — use batch/joined queries instead.
8. **Auto-creating migration.sql**: Do not generate or edit migration SQL files. If migrations are needed, tell the user to run `ci_cd/run_migration.py` to generate new schemas.
9. **UI/Backend Contract Mismatch**: When adding a new entity type to the UI, always check whether the backend endpoint accepts a single value or an array. Match the UI control accordingly (single-select vs. multi-select) to avoid silently dropping user selections
10. **Missing Tests for New Entity Types**: When adding a new entity type (e.g., in `EntityUsage`, `UsageViewSelect`), always add corresponding tests in the existing test files and update any icon/component mocks

## HELPFUL RESOURCES

Expand All @@ -189,4 +203,5 @@ When opening issues or pull requests, follow these templates:
- Check similar provider implementations
- Ensure comprehensive test coverage
- Update documentation appropriately
- Consider backward compatibility impact
- Consider backward compatibility impact
- For proxy/MCP code, confirm no new duplicate or N+1 DB queries in request paths
10 changes: 8 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,13 @@ LiteLLM is a unified interface for 100+ LLM providers with two main components:
- Async/await patterns throughout
- Type hints required for all public APIs
- **Avoid imports within methods** — place all imports at the top of the file (module-level). Inline imports inside functions/methods make dependencies harder to trace and hurt readability. The only exception is avoiding circular imports where absolutely necessary.
- Follow [Google's Python Style Guide](https://google.github.io/styleguide/pyguide.html) for Python style and structure; the repo uses Black/Ruff/MyPy on top of that.

### Performance and Database

- **Hot paths:** Avoid adding duplicate or redundant DB queries in hot request paths (e.g. proxy auth, MCP request path). Each extra query per request multiplies load at scale. Prefer reusing already-fetched auth/context (e.g. key/team/end-user objects) instead of re-querying.
- **List endpoints and N+1:** When implementing list endpoints (e.g. agents, MCP servers), avoid N+1 patterns: do not loop over a list and perform a separate DB query per item. Use batch loads, `include`/joins, or a single query with the needed relations so the number of DB round-trips is constant.
- **Review before merging:** For proxy and MCP code, check that new DB access in request-handling paths does not introduce duplicate or N+1 queries.

### Testing Strategy
- Unit tests in `tests/test_litellm/`
Expand All @@ -103,8 +110,7 @@ LiteLLM is a unified interface for 100+ LLM providers with two main components:
- When wiring a new UI entity type to an existing backend endpoint, verify the backend API contract (single value vs. array, required vs. optional params) and ensure the UI controls match — e.g., use a single-select dropdown when the backend accepts a single value, not a multi-select

### Database Migrations
- Prisma handles schema migrations
- Migration files auto-generated with `prisma migrate dev`
- Prisma handles schema migrations. **Do not auto-create or write `migration.sql` files.** If schema changes are needed or a migration is missing, surface the error to the user and instruct them to run `ci_cd/run_migration.py` to generate new schemas/migrations.
- Always test migrations against both PostgreSQL and SQLite

### Enterprise Features
Expand Down
88 changes: 49 additions & 39 deletions litellm/proxy/guardrails/guardrail_endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,35 +14,28 @@
from litellm.integrations.custom_guardrail import CustomGuardrail
from litellm.proxy._types import LitellmUserRoles, UserAPIKeyAuth
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
from litellm.proxy.guardrails.guardrail_registry import GuardrailRegistry
from litellm.proxy.guardrails.guardrail_hooks.custom_code.code_validator import (
CustomCodeValidationError,
validate_custom_code,
)
from litellm.proxy.guardrails.guardrail_hooks.custom_code.primitives import (
get_custom_code_primitives,
)
from litellm.proxy.guardrails.usage_endpoints import router as guardrails_usage_router
from litellm.types.guardrails import (
PII_ENTITY_CATEGORIES_MAP,
ApplyGuardrailRequest,
ApplyGuardrailResponse,
BaseLitellmParams,
BedrockGuardrailConfigModel,
Guardrail,
GuardrailEventHooks,
GuardrailInfoResponse,
GuardrailUIAddGuardrailSettings,
LakeraV2GuardrailConfigModel,
ListGuardrailsResponse,
LitellmParams,
PatchGuardrailRequest,
PiiAction,
PiiEntityType,
PresidioPresidioConfigModelUserInterface,
SupportedGuardrailIntegrations,
ToolPermissionGuardrailConfigModel,
)
CustomCodeValidationError, validate_custom_code)
from litellm.proxy.guardrails.guardrail_hooks.custom_code.primitives import \
get_custom_code_primitives
from litellm.proxy.guardrails.guardrail_registry import GuardrailRegistry
from litellm.proxy.guardrails.usage_endpoints import \
router as guardrails_usage_router
from litellm.types.guardrails import (PII_ENTITY_CATEGORIES_MAP,
ApplyGuardrailRequest,
ApplyGuardrailResponse,
BaseLitellmParams,
BedrockGuardrailConfigModel, Guardrail,
GuardrailEventHooks,
GuardrailInfoResponse,
GuardrailUIAddGuardrailSettings,
LakeraV2GuardrailConfigModel,
ListGuardrailsResponse, LitellmParams,
PatchGuardrailRequest, PiiAction,
PiiEntityType,
PresidioPresidioConfigModelUserInterface,
SupportedGuardrailIntegrations,
Comment on lines 17 to +37

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import style reformatted by non-standard formatter

The imports in this file were reformatted from the standard Black-compatible from X import (\n A,\n B,\n) multi-line style to backslash-continuation style (from X import \\\n Y) and compressed grouping. This is inconsistent with the codebase's Black/Ruff formatting conventions (per CLAUDE.md), and the change is unrelated to the feature being added.

These formatting changes add noise to the diff and may conflict with future Black formatting runs. Consider reverting the import reformatting to maintain consistency.

Context Used: Context from dashboard - CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

ToolPermissionGuardrailConfigModel)

#### GUARDRAILS ENDPOINTS ####

Expand Down Expand Up @@ -161,7 +154,8 @@ async def list_guardrails_v2():
```
"""
from litellm.litellm_core_utils.litellm_logging import _get_masked_values
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.guardrails.guardrail_registry import \
IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.proxy_server import prisma_client

if prisma_client is None:
Expand Down Expand Up @@ -303,7 +297,8 @@ async def create_guardrail(
}
```
"""
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.guardrails.guardrail_registry import \
IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.proxy_server import prisma_client

if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
Expand Down Expand Up @@ -401,7 +396,8 @@ async def update_guardrail(
}
```
"""
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.guardrails.guardrail_registry import \
IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.proxy_server import prisma_client

if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
Expand Down Expand Up @@ -477,7 +473,8 @@ async def delete_guardrail(
}
```
"""
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.guardrails.guardrail_registry import \
IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.proxy_server import prisma_client

if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
Expand Down Expand Up @@ -579,7 +576,8 @@ async def patch_guardrail(
}
```
"""
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.guardrails.guardrail_registry import \
IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.proxy_server import prisma_client

if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
Expand Down Expand Up @@ -707,7 +705,8 @@ async def get_guardrail_info(guardrail_id: str):
"""

from litellm.litellm_core_utils.litellm_logging import _get_masked_values
from litellm.proxy.guardrails.guardrail_registry import IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.guardrails.guardrail_registry import \
IN_MEMORY_GUARDRAIL_HANDLER
from litellm.proxy.proxy_server import prisma_client
from litellm.types.guardrails import GUARDRAIL_DEFINITION_LOCATION

Expand Down Expand Up @@ -782,10 +781,8 @@ async def get_guardrail_ui_settings():
- Content filter settings (patterns and categories)
"""
from litellm.proxy.guardrails.guardrail_hooks.litellm_content_filter.patterns import (
PATTERN_CATEGORIES,
get_available_content_categories,
get_pattern_metadata,
)
PATTERN_CATEGORIES, get_available_content_categories,
get_pattern_metadata)

# Convert the PII_ENTITY_CATEGORIES_MAP to the format expected by the UI
category_maps = []
Expand Down Expand Up @@ -1205,11 +1202,22 @@ def _build_field_dict(
# Add options if they exist in json_schema_extra (this takes precedence)
if field_json_schema_extra and "options" in field_json_schema_extra:
field_dict["options"] = field_json_schema_extra["options"]
elif field_type == "select":
# For Literal types, populate options so the UI can render a dropdown
literal_options = _extract_literal_values(field_annotation)
if literal_options:
field_dict["options"] = literal_options

# Add default value if it exists
if field.default is not None and field.default is not ...:
field_dict["default_value"] = field.default

# Copy min, max, step from json_schema_extra for number/percentage inputs
if field_json_schema_extra:
for key in ("min", "max", "step", "default_value"):
if key in field_json_schema_extra:
field_dict[key] = field_json_schema_extra[key]

return field_dict


Expand Down Expand Up @@ -1357,7 +1365,8 @@ async def get_provider_specific_params():
}

### get the config model for the guardrail - go through the registry and get the config model for the guardrail
from litellm.proxy.guardrails.guardrail_registry import guardrail_class_registry
from litellm.proxy.guardrails.guardrail_registry import \
guardrail_class_registry

for guardrail_name, guardrail_class in guardrail_class_registry.items():
guardrail_config_model = guardrail_class.get_config_model()
Expand Down Expand Up @@ -1485,6 +1494,7 @@ async def test_custom_code_guardrail(
```
"""


if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
raise HTTPException(
status_code=403,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
"""Block Code Execution guardrail: blocks or masks fenced code blocks by language."""

from typing import TYPE_CHECKING, Any, List, Literal, Optional, Union, cast

from litellm.types.guardrails import (GuardrailEventHooks,
SupportedGuardrailIntegrations)

from .block_code_execution import BlockCodeExecutionGuardrail

if TYPE_CHECKING:
from litellm.types.guardrails import Guardrail, LitellmParams

# Default: run on both request and response (and during_call is supported too)
DEFAULT_EVENT_HOOKS = [
GuardrailEventHooks.pre_call.value,
GuardrailEventHooks.post_call.value,
]


def _get_param(
litellm_params: "LitellmParams",
guardrail: "Guardrail",
key: str,
default: Any = None,
) -> Any:
"""Get a param from litellm_params, with fallback to raw guardrail litellm_params (for extra fields not on LitellmParams)."""
value = getattr(litellm_params, key, default)
if value is not None:
return value
raw = guardrail.get("litellm_params")
if isinstance(raw, dict) and key in raw:
return raw[key]
return default


def initialize_guardrail(
litellm_params: "LitellmParams",
guardrail: "Guardrail",
) -> BlockCodeExecutionGuardrail:
"""Initialize the Block Code Execution guardrail from config."""
import litellm

guardrail_name = guardrail.get("guardrail_name")
if not guardrail_name:
raise ValueError(
"Block Code Execution guardrail requires a guardrail_name"
)

blocked_languages: Optional[List[str]] = cast(
Optional[List[str]],
_get_param(litellm_params, guardrail, "blocked_languages"),
)
action = cast(
Literal["block", "mask"],
_get_param(litellm_params, guardrail, "action", "block"),
)
confidence_threshold = float(
cast(
Union[int, float, str],
_get_param(litellm_params, guardrail, "confidence_threshold", 0.5),
)
)
mode = _get_param(litellm_params, guardrail, "mode")
event_hook = cast(
Optional[Union[Literal["pre_call", "post_call", "during_call"], List[str]]],
mode if mode is not None else DEFAULT_EVENT_HOOKS,
)

instance = BlockCodeExecutionGuardrail(
guardrail_name=guardrail_name,
blocked_languages=blocked_languages,
action=action,
confidence_threshold=confidence_threshold,
event_hook=event_hook,
default_on=bool(_get_param(litellm_params, guardrail, "default_on", False)),
)
litellm.logging_callback_manager.add_litellm_callback(instance)
return instance


guardrail_initializer_registry = {
SupportedGuardrailIntegrations.BLOCK_CODE_EXECUTION.value: initialize_guardrail,
}

guardrail_class_registry = {
SupportedGuardrailIntegrations.BLOCK_CODE_EXECUTION.value: BlockCodeExecutionGuardrail,
}

__all__ = [
"BlockCodeExecutionGuardrail",
"initialize_guardrail",
]
Loading
Loading