LiteLLM Minor Fixes & Improvements (10/15/2024)

docs/my-website/docs/proxy/customer_routing.md

@@ -24,21 +24,25 @@ curl -X POST --location 'http://0.0.0.0:4000/end_user/new' \
 
 ### 2. Add eu models to model-group 
 
-Add eu models to a model group. For azure models, litellm can automatically infer the region (no need to set it). 
+Add eu models to a model group. Use the 'region_name' param to specify the region for each model.
+
+Supported regions are 'eu' and 'us'.
 
 ```yaml
 model_list:
-    - model_name: gpt-3.5-turbo
-      litellm_params:
-        model: azure/gpt-35-turbo-eu # 👈 EU azure model
-        api_base: https://my-endpoint-europe-berri-992.openai.azure.com/
-        api_key: os.environ/AZURE_EUROPE_API_KEY
-    - model_name: gpt-3.5-turbo
-      litellm_params:
-        model: azure/chatgpt-v-2
-        api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
-        api_version: "2023-05-15"
-        api_key: os.environ/AZURE_API_KEY
+  - model_name: gpt-3.5-turbo
+    litellm_params:
+      model: azure/gpt-35-turbo # 👈 EU azure model
+      api_base: https://my-endpoint-europe-berri-992.openai.azure.com/
+      api_key: os.environ/AZURE_EUROPE_API_KEY
+      region_name: "eu"
+  - model_name: gpt-3.5-turbo
+    litellm_params:
+      model: azure/chatgpt-v-2
+      api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
+      api_version: "2023-05-15"
+      api_key: os.environ/AZURE_API_KEY
+      region_name: "us"
 
 router_settings:
   enable_pre_call_checks: true # 👈 IMPORTANT
@@ -74,10 +78,11 @@ Expected API Base in response headers
 
 ```
 x-litellm-api-base: "https://my-endpoint-europe-berri-992.openai.azure.com/"
+x-litellm-model-region: "eu" # 👈 CONFIRMS REGION-BASED ROUTING WORKED
 ```
 
 ### FAQ 
 
 **What happens if there are no available models for that region?**
 
-Since the router filters out models not in the specified region, it will return back as an error to the user, if no models in that region are available. 
+Since the router filters out models not in the specified region, it will return back as an error to the user, if no models in that region are available. 

docs/my-website/docs/proxy/guardrails/guardrails_ai.md

@@ -0,0 +1,118 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Guardrails.ai
+
+Use [Guardrails.ai](https://www.guardrailsai.com/) to add checks to LLM output.
+
+## Pre-requisites
+
+- Setup Guardrails AI Server. [quick start](https://www.guardrailsai.com/docs/getting_started/guardrails_server)
+
+## Usage
+
+1. Setup config.yaml 
+
+```yaml
+model_list:
+  - model_name: gpt-3.5-turbo
+    litellm_params:
+      model: gpt-3.5-turbo
+      api_key: os.environ/OPENAI_API_KEY
+
+guardrails:
+  - guardrail_name: "guardrails_ai-guard"
+    litellm_params:
+      guardrail: guardrails_ai
+      guard_name: "gibberish_guard" # 👈 Guardrail AI guard name
+      mode: "post_call"
+      api_base: os.environ/GUARDRAILS_AI_API_BASE # 👈 Guardrails AI API Base. Defaults to "http://0.0.0.0:8000"
+```
+
+2. Start LiteLLM Gateway 
+
+```shell
+litellm --config config.yaml --detailed_debug
+```
+
+3. Test request 
+
+**[Langchain, OpenAI SDK Usage Examples](../proxy/user_keys#request-format)**
+
+```shell
+curl -i http://localhost:4000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
+  -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [
+      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
+    ],
+    "guardrails": ["guardrails_ai-guard"]
+  }'
+```
+
+
+## ✨ Control Guardrails per Project (API Key)
+
+:::info
+
+✨ This is an Enterprise only feature [Contact us to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
+
+:::
+
+Use this to control what guardrails run per project. In this tutorial we only want the following guardrails to run for 1 project (API Key)
+- `guardrails`: ["aporia-pre-guard", "aporia-post-guard"]
+
+**Step 1** Create Key with guardrail settings
+
+<Tabs>
+<TabItem value="/key/generate" label="/key/generate">
+
+```shell
+curl -X POST 'http://0.0.0.0:4000/key/generate' \
+    -H 'Authorization: Bearer sk-1234' \
+    -H 'Content-Type: application/json' \
+    -D '{
+            "guardrails": ["guardrails_ai-guard"]
+        }
+    }'
+```
+
+</TabItem>
+<TabItem value="/key/update" label="/key/update">
+
+```shell
+curl --location 'http://0.0.0.0:4000/key/update' \
+    --header 'Authorization: Bearer sk-1234' \
+    --header 'Content-Type: application/json' \
+    --data '{
+        "key": "sk-jNm1Zar7XfNdZXp49Z1kSQ",
+        "guardrails": ["guardrails_ai-guard"]
+        }
+}'
+```
+
+</TabItem>
+</Tabs>
+
+**Step 2** Test it with new key
+
+```shell
+curl --location 'http://0.0.0.0:4000/chat/completions' \
+    --header 'Authorization: Bearer sk-jNm1Zar7XfNdZXp49Z1kSQ' \
+    --header 'Content-Type: application/json' \
+    --data '{
+    "model": "gpt-3.5-turbo",
+    "messages": [
+        {
+        "role": "user",
+        "content": "my email is ishaan@berri.ai"
+        }
+    ]
+}'
+```
+
+
+

docs/my-website/sidebars.js

@@ -95,6 +95,7 @@ const sidebars = {
           items: [
             "proxy/guardrails/quick_start", 
             "proxy/guardrails/aporia_api", 
+            "proxy/guardrails/guardrails_ai",
             "proxy/guardrails/lakera_ai", 
             "proxy/guardrails/bedrock",  
             "proxy/guardrails/pii_masking_v2", 

litellm/__init__.py

@@ -1005,10 +1005,11 @@ class LlmProviders(str, Enum):
 from .llms.volcengine import VolcEngineConfig
 from .llms.text_completion_codestral import MistralTextCompletionConfig
 from .llms.AzureOpenAI.azure import (
-    AzureOpenAIConfig,
     AzureOpenAIError,
     AzureOpenAIAssistantsAPIConfig,
 )
+
+from .llms.AzureOpenAI.chat.gpt_transformation import AzureOpenAIConfig
 from .llms.hosted_vllm.chat.transformation import HostedVLLMChatConfig
 from .llms.AzureOpenAI.chat.o1_transformation import AzureOpenAIO1Config
 from .llms.watsonx import IBMWatsonXAIConfig

litellm/integrations/custom_guardrail.py

@@ -1,4 +1,4 @@
-from typing import Literal, Optional
+from typing import List, Literal, Optional
 
 from litellm._logging import verbose_logger
 from litellm.integrations.custom_logger import CustomLogger
@@ -10,11 +10,20 @@ class CustomGuardrail(CustomLogger):
     def __init__(
         self,
         guardrail_name: Optional[str] = None,
+        supported_event_hooks: Optional[List[GuardrailEventHooks]] = None,
         event_hook: Optional[GuardrailEventHooks] = None,
         **kwargs,
     ):
         self.guardrail_name = guardrail_name
+        self.supported_event_hooks = supported_event_hooks
         self.event_hook: Optional[GuardrailEventHooks] = event_hook
+
+        if supported_event_hooks:
+            ## validate event_hook is in supported_event_hooks
+            if event_hook and event_hook not in supported_event_hooks:
+                raise ValueError(
+                    f"Event hook {event_hook} is not in the supported event hooks {supported_event_hooks}"
+                )
         super().__init__(**kwargs)
 
     def should_run_guardrail(self, data, event_type: GuardrailEventHooks) -> bool: