Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/guides/security_settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,17 @@ guardrails:
ssl_verify: "/path/to/aim_cert.pem" # Use specific cert for AIM
```

### Cato Networks Guardrail (Proxy)
You can configure `ssl_verify` per guardrail in your `config.yaml`.

```yaml
guardrails:
- guardrail_name: cato-protected-app
litellm_params:
guardrail: cato_networks
ssl_verify: "/path/to/cato_cert.pem" # Use specific cert for AIM
```

### Priority Logic
LiteLLM resolves `ssl_verify` using the following priority:
1. **Explicit Parameter**: Passed in `completion()` or guardrail config.
Expand Down
137 changes: 137 additions & 0 deletions docs/proxy/guardrails/cato_networks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
import Image from '@theme/IdealImage';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Cato Networks

## Quick Start
### 1. Create a new Cato Networks AI Security Guard

Go to [Cato Networks CMA](https://cc.catonetworks.com/) and create a new AI Security guard.

Name your guard and select the AI Gateway option.

:::info
When using LiteLLM with virtual keys, use the virtual key alias as the name of the guard to be able to set key-specific policies.

Only the aliases of your virtual keys (and not the actual key secrets) will be sent to Cato Networks.
:::

### 2. Configure your Cato Networks AI Security Guard policies

Create an Engine Profile, selecting which detections to enable.
Create a Guard Policy rule that references this Engine Profile and your Guard, then configure the action to apply.

### 3. Add Cato Networks Guardrail on your LiteLLM config.yaml

Define your guardrails under the `guardrails` section
```yaml
model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: openai/gpt-3.5-turbo
api_key: os.environ/OPENAI_API_KEY

guardrails:
- guardrail_name: cato-protected-app
litellm_params:
guardrail: cato_networks
mode: [pre_call, post_call] # "During_call" is also available
api_key: os.environ/CATO_API_KEY
api_base: os.environ/CATO_API_BASE
ssl_verify: False # Optional, set to False to disable SSL verification or a string path to a custom CA bundle
```

Under the `api_key`, insert the API key you were issued. The key can be found in the guard's page.
You can also set `CATO_API_KEY` as an environment variable.

By default, the `api_base` is set to `https://api.aisec.catonetworks.com`. Set the correct url for your region.
If you are using a self-hosted Outpost, you can set the `api_base` to your Outpost's URL.

### 4. Start LiteLLM Gateway
```shell
litellm --config config.yaml
```

### 5. Make your first request

:::note
The following example depends on enabling *PII* detection in your policy.
You can adjust the request content to match different guard's policies.
:::

<Tabs>
<TabItem label="Successfully blocked request" value = "blocked">

:::note
When using LiteLLM with virtual keys, an `Authorization` header with the virtual key is required.
:::

```shell
curl -i http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer sk-1234" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [
{"role": "user", "content": "hi my email is ishaan@berri.ai"}
],
"guardrails": ["cato-protected-app"]
}'
```

If configured correctly, since `ishaan@berri.ai` would be detected by the Cato Networks AI Security Guard as PII, you'll receive a response similar to the following with a `400 Bad Request` status code:

```json
{
"error": {
"message": "\"ishaan@berri.ai\" detected as email",
"type": "None",
"param": "None",
"code": "400"
}
}
```

</TabItem>

<TabItem label="Successfully permitted request" value = "allowed">

:::note
When using LiteLLM with virtual keys, an `Authorization` header with the virtual key is required.
:::

```shell
curl -i http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-3.5-turbo",
"messages": [
{"role": "user", "content": "hi what is the weather"}
],
"guardrails": ["cato-protected-app"]
}'
```

The above request should not be blocked, and you should receive a regular LLM response (simplified for brevity):

```json
{
"model": "gpt-3.5-turbo-0125",
"choices": [
{
"finish_reason": "stop",
"index": 0,
"message": {
"content": "I can’t provide live weather updates without the internet. Let me know if you’d like general weather trends for a location and season instead!",
"role": "assistant"
}
}
]
}
```

</TabItem>


</Tabs>
8 changes: 4 additions & 4 deletions docs/proxy/guardrails/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ model_list:
guardrails:
- guardrail_name: general-guard
litellm_params:
guardrail: aim
guardrail: cato_networks
mode: [pre_call, post_call]
api_key: os.environ/AIM_API_KEY
api_base: os.environ/AIM_API_BASE
api_key: os.environ/CATO_API_KEY
api_base: os.environ/CATO_API_BASE
default_on: true # Optional

- guardrail_name: "aporia-pre-guard"
Expand Down Expand Up @@ -114,7 +114,7 @@ litellm_settings:

**Where this applies:** Only the **unified** guardrail path (providers that implement `apply_guardrail` and run through LiteLLM’s message translation layer) on **OpenAI Chat Completions** (`/v1/chat/completions`) and **Anthropic Messages** (`/v1/messages`). Examples include Presidio, Bedrock guardrails, `litellm_content_filter`, OpenAI Moderation, Generic Guardrail API, and custom code guardrails that define `apply_guardrail`.

**Where this does *not* apply:** Guardrails that run only via direct hooks on the raw request (e.g. Lakera v2, Aporia, DynamoAI, Javelin, Lasso, Pangea, Model Armor, Azure Content Safety hooks, Guardrails AI, AIM, tool permission, MCP security). It also does not apply to other routes until those endpoints use the same translation layer (e.g. Responses API, embeddings, speech).
**Where this does *not* apply:** Guardrails that run only via direct hooks on the raw request (e.g. Lakera v2, Aporia, DynamoAI, Javelin, Lasso, Pangea, Model Armor, Azure Content Safety hooks, Guardrails AI, AIM, Cato Networks, tool permission, MCP security). It also does not apply to other routes until those endpoints use the same translation layer (e.g. Responses API, embeddings, speech).

### Load Balancing Guardrails

Expand Down