Skip to content

(fix proxy redis) Add redis sentinel support #6154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ishaan-jaff merged 3 commits into from
Nov 13, 2024
Merged

(fix proxy redis) Add redis sentinel support #6154

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
68fab2b
add sentinel_password support
ishaan-jaff Oct 10, 2024
671ac7c
add doc for setting redis sentinel password
ishaan-jaff Oct 10, 2024
4506fa9
fix redis sentinel - use sentinel password
ishaan-jaff Oct 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/my-website/docs/proxy/caching.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ litellm_settings:
type: "redis"
service_name: "mymaster"
sentinel_nodes: [["localhost", 26379]]
sentinel_password: "password" # [OPTIONAL]
```

</TabItem>
Expand All @@ -143,6 +144,7 @@ You can configure redis sentinel in your .env by setting `REDIS_SENTINEL_NODES`
```env
REDIS_SENTINEL_NODES='[["localhost", 26379]]'
REDIS_SERVICE_NAME = "mymaster"
REDIS_SENTINEL_PASSWORD = "password"
```

:::note
Expand Down
18 changes: 15 additions & 3 deletions litellm/_redis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@

# s/o [@Frank Colson](https://www.linkedin.com/in/frank-colson-422b9b183/) for this redis implementation
import os
from typing import List, Optional, Union
from typing import Dict, List, Optional, Union

import redis # type: ignore
import redis.asyncio as async_redis # type: ignore

import litellm
from litellm import get_secret
from litellm import get_secret, get_secret_str

from ._logging import verbose_logger

Expand Down Expand Up @@ -139,6 +139,13 @@ def _get_redis_client_logic(**env_overrides):
if _sentinel_nodes is not None and isinstance(_sentinel_nodes, str):
redis_kwargs["sentinel_nodes"] = json.loads(_sentinel_nodes)

_sentinel_password: Optional[str] = redis_kwargs.get(
"sentinel_password", None
) or get_secret_str("REDIS_SENTINEL_PASSWORD")

if _sentinel_password is not None:
redis_kwargs["sentinel_password"] = _sentinel_password

_service_name: Optional[str] = redis_kwargs.get("service_name", None) or get_secret( # type: ignore
"REDIS_SERVICE_NAME"
)
Expand Down Expand Up @@ -215,6 +222,7 @@ def _init_redis_sentinel(redis_kwargs) -> redis.Redis:

def _init_async_redis_sentinel(redis_kwargs) -> async_redis.Redis:
sentinel_nodes = redis_kwargs.get("sentinel_nodes")
sentinel_password = redis_kwargs.get("sentinel_password")
service_name = redis_kwargs.get("service_name")

if not sentinel_nodes or not service_name:
Expand All @@ -225,7 +233,11 @@ def _init_async_redis_sentinel(redis_kwargs) -> async_redis.Redis:
verbose_logger.debug("init_redis_sentinel: sentinel nodes are being initialized.")

# Set up the Sentinel client
sentinel = async_redis.Sentinel(sentinel_nodes, socket_timeout=0.1)
sentinel = async_redis.Sentinel(
sentinel_nodes,
socket_timeout=0.1,
password=sentinel_password,
)

# Return the master instance for the given service

Expand Down