feat(proxy): configure the coordination redis independently of the response cache

[yassin-berriai]

Relevant issues

Linear ticket

Resolves LIT-3861

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Stacked on #32635, which is the minimal defect fix. This PR is the full decoupling and should merge after it. Review the diff against litellm_redis_usage_cache_env_fallback

What this enables: two Redis roles, configured separately, running side by side. The LLM response cache (litellm_settings.cache_params) and the coordination Redis (general_settings.coordination_redis) are now independent connections. A deployment can point the response cache at a redis-semantic backend for semantic cache hits, and simultaneously point the coordination Redis at a plain Redis (or a cluster, or a sentinel set) for cross-pod tpm/rpm limits, parallel request limits, spend tracking, the pod lock manager and shared health checks. They may be different servers, different topologies, or the same server; nothing forces them to be the same connection any more, and configuring one no longer implies or disables the other

Screenshots / Proof of Fix

Two pods share one Postgres and one Redis 8, real OpenAI calls to gpt-5.4-mini

The explicit block works with no response cache at all

Before this PR there was no way to get a coordination Redis without configuring a response cache. With general_settings.coordination_redis and no litellm_settings.cache at all, an rpm_limit: 2 key is enforced across both pods

$ # config.yaml has general_settings.coordination_redis, no litellm_settings.cache
req 1 -> pod 4863 : HTTP 200
req 2 -> pod 4864 : HTTP 200
req 3 -> pod 4863 : HTTP 429
req 4 -> pod 4864 : HTTP 429

$ docker exec <redis> redis-cli --scan | sort
{api_key:8d219d83...}:max_parallel_requests
{api_key:8d219d83...}:requests
{api_key:8d219d83...}:window
spend:key:8d219d83...

On the parent commit the same config serves all four requests (limits fall back to per-pod memory)

A semantic response cache and a plain coordination Redis coexist

This is the case the ticket was filed for. cache_params.type: redis-semantic drives the response cache while general_settings.coordination_redis independently drives rate limiting and spend, both live on the same two pods at the same time. The Redis key scan below shows the two roles side by side: litellm_semantic_cache_index:* from the response cache, and the rate limit window, parallel request slots and spend counters from the coordination Redis

req 1 -> pod 4863 : HTTP 200
req 2 -> pod 4864 : HTTP 200
req 3 -> pod 4863 : HTTP 429
req 4 -> pod 4864 : HTTP 429

# identical prompt to pod 4864 then pod 4863, cross pod semantic cache hit
pod 4864 id: chatcmpl-DzmucJO25OYl9duoY6NxrkeOB7TKl
pod 4863 id: chatcmpl-DzmucJO25OYl9duoY6NxrkeOB7TKl x-litellm-cache-key: 09756e48...

$ docker exec <redis> redis-cli --scan | sort
{api_key:136143b4...}:requests
{api_key:136143b4...}:window
litellm_semantic_cache_index:10b8a13ab46cbce4d2be2c5d3c8946e496d14db13e08e0714c068547dde41d61
spend:key:136143b4...

The management endpoints

GET reports which source is in effect and redacts credentials

$ curl -s $PROXY/coordination_redis/settings -H "Authorization: Bearer $MASTER_KEY"
source: coordination_redis
values: {"host": "127.0.0.1", "port": 7240, "password": "***REDACTED***"}

$ curl -s -o /dev/null -w "%{http_code}\n" $PROXY/coordination_redis/settings
401

A connection test against a port with nothing listening reports unhealthy, and the submitted password never appears in the error

$ curl -s -X POST $PROXY/coordination_redis/settings/test -H "Authorization: Bearer $MASTER_KEY" \
    -d '{"settings": {"host": "127.0.0.1", "port": 6399, "password": "supersecret123"}}'
{"status":"unhealthy","error":"Error 61 connecting to 127.0.0.1:6399. Connection refused."}

$ curl -s -X POST $PROXY/coordination_redis/settings/test ... | grep -c supersecret123
0

A block with no connection target is rejected rather than silently running without coordination

$ curl -s -w "\nHTTP %{http_code}\n" -X POST $PROXY/coordination_redis/settings -H "Authorization: Bearer $MASTER_KEY" \
    -d '{"settings": {"ssl": true}}'
{"detail":{"error":"coordination_redis needs a connection target: set one of host, url, startup_nodes, or sentinel_nodes"}}
HTTP 400

Settings saved from the dashboard take effect on restart

Saved via POST /coordination_redis/settings, then the proxy restarted with a config file that has no coordination block and with no REDIS_* variables in the environment

$ docker exec <pg> psql -U litellm -tc "select param_value from \"LiteLLM_Config\" where param_name='general_settings';"
 {"coordination_redis": {"host": "127.0.0.1", "port": 7240}}

$ # restart, then drive an rpm_limit: 2 key
req 1: HTTP 200
req 2: HTTP 200
req 3: HTTP 429

$ docker exec <redis> redis-cli --scan | sort
{api_key:5cf13d1e...}:max_parallel_requests
{api_key:5cf13d1e...}:requests
{api_key:5cf13d1e...}:window
litellm_config:param:general_settings
spend:key:5cf13d1e...

For the Admin UI, run the proxy and the dashboard, go to http://localhost:4000/ui/?page=caching, open the Coordination Redis tab, set a host and port, press Test Connection, then Save Changes, and restart the proxy

Type

🆕 New Feature

Changes

redis_usage_cache could only ever be populated from the response-cache backend. #32635 added a REDIS_* environment fallback so a semantic cache no longer silently disables cross-pod coordination. This PR finishes the decoupling by giving the coordination Redis its own configuration surface

general_settings.coordination_redis accepts host, port, username, password, url, ssl, startup_nodes, sentinel_nodes, sentinel_password and service_name, resolves os.environ/ references the way cache_params does, and is validated by a CoordinationRedisParams model that rejects a block naming no connection target. Resolution order is the explicit block, then a plain-Redis response-cache backend, then the environment. A cluster target, whether from startup_nodes or REDIS_CLUSTER_NODES, now builds a RedisClusterCache so consumers that branch on cluster mode take the cluster path; sentinel-only environments are covered too, closing a gap the environment fallback inherited from the transaction-buffer helper

GET, POST and POST /test under /coordination_redis/settings back a new Coordination Redis tab on the Caching page of the dashboard. The read reports whether the coordination Redis comes from the explicit block, a borrowed response cache, or the environment, so an operator can see what is actually in effect. Credentials are redacted on read and scrubbed from connection-test errors. Settings saved there are persisted to the config row and read back during startup, before the coordination Redis is published to its consumers, so the restart actually applies them

One bug fix falls out of building the connection test. _get_redis_client_logic merged the environment before the caller's arguments but then let a url from REDIS_URL strip the caller's host and port. An explicitly configured host therefore lost to the environment, which would have made both an explicit coordination_redis block and a connection test typed into the dashboard silently target whatever REDIS_URL named. An explicit connection target now outranks the environment, while an explicit url and the environment-only path are unchanged

Deploy artifacts and documentation are handled separately, in the helm and terraform branch and in the docs repo

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ yucheng-berri
❌ yassin-berriai
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[yassin-berriai]

osv-scan is red here because this PR stacks on #32635, whose base predates the soupsieve 2.8.4 bump that landed on staging in #32643; the scan reads the stacked base's uv.lock (soupsieve 2.8.3). Nothing in this diff touches dependencies. It clears as soon as #32635 merges and this branch rebases onto staging, which is the intended merge order anyway

[greptile-apps]

Greptile Summary

This PR decouples the proxy's coordination Redis (cross-pod rate limits, spend tracking, pod lock manager) from the response-cache backend by introducing general_settings.coordination_redis in the YAML config, a DB-persisted override applied at startup, and GET/POST/test management endpoints backed by a new dashboard tab.

• _build_redis_usage_cache is a new shared builder that picks up REDIS_CLUSTER_NODES from the environment even when the caller supplies an explicit host or URL; this silently redirects the startup client and the dashboard "Test Connection" call to env-defined cluster nodes instead of the operator-specified target.
• _scrub_credentials in the test endpoint redacts the literal os.environ/VAR reference strings but not the resolved password value, so a Redis auth/connection error that echoes the real password would reach the caller unscrubbed when env references are used.
• The _redis.py bugfix (explicit host outranks REDIS_URL from env), the CoordinationRedisParams validation model, the audit-log and redaction logic, and the comprehensive mock test suite are all well-implemented.

Confidence Score: 3/5

The core decoupling logic is sound but _build_redis_usage_cache can silently connect to the wrong Redis when REDIS_CLUSTER_NODES is present in the environment alongside an explicit coordination_redis host block, including inside the dashboard connection-test handler.

The shared _build_redis_usage_cache helper always consults REDIS_CLUSTER_NODES from the environment when startup_nodes is absent from the caller's params. In a cluster deployment, an explicit coordination_redis: {host: "dedicated-redis"} would silently build a cluster client against the env nodes at startup, and the dashboard Test Connection would ping those same cluster nodes rather than the host the operator typed in.

litellm/proxy/proxy_server.py — specifically _build_redis_usage_cache and all three of its call sites (_init_coordination_redis, _init_coordination_redis_from_db, check_coordination_redis_connection).

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Adds coordination Redis initialization from both file config and DB; _build_redis_usage_cache silently mixes REDIS_CLUSTER_NODES env into explicit host-based configs, which can redirect the startup client and the dashboard test connection to the wrong Redis.
litellm/proxy/management_endpoints/coordination_redis_endpoints.py	New endpoints for GET/POST/TEST coordination Redis; credential redaction and audit logging are well-structured, but the test handler's scrub logic misses env-ref-resolved passwords in Redis error strings.
litellm/_redis.py	Adds an explicit-target guard that prevents REDIS_URL from the environment from overriding an explicit host/startup_nodes/sentinel_nodes argument — a clean targeted fix.
litellm/proxy/_types.py	Adds CoordinationRedisParams and CoordinationRedisNode Pydantic models with a has_connection_target() validator; types are well-defined and the model uses extra="allow" to pass through additional Redis kwargs.
litellm/types/management_endpoints/coordination_redis_endpoints.py	New field definitions and type exports for coordination Redis settings; clean and well-structured.
tests/test_litellm/proxy/management_endpoints/test_coordination_redis_endpoints.py	Comprehensive mock-only unit tests covering redaction, source detection, merge-over-saved, credential scrubbing, and validation edge cases; all look correct.
tests/test_litellm/proxy/test_proxy_server.py	Adds coordination Redis init tests; the _run_init_cache_with_backend helper was updated to use the returned value rather than reading the global, which matches the new API but loses direct global-state verification.
tests/test_litellm/test_redis.py	New tests for _get_redis_client_logic explicit-target precedence over env URL; straightforward and correct.
ui/litellm-dashboard/src/app/(dashboard)/caching/components/coordination_redis_settings/coordinationRedisUtils.ts	Form utilities for building and parsing the coordination Redis payload; boolean fields always serialize as false rather than undefined when unset, meaning ssl: false is always included in POST bodies.
ui/litellm-dashboard/src/app/(dashboard)/caching/components/coordination_redis_settings/index.tsx	Main coordination Redis settings UI component; form lifecycle, test connection, and save flows look correct.
ui/litellm-dashboard/src/components/networking.tsx	Adds three new API call helpers for coordination Redis GET/POST/test; consistent with existing patterns in the file.

_{Reviews (1): Last reviewed commit: "feat(proxy): configure the coordination ..." | Re-trigger Greptile}

[greptile-apps]

REDIS_CLUSTER_NODES leaks into explicit and test-connection paths

_build_redis_usage_cache unconditionally reads REDIS_CLUSTER_NODES from the environment when startup_nodes is absent from the caller's params. This means an explicit coordination_redis: {host: "myhost"} block silently builds a RedisClusterCache against the environment cluster nodes instead of "myhost", defeating the PR's stated goal of "explicit takes precedence."

The same bug hits the /coordination_redis/settings/test endpoint: a user who types host: "new-coord-redis" in the dashboard gets a cluster ping to whatever REDIS_CLUSTER_NODES names, not to the host they specified, so the test result is for an entirely different Redis.

A simple guard fixes both call sites — only consult the environment cluster nodes when the caller provided no explicit connection target: add an allow_env_cluster: bool = False keyword argument, pass True only from _build_redis_usage_cache_from_environment, and leave the explicit and test-connection paths at the default False.

[greptile-apps]

_scrub_credentials doesn't scrub resolved env-ref passwords

_credential_values collects the literal string from settings, so when a credential is stored as an os.environ/REDIS_PASSWORD reference the scrubber removes the reference string, not the actual password. If Redis echoes the real password in a connection-refused or auth-failure message (e.g. via an embedded URL), the resolved secret leaks to the caller of the test endpoint.

The resolved values from params (returned by _validated_params) should be included in the scrub set alongside the raw setting values, since those are what the Redis client actually uses and what would appear in error output.

[codecov]

Codecov Report

❌ Patch coverage is 95.39007% with 13 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...nagement_endpoints/coordination_redis_endpoints.py	95.20%	8 Missing ⚠️
litellm/proxy/proxy_server.py	93.67%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

Medium: Redis credentials stored in plaintext

settings can contain password, sentinel_password, or a credential-bearing url, and this writes them directly to LiteLLM_Config. An attacker with read access to that table or its backups can recover the Redis credentials and use them to alter shared spend and rate-limit state. Encrypt credential fields before persistence and decrypt them when loading the saved coordination configuration, following the existing cache-settings storage path.

[veria-ai]

PR overview

This PR adds proxy management support for configuring the coordination Redis separately from the response cache. It introduces endpoints and persistence for coordination Redis settings used by shared proxy coordination state.

There is one open security issue: coordination Redis credentials may be persisted in plaintext, including passwords or credential-bearing URLs. If someone can read the configuration table or its backups, they could recover those credentials and potentially modify shared spend and rate-limit coordination state. No issues have been marked fixed yet, so this credential-handling gap remains the current security concern.

Open issues (1)

• Medium: Redis credentials stored in plaintext — litellm/proxy/management_endpoints/coordination_redis_endpoints.py:368

Fixed/addressed: 0 · PR risk: 4/10

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_coordination_redis_config (98b2d72) with litellm_internal_staging (eb7e4a5)}