fix(redis): re-establish async cluster connections after a node restart

[yassin-berriai]

Relevant issues

Linear ticket

Resolves LIT-4083

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Root cause

When cache_params.redis_startup_nodes is set, litellm builds a redis-py async RedisCluster in get_redis_async_client. That client was constructed with no health check (health_check_interval=0, disabled) and no TCP keepalive (socket_keepalive=False). The async cluster client is created once and reused for the life of the process (cached in in_memory_llm_clients_cache / RedisClusterCache.redis_async_redis_cluster_client), so when a cluster restarts (the customer hit periodic ElastiCache Serverless maintenance restarts) the dropped connection sits in the pool and gets reused while dead. The first command after the restart stalls in RedisCluster.initialize(), and the spend-counter path runs that command under the LoggingWorker's asyncio.wait_for, so the stall is cancelled and surfaces exactly as reported

redis/asyncio/cluster.py execute_command -> await self.initialize()
redis/asyncio/cluster.py initialize -> async with self._lock
asyncio.exceptions.CancelledError
-> TimeoutError  (logging_worker.py _process_log_task)

The change

Build the async cluster client with a 25s health_check_interval and socket_keepalive=True, and let an explicit value from config still win. A non-zero health_check_interval is redis-py's documented mechanism for exactly this: before reusing an idle connection it sends a PING, and a dead connection is re-established before the real command runs

Proof 1 - the constructed client now carries the resilience config (real litellm client, against a real cluster)

Same get_redis_async_client(startup_nodes=...) the proxy uses, before and after the change

# base (litellm_internal_staging)
CONSTRUCTED health_check_interval=0  socket_keepalive=False

# this PR
CONSTRUCTED health_check_interval=25 socket_keepalive=True

Proof 2 - a non-zero health_check_interval actually re-validates an idle connection

Driving the real redis-py async cluster against a live single-node cluster, resetting the server command stats, idling past the interval, then issuing one GET, and counting health-check PINGs the server saw

interval=0  (base default) , idle 35s -> server saw 0 health-check PINGs   (dead conn would be reused)
interval=1                 , idle 6s  -> server saw 1 health-check PING
interval=25 (this PR)      , idle 30s -> server saw 1 health-check PING     (idle conn validated + reconnected)

Proof 3 - live proxy end to end on the failing path

Proxy launched with the cluster cache pointed at a live redis cluster, real Anthropic calls

litellm_settings:
  cache: true
  cache_params:
    type: redis
    redis_startup_nodes:
      - host: "127.0.0.1"
        port: 6883
    ttl: 600

# the cluster cache client pings + sets against the cluster
curl -s :4483/cache/ping -H "Authorization: Bearer $KEY"
{"status":"healthy","cache_type":"redis","ping_response":true,"set_cache_response":"success",
 "health_check_cache_params":{"redis_kwargs":{"startup_nodes":[{"host":"127.0.0.1","port":6883}],...},"redis_version":"7.4.9"}}

# two identical chat completions: the 2nd is served from the cluster cache
REQUEST 1 (cold):  content="redis cluster reconnect ok"  id=chatcmpl-3f366c60-...  latency 1.21s
REQUEST 2 (warm):  content="redis cluster reconnect ok"  id=chatcmpl-3f366c60-...  latency 0.029s

Same response id on the second call with a 40x latency drop confirms the async cluster cache get/set path (the path in the stack trace) works end to end with the resilient client, and the proxy log shows no CancelledError / redis exceptions

A laptop cannot reproduce the precise ElastiCache restart-window stall (a local docker restart closes connections gracefully, so redis-py recovers immediately regardless), but the fix activates redis-py's documented idle-connection health check plus keepalive, which is the mechanism that detects and re-establishes a connection silently dropped by a restart

Type

🐛 Bug Fix

Changes

get_redis_async_client now builds the async RedisCluster with reconnection-resilient defaults (health_check_interval=25, socket_keepalive=True) via a small _async_cluster_reconnect_kwargs helper, and health_check_interval / socket_keepalive are added to the cluster kwarg allow-list so they can be tuned or disabled per config. New regression tests in tests/test_litellm/test_redis.py assert the resilient defaults are applied and that an explicit config value overrides them

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[yassin-berriai]

@greptileai

[greptile-apps]

Greptile Summary

Fixes a connection-resurrection bug in the async RedisCluster client: when a cluster restarts (e.g., ElastiCache Serverless maintenance), a silently dropped idle connection was being reused, causing a stall inside RedisCluster.initialize() that was cancelled as a TimeoutError by the logging worker.

• get_redis_async_client now calls setdefault to inject health_check_interval=25 and socket_keepalive=True before constructing the async RedisCluster, so an idle connection is validated with a PING and reconnected before a real command runs; explicit config values still override the defaults.
• health_check_interval and socket_keepalive are added to the cluster kwarg allow-list (_get_redis_cluster_kwargs) so they are forwarded from user config.
• A new constant REDIS_CLUSTER_HEALTH_CHECK_INTERVAL = 25 is added to constants.py, and three new mock-only unit tests cover the allow-list membership, default injection, and user-override behaviour.

Confidence Score: 5/5

Safe to merge; the change is narrowly scoped to the async cluster construction path and uses setdefault so existing explicit config is never overwritten.

The fix is minimal and well-targeted: two setdefault calls guarded by the existing allow-list, a new constant in the right file, and three mock-only unit tests. No existing tests are modified, no auth or data paths are touched, and user-supplied config always wins over the new defaults.

No files require special attention. The sync cluster path (init_redis_cluster) is not updated, which may be worth revisiting as a follow-up.

Important Files Changed

Filename	Overview
litellm/_redis.py	Adds health_check_interval and socket_keepalive to the async RedisCluster allow-list and applies resilient defaults via setdefault, so user-supplied config still wins. Sync path (init_redis_cluster) is unchanged.
litellm/constants.py	Adds REDIS_CLUSTER_HEALTH_CHECK_INTERVAL = 25 in the right place; hardcoded (no os.getenv), unlike most peer constants, but overridable via cache_params config.
tests/test_litellm/test_redis.py	Three new mock-only tests cover the allow-list addition, default application, and user-override path. No real network calls made.

_{Reviews (3): Last reviewed commit: "fix(redis): re-establish async cluster c..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR makes async Redis cluster clients recover better after cluster node restarts. The main changes are:

• Adds default health_check_interval and socket_keepalive settings for async RedisCluster construction
• Allows health_check_interval and socket_keepalive to pass through the Redis cluster kwarg allow-list
• Adds tests for reconnect defaults and explicit config overrides

Confidence Score: 5/5

The changes are narrowly scoped to Redis async cluster client construction and related allow-listing/tests.

No correctness issues were identified in the modified Redis configuration path, and the tests cover both the new defaults and explicit override behavior.

T-Rex Logs

What T-Rex did

• Observed the Redis client construction state before changes, noting missing ALLOWLIST_HAS_HEALTH_CHECK_INTERVAL and ALLOWLIST_HAS_SOCKET_KEEPALIVE keys and the absence of DEFAULT_CALL and OVERRIDE_CALL keys.
• Verified the Redis client construction after changes, showing ALLOWLIST_HAS_HEALTH_CHECK_INTERVAL and ALLOWLIST_HAS_SOCKET_KEEPALIVE enabled, with DEFAULT_CALL_HEALTH_CHECK_INTERVAL=25, DEFAULT_CALL_SOCKET_KEEPALIVE=True, OVERRIDE_CALL_HEALTH_CHECK_INTERVAL=7, and OVERRIDE_CALL_SOCKET_KEEPALIVE=False.
• Initial idle healthcheck run showed missing LiteLLM helper and only startup_nodes, with health_check_interval omitted and redis-py interval 0 causing a GET only.
• Idle healthcheck after changes showed health_check_interval=25, LiteLLM helper present, explicit override value 1, socket_keepalive=True, and redis-py interval=1 causing a PING followed by GET.

_{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "fix(redis): re-establish async cluster c..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai

[yassin-berriai]

CI note: the three red checks are CircleCI jobs (litellm_router_testing, llm_translation_testing, proxy_pass_through_endpoint_tests) that are also red on several other open PRs against litellm_internal_staging right now, so they read as flaky infra rather than anything from this change. This PR only adds health_check_interval and socket_keepalive defaults to the async Redis cluster client, which none of those suites exercise. All GitHub Actions checks including lint are green, and Greptile is at 5/5 on the current commit. A maintainer rerun of the failed CircleCI jobs should clear them