fix(proxy): stop stale auth cache re-publish to Redis so key updates propagate across replicas

[yassin-berriai]

Relevant issues

Linear ticket

Resolves LIT-4219

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Rig: two proxy replicas (ports 4219 and 4220) sharing one Postgres and one Redis, enable_redis_auth_cache: true, real gpt-5.4-mini completions against the live OpenAI API (no mocks). Config mirrors the reproduction in the Linear ticket

model_list:
  - model_name: gpt-5.4-mini
    litellm_params:
      model: gpt-5.4-mini
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: sk-litfix4219

litellm_settings:
  cache: true
  enable_redis_auth_cache: true
  cache_params:
    type: redis
    host: 127.0.0.1
    port: 6598

Steps, identical for both runs: create a key with model_rpm_limit {"gpt-5.4-mini": 5}, send one completion through each replica to prime both in-memory caches, then raise the limit and keep traffic flowing

curl -s -X POST http://127.0.0.1:4219/key/generate -H "Authorization: Bearer sk-litfix4219" \
  -H "Content-Type: application/json" -d '{"model_rpm_limit": {"gpt-5.4-mini": 5}}'
for PORT in 4219 4220; do
  curl -s -X POST "http://127.0.0.1:${PORT}/v1/chat/completions" -H "Authorization: Bearer $KEY" \
    -H "Content-Type: application/json" -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"Say ok"}]}'
done
curl -s -X POST http://127.0.0.1:4219/key/update -H "Authorization: Bearer sk-litfix4219" \
  -H "Content-Type: application/json" -d '{"key": "'$KEY'", "model_rpm_limit": {"gpt-5.4-mini": 20}}'

Before (unfixed, commit 69a491e)

/key/update deletes the Redis auth blob (TTL -2) and the DB reads back the new limit, but 20 real completions alternating across both replicas resurrect the OLD blob in Redis with a fresh 60s TTL

updated 5 -> 20; /key/info now: {'gpt-5.4-mini': 20}
redis TTL right after update (want -2): -2
after 20 real mixed-replica completions:
redis auth blob model_rpm_limit: {'gpt-5.4-mini': 5}
redis TTL: 60

A burst of 8 parallel completions on replica 4220 in a fresh rate window still enforces the old limit of 5

200 200 200 200 200 429 429 429

This never converges while traffic continues: every request on a replica holding the stale in-memory copy re-publishes it to Redis, and replicas whose copies expire re-prime themselves from the poisoned blob

After (fixed, commit e1c9b0a)

Note: later commits consolidated the sibling branch (the post-auth write-back captured below as a local-only write is now removed outright, and the key blob no longer enters the spend writeback at all). The Redis-facing behavior this run proves is unchanged by that consolidation; the write path only became stricter

Same steps. The Redis blob is only repopulated by a fresh DB load, so it carries the NEW limit under the same sustained traffic, and the second replica converges within the 60s auth cache TTL while traffic keeps flowing (replica2_limit=429 rows are its stale limit-5 window rejecting requests before convergence)

updated 5 -> 20 at 11:15:25
redis TTL right after update (want -2): -2
t=+0s   replica1_limit=20 replica2_limit=429 redis_blob_limit=20
t=+18s  replica1_limit=20 replica2_limit=429 redis_blob_limit=20
t=+43s  replica1_limit=20 replica2_limit=429 redis_blob_limit=20
t=+49s  replica1_limit=20 replica2_limit=20  redis_blob_limit=20
t=+74s  replica1_limit=20 replica2_limit=20  redis_blob_limit=20

The same burst of 8 parallel completions on replica 4220 now passes in full at the new limit

200 200 200 200 200 200 200 200

Independent e2e verification (Devin)

Reran this scenario independently on a fresh live rig (two proxy replicas on ports 4001 and 4002 sharing one Postgres and one Redis, enable_redis_auth_cache: true) using real gemini-2.5-flash completions against the live Gemini API, no mocks, comparing merge-base 69a491e16887353a4170f571e7698566bd0db607 (unfixed) against PR head e1c9b0a3bcd911ec8c05cbc35be2abef56100ca8 (fixed). Each run generates a key at model_rpm_limit {"gemini-2.5-flash": 5}, primes both in-memory caches with one completion per replica, raises the limit to 20 (/key/info returns 20 and the redis auth blob TTL is -2 right after the update), alternates completions across both replicas for ~90s while sampling redis and each replica's enforced limit every ~10s, then fires 8 parallel completions at replica B once the rate window clears. On the merge-base the redis blob reverts to 5 and replica B keeps enforcing 5 for the whole window (burst 5x200 then 429s); on the PR head the redis blob stays at 20 and replica B converges from 5 to 20 at t=62s while traffic keeps flowing (burst 8x200)

Unfixed run (merge-base 69a491e), full flow

Fixed run (PR head e1c9b0a), full flow

Unfixed: redis auth blob reverts to 5 and both replicas keep enforcing 5

Fixed: redis stays at 20 and replica B converges to 20 at t=62s

Fixed: 8 parallel completions at replica B all pass at the new limit

Unfixed, merge-base 69a491e, sustained-traffic sampling and burst

/key/update -> HTTP 200
/key/info model_rpm_limit right after update: {'gemini-2.5-flash': 20}
redis TTL of auth blob immediately after update (want -2): -2

  t(s) | redis model_rpm_limit      | A enforces(code) | B enforces(code)
-------+----------------------------+------------------+------------------
     0 | {'gemini-2.5-flash': 5}    | 20(200)          | 5(429)
     7 | {'gemini-2.5-flash': 5}    | 20(200)          | 5(429)
    13 | {'gemini-2.5-flash': 5}    | 20(200)          | 5(429)
    19 | {'gemini-2.5-flash': 5}    | 20(200)          | 5(429)
    25 | {'gemini-2.5-flash': 5}    | 20(429)          | 5(429)
    31 | {'gemini-2.5-flash': 5}    | 20(429)          | 5(429)
    37 | {'gemini-2.5-flash': 5}    | 20(429)          | 5(429)
    43 | {'gemini-2.5-flash': 5}    | 20(429)          | 5(429)
    50 | {'gemini-2.5-flash': 5}    | 20(429)          | 5(429)
    56 | {'gemini-2.5-flash': 5}    | 20(429)          | 5(429)
    62 | {'gemini-2.5-flash': 5}    | 5(429)           | 5(429)
    68 | {'gemini-2.5-flash': 5}    | 5(429)           | 5(429)
    74 | {'gemini-2.5-flash': 5}    | 5(429)           | 5(429)
    80 | {'gemini-2.5-flash': 5}    | 5(429)           | 5(429)
    87 | {'gemini-2.5-flash': 5}    | 5(429)           | 5(429)

8 parallel completions at replica B after the window clears:
200 200 200 200 200 429 429 429  (5x200, 3x429)

Fixed, PR head e1c9b0a, sustained-traffic sampling and burst

/key/update -> HTTP 200
/key/info model_rpm_limit right after update: {'gemini-2.5-flash': 20}
redis TTL of auth blob immediately after update (want -2): -2

  t(s) | redis model_rpm_limit      | A enforces(code) | B enforces(code)
-------+----------------------------+------------------+------------------
     0 | {'gemini-2.5-flash': 20}   | 20(200)          | 5(429)
     6 | {'gemini-2.5-flash': 20}   | 20(200)          | 5(429)
    12 | {'gemini-2.5-flash': 20}   | 20(200)          | 5(429)
    18 | {'gemini-2.5-flash': 20}   | 20(200)          | 5(429)
    25 | {'gemini-2.5-flash': 20}   | 20(429)          | 5(429)
    31 | {'gemini-2.5-flash': 20}   | 20(429)          | 5(429)
    37 | {'gemini-2.5-flash': 20}   | 20(429)          | 5(429)
    43 | {'gemini-2.5-flash': 20}   | 20(429)          | 5(429)
    49 | {'gemini-2.5-flash': 20}   | 20(429)          | 5(429)
    55 | {'gemini-2.5-flash': 20}   | 20(429)          | 5(429)
    62 | {'gemini-2.5-flash': 20}   | 20(200)          | 20(200)
    68 | {'gemini-2.5-flash': 20}   | 20(200)          | 20(200)
    74 | {'gemini-2.5-flash': 20}   | 20(429)          | 20(429)
    80 | {'gemini-2.5-flash': 20}   | 20(429)          | 20(429)
    86 | {'gemini-2.5-flash': 20}   | 20(429)          | 20(429)
    92 | {'gemini-2.5-flash': 20}   | 20(429)          | 20(429)

8 parallel completions at replica B after the window clears:
200 200 200 200 200 200 200 200  (8x200)

Type

🐛 Bug Fix

Changes

With enable_redis_auth_cache and multiple replicas, /key/update and /key/delete delete the Redis auth blob and the handling pod's in-memory entry, but two read-path writers re-published the stale blob from any other replica's per-pod memory back to Redis with a fresh 60s TTL on every request: the post-auth re-cache in user_api_key_auth and the post-response spend writeback pipeline in update_cache. Replicas whose in-memory entries expired then re-primed themselves from the poisoned Redis entry, so key limit and access changes never took effect fleet-wide while traffic continued

The auth object is now written only by the DB-load paths (IdentityStore._resolve_key, get_key_object) and deleted on mutation: the post-auth re-cache is removed outright (even a local-only write could race an invalidation and resurrect a revoked key on the worker that served it) and spend tracking no longer writes the auth object back at all, since spend flows through the spend:key:* counters. The remaining spend writebacks for user, team, end-user, and tag objects become local-only so they cannot republish stale management objects either, with one deliberate exception: the proxy-wide {litellm_proxy_admin_name}:spend scalar keeps its shared Redis write because the global max_budget check reads it between authoritative DB reloads, and it carries no limits or permissions so sharing it cannot resurrect an invalidated auth blob. Cross-pod budget enforcement is otherwise unaffected; it reads the dedicated Redis spend counters through get_current_spend. Deployments without enable_redis_auth_cache see no behavior change on the Redis side because that cache has no Redis attached. This consolidates the sibling branch from PR #33560, which reached the same root cause for the delete-propagation case (LIT-4350, closed as a duplicate). Cross-pod budget enforcement is unaffected: it reads the dedicated Redis spend counters through get_current_spend, with the auth blob's spend only ever used as a local fallback. Deployments without enable_redis_auth_cache see no behavior change because that cache has no Redis attached

DualCache's redis-to-memory read backfill also ignored default_in_memory_ttl, pinning backfilled entries for InMemoryCache's 600s default instead of the configured 60s auth TTL, which let a replica primed from Redis serve a stale key for 10 minutes even with the writers fixed. The backfill now injects the configured default exactly like every write path already does

Regression tests: the auth builder must not write a cache-served token to Redis (fails on the unfixed builder), get_key_object's DB load must still publish to Redis (guards against over-correcting), the spend writeback pipeline must be local-only, and the DualCache backfill must honor default_in_memory_ttl. Each was mutation-checked by reverting the corresponding production change and confirming the test fails

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/54cb6081085149f6965d2b8568ab9b44

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a stale-auth-blob republication bug in multi-replica deployments using enable_redis_auth_cache: /key/update and /key/delete correctly invalidate Redis, but two read-path writers (post-auth re-cache and spend writeback) were re-publishing stale blobs from any replica's in-memory copy with a fresh TTL on every request, preventing limit changes from propagating fleet-wide. A secondary bug caused Redis-to-memory backfills to ignore default_in_memory_ttl, pinning entries at InMemoryCache's 600 s default instead of the configured 60 s auth TTL.

• Auth write-path narrowed: the post-auth _cache_key_object fire-and-forget task is removed from _user_api_key_auth_builder; Redis writes now occur only in DB-load paths (get_key_object, IdentityStore._resolve_key). The spend pipeline no longer writes the key auth object back at all.
• Spend writeback split: all management-object updates (user, team, end-user, tag spend) become local-only; only the global proxy spend scalar keeps its shared-Redis write so cross-replica max_budget enforcement is unaffected.
• DualCache backfill TTL fixed: _backfill_kwargs helper injects default_in_memory_ttl into all three read-path backfills (sync get_cache, async_get_cache, async_batch_get_cache).

Confidence Score: 5/5

Safe to merge; changes are narrowly scoped to the Redis write paths, all three fixes are independently tested, and the global proxy spend scalar deliberately retains its shared write so cross-replica budget enforcement is not disrupted.

The auth writeback removal, spend-pipeline split, and DualCache backfill TTL fix are all logically correct. DB-load paths still publish to Redis (verified by a dedicated regression test), management-object writes are local-only (verified by pipeline-mock assertions), and the backfill TTL tests confirm the in-memory expiry matches default_in_memory_ttl. No regressions were introduced on the critical auth path.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/caching/dual_cache.py	Adds _backfill_kwargs helper to inject default_in_memory_ttl when backfilling Redis hits into the in-memory tier; applied correctly to all three read paths (sync get_cache, async_get_cache, async_batch_get_cache). Fixes a TTL bug where backfilled entries would survive the full InMemoryCache default (600 s) instead of the configured auth-cache TTL (60 s).
litellm/proxy/auth/user_api_key_auth.py	Removes the _cache_key_object fire-and-forget task that re-wrote a cache-hit token back into shared Redis on every authenticated request. DB-load paths still populate Redis; cache-hit paths no longer republish stale blobs.
litellm/proxy/proxy_server.py	Removes the auth-object write-back from the spend pipeline (key's team_spend/team_member_spend/spend mutations on existing_spend_obj no longer reach values_to_update_in_cache). Splits the remaining pipeline write into local-only for all management objects (user, team, end-user, tag) and shared-Redis for the global proxy spend scalar only.
tests/test_litellm/caching/test_dual_cache.py	Adds two new async tests that verify async_get_cache and async_batch_get_cache backfill paths honor default_in_memory_ttl; both check the TTL dict directly in InMemoryCache. Tests are correctly scoped to in-memory verification only (no real Redis).
tests/test_litellm/proxy/auth/test_user_api_key_auth.py	Adds test_auth_does_not_rewrite_cached_key_object_back_into_cache which seeds a stale cache entry, runs _user_api_key_auth_builder, drains pending tasks, and asserts the cached value's last_refreshed_at and metadata are unchanged — a clean mutation-check that fails on the pre-fix builder.
tests/test_litellm/proxy/auth/test_auth_checks.py	Adds TestAuthCacheRedisWritePolicy with a test that verifies DB-load paths still publish to Redis (get_key_object with a fake Redis client asserts async_set_cache is awaited exactly once). Guards against over-correcting the fix.
tests/test_litellm/proxy/test_proxy_server.py	Adds two tests: test_spend_tracking_never_writes_the_auth_object_back and test_update_cache_global_proxy_spend_scalar_stays_shared. Both correctly drain async tasks before asserting and cover the two sides of the spend writeback split.

_{Reviews (4): Last reviewed commit: "fix(proxy): stop stale auth cache re-pub..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai please review the current head 2357874; it replaces the asyncio.sleep timing guard in the new regression tests with a deterministic drain of pending tasks, addressing the one uncertainty from the previous review

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_stale_auth_redis_recache (e482d7d) with litellm_internal_staging (2162da5)}

[yassin-berriai]

@greptileai please review the current head f6342e7; it splits the spend writeback so the proxy-wide spend scalar keeps its shared Redis write (addresses the Veria finding on cross-replica global budget enforcement) while management-object blobs stay pod-local

[yassin-berriai]

@greptileai please review the current head e482d7d; it consolidates the sibling branch from PR #33560 (the post-auth re-cache is now removed outright rather than made local-only, and spend tracking no longer writes the key auth object back at all) and rebases onto current litellm_internal_staging as a single commit

[yassin-berriai]

The osv-scan failure on this PR is base drift, not this diff: this branch has a zero-line uv.lock delta against litellm_internal_staging, and the flagged package (mcp 1.26.0, three known High findings) comes from the base lockfile itself, so every branch cut from the current staging tip fails the same way. Fix filed separately as #33591; osv-scan here will be rerun once that lands