fix(auth): stop re-caching stale auth objects so key updates propagate across replicas

[yassin-berriai]

Relevant issues

Linear ticket

Resolves LIT-4350

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The branch was rebased on current staging and squashed to a single commit after all proofs were captured; the commit hashes below refer to the pre-squash history and the diff is unchanged (verified hunk-for-hunk against the squashed head).

All runs against two live proxy replicas sharing Postgres and Redis, with enable_redis_auth_cache: true and real gpt-5.2 calls through the OpenAI API. Config:

model_list:
  - model_name: gpt-5.2
    litellm_params:
      model: gpt-5.2
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: sk-lit4350-master
  store_model_in_db: true

litellm_settings:
  cache: true
  enable_redis_auth_cache: true
  cache_params:
    type: redis
    url: redis://127.0.0.1:63350/0

Replica A on port 44350, replica B on port 44351, launched with python litellm/proxy/proxy_cli.py --config config.yaml --port <port>

Before the fix (commit 69a491e, current staging)

Generate a key with model_rpm_limit: {"gpt-5.2": 3}, prime BOTH replicas with one real completion each, then raise the limit on replica A:

$ curl -s -X POST http://127.0.0.1:44350/key/update \
    -H "Authorization: Bearer sk-lit4350-master" -H "Content-Type: application/json" \
    -d '{"key": "'$KEY'", "model_rpm_limit": {"gpt-5.2": 20}}'
{'model_rpm_limit': {'gpt-5.2': 20}}          # DB updated
$ docker exec lit4350-redis redis-cli TTL "$HASH"
-2                                             # Redis auth blob deleted by /key/update

Keep traffic on replica B (stale in-memory auth). The stale blob reappears in Redis with the OLD limit and a fresh TTL:

$ for i in $(seq 1 6); do curl -s -o /dev/null -w "%{http_code} " ... port 44351 ...; sleep 1; done
200 429 429 429 429 429                        # old RPM=3 still enforced
$ docker exec lit4350-redis redis-cli GET "$HASH" | jq .metadata
{"model_rpm_limit": {"gpt-5.2": 3}}            # STALE blob re-published by replica B
TTL: 59                                        # fresh full TTL

Deleting the key is worse: it keeps working indefinitely under continuous traffic:

$ curl -s -X POST http://127.0.0.1:44350/key/delete ... -d '{"keys": ["'$KEY'"]}'
{"deleted_keys":["sk-14hpA..."]}
$ # traffic every 20s on replica B, 60s+ after deletion:
t+1x20s replica-B: 200
t+2x20s replica-B: 200
t+3x20s replica-B: 200                         # deleted key still authenticates
  redis TTL now: 60                            # stale blob kept alive forever

After the fix (commit 365c8fb)

Same rig, same steps. The stale replica no longer republishes its in-memory token; Redis stays clean after /key/update:

$ # update RPM 3 -> 20 on replica A, then 6 requests on replica B:
200 429 429 429 429 429                        # B's in-memory copy lasts <= its 60s TTL
$ docker exec lit4350-redis redis-cli EXISTS "$HASH"
0                                              # no stale re-publish to Redis

Delete now converges within one auth-cache TTL even under continuous traffic (the exact feedback-loop condition, request interval far below the 60s TTL):

$ # /key/delete on replica A at 17:19:47, then traffic on replica B every 5s:
17:19:48 replica-B: 200
17:19:53 replica-B: 200
...
17:20:44 replica-B: 200
17:20:49 replica-B: 401                        # dead within 62s, no restarts, no manual Redis ops
$ docker exec lit4350-redis redis-cli EXISTS "$HASH"
0

DB-load caching is unchanged: after priming, the auth blob is present in Redis with the correct metadata and TTL, so the cache hit rate for steady-state traffic is the same as before

Independent e2e verification (Devin)

Screen recording

Recorded at commit f021f429ba with two live proxy replicas sharing PostgreSQL and Redis and real gpt-5.2 calls. Replica A populated Redis before replica B's first request, exercising the Redis-to-memory backfill; updating the key cleared Redis and ten stale-replica requests over 15 seconds did not recreate it. After deletion, replica B changed from HTTP 200 to 401 at 45.1 seconds, 61.9 seconds after hydration, with no restarts or manual Redis changes.

Type

🐛 Bug Fix

Changes

user_api_key_auth.py ran _cache_key_object after every successful key auth, writing the request's auth object back into DualCache regardless of whether it had been loaded from cache or from the DB. Both load paths already cache the pristine object at load time (IdentityStore._resolve_key and get_key_object), so this write only ever persisted request-mutated state, and under enable_redis_auth_cache with multiple replicas it created a feedback loop: a replica holding a stale in-memory token re-published it to shared Redis with a full TTL on every request, so /key/update, /key/delete, /key/block, and rate-limit changes never took effect while the key kept calling. In a multi-region topology (separate Redis per region, shared Postgres) there is no reliable way to kill a runaway key at all without coordinating a DB update, worker restarts, and Redis deletes across every region at once. This PR removes that post-auth write

The per-request key-spend writeback in proxy_server.update_cache republished the full cached auth object the same way after every priced request. It is removed entirely: an earlier revision made it a local-only in-memory write, but review flagged that even that can race an invalidation (an in-flight priced request reads the old object, /key/delete clears the cache, and the deferred task re-inserts the revoked object with a fresh TTL on that worker), so spend tracking now performs no auth-object cache write at all. Cross-pod spend is tracked by the spend:key:* Redis counters, which are unaffected, and budget enforcement reads through get_current_spend which prefers those counters. The soft-budget projected-limit alerting that lived in the same function is retained

Independent e2e verification caught a third leg of the same invariant: a replica whose in-memory copy of the auth object was populated by the DualCache Redis-to-memory backfill (rather than a DB load) held it for InMemoryCache's own 600s default because the backfill passed no ttl, so a deleted key converged at 586s instead of within the 60s auth TTL. The backfill in get_cache, async_get_cache, and async_batch_get_cache now injects default_in_memory_ttl exactly like async_set_cache already did

Regression tests: test_auth_does_not_rewrite_cached_key_object_back_into_cache runs the full auth builder against a primed UserApiKeyCache and asserts the cached payload is byte-for-byte untouched afterwards (fails on the old code), and test_spend_tracking_never_writes_the_auth_object_back asserts spend tracking performs zero auth-object cache writes (fails on the old code)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR stops stale auth objects from being re-written back into DualCache after every successful request, closing a multi-replica feedback loop where one worker could indefinitely re-publish an outdated token to shared Redis. It also injects default_in_memory_ttl into the Redis-to-memory backfill paths so that replicas whose in-memory copy came from a Redis read converge within the configured auth-cache TTL rather than the InMemoryCache default of 600 s.

• user_api_key_auth.py: removes the _cache_key_object call that followed every successful auth; the DB-load paths (IdentityStore._resolve_key / get_key_object) already cache the pristine object at load time, so the post-auth write only ever persisted mutated, request-scoped state.
• proxy_server.py: removes team_spend, team_member_spend, and spend writeback from _update_key_cache (spend is now tracked exclusively via spend:key:* Redis counters); adds an emptiness guard to avoid creating a no-op async_set_cache_pipeline task on token-only requests.
• dual_cache.py: injects default_in_memory_ttl into the Redis-to-memory backfill in get_cache, async_get_cache, and async_batch_get_cache, mirroring the existing behavior in async_set_cache.

Confidence Score: 5/5

Safe to merge — the changes are narrowly scoped deletions of post-auth and post-spend cache writes, with no new code paths added to the auth hot path.

All three legs of the fix (post-auth write removal, spend-writeback removal, Redis-to-memory TTL injection) are independently correct and each backed by a focused regression test. The PR description includes live multi-replica proof with Redis TTL observations, and an independent e2e recording. No new code paths are introduced in the auth critical path — only writes are removed. The values_to_update_in_cache emptiness guard is a clean symmetry fix. No existing tests were weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/caching/dual_cache.py	Injects default_in_memory_ttl into the Redis-to-memory backfill in get_cache, async_get_cache, and async_batch_get_cache, preventing Redis-loaded values from being held in memory for up to 600 s instead of the configured auth-cache TTL.
litellm/proxy/auth/user_api_key_auth.py	Removes the post-auth _cache_key_object call that was re-writing the (potentially mutated) auth object back into DualCache on every request, closing the stale-cache feedback loop.
litellm/proxy/proxy_server.py	Removes key-spend writeback from _update_key_cache (auth object no longer appended to values_to_update_in_cache) and adds an emptiness guard so no no-op pipeline task is created for token-only requests.
tests/test_litellm/caching/test_dual_cache.py	Adds two new tests verifying that the Redis-to-memory backfill in async_get_cache and async_batch_get_cache respects default_in_memory_ttl.
tests/test_litellm/proxy/auth/test_user_api_key_auth.py	Adds test_auth_does_not_rewrite_cached_key_object_back_into_cache — runs full auth builder against a primed cache and asserts the cached payload is byte-for-byte unchanged afterwards; task synchronization uses asyncio.wait with a 5-second cap.
tests/test_litellm/proxy/test_proxy_server.py	Adds test_spend_tracking_never_writes_the_auth_object_back — patches async_set_cache_pipeline and async_set_cache, invokes update_cache, drains async tasks, then asserts zero pipeline writes targeting the hashed token.

_{Reviews (5): Last reviewed commit: "fix(auth): stop re-caching stale auth ob..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/caching/dual_cache.py	66.66%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai please review the current head d154fd3

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit4350_stale_auth_recache (eaca0bf) with litellm_internal_staging (2162da5)}

[yassin-berriai]

@greptileai please review the current head 0f2ef6b

[yassin-berriai]

@greptileai please review the current head f021f42

[yassin-berriai]

@greptileai please review the current head 7f200ab

[yassin-berriai]

Consolidated into #33565, which now carries this branch's approach (post-auth re-cache removed outright, key auth object never written back by spend tracking) together with the local-only split for the remaining spend writebacks, the shared global proxy spend scalar, and the DualCache backfill TTL fix. LIT-4350 is marked as a duplicate of LIT-4219, which #33565 resolves