fix(proxy): invalidate cached project object on project update and delete

[ryan-crabbe-berri]

TLDR

Problem this solves:

• Project allowlist changes were invisible to auth for up to 60s
• Stale cached project with empty allowlist bypassed project model restrictions
• Other workers kept their stale in-memory copy even after eviction
• Deleted projects stayed enforceable from cache

How it solves it:

• /project/update and /project/delete now evict the cached project
• Eviction broadcasts the cache key over coordination Redis pub/sub
• Every worker drops its local copy and refetches on next auth read
• Eviction is best-effort: cache backend errors never fail the committed write

Relevant issues

Linear ticket

Resolves LIT-3803

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

All runs are against live local proxies backed by Postgres, making real Groq calls. Setup: a team with two models, a project under it created with no allowlist, and a virtual key bound to the project with no models assigned

curl -s http://localhost:4173/team/new -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"team_alias":"lit3803-qa-team","models":["gpt-oss-120b","gpt-oss-20b"]}'
curl -s http://localhost:4173/project/new -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"project_alias":"lit3803-qa-project","team_id":"6f39c40b-..."}'
curl -s http://localhost:4173/key/generate -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"key_alias":"lit3803-qa-key","team_id":"6f39c40b-...","project_id":"7629ec45-..."}'

Single worker: stale cache bypass

The repro sequence is: make one completion with the key so auth caches the project (still no allowlist), then add the allowlist, then immediately call a team model outside it

curl -s http://localhost:4173/v1/chat/completions -H 'Authorization: Bearer sk-...' -H 'Content-Type: application/json' \
  -d '{"model":"gpt-oss-20b","messages":[{"role":"user","content":"Say WARMUP and nothing else"}]}'
curl -s http://localhost:4173/project/update -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"project_id":"7629ec45-...","models":["gpt-oss-120b"]}'
curl -s -w "\nHTTP %{http_code}\n" http://localhost:4173/v1/chat/completions -H 'Authorization: Bearer sk-...' -H 'Content-Type: application/json' \
  -d '{"model":"gpt-oss-20b","messages":[{"role":"user","content":"this should be blocked"}]}'

Before, with the un-fixed endpoints (parent commit c2c795f), the out-of-allowlist call went straight through to the provider:

{"id":"chatcmpl-ac54496f-fe29-4b05-894c-1317953d444e","created":1785979061,"model":"gpt-oss-20b","object":"chat.completion", ...}
HTTP 200

After (70b3ed3), the same sequence is blocked immediately:

{"error":{"message":"project not allowed to access model. This project can only access models=['gpt-oss-120b']. Tried to access gpt-oss-20b","type":"project_model_access_denied","param":"model","code":"403"}}
HTTP 403

Multi worker: cross-worker broadcast

Two proxies (A on :4173, B on :4174) share the same Postgres and a coordination Redis (general_settings.coordination_redis). Both warm their local project cache with one completion each while the allowlist is still empty, the allowlist is added via A, and the out-of-allowlist model is called via B immediately after

curl -s http://localhost:4173/v1/chat/completions -H 'Authorization: Bearer sk-...' -d '{"model":"gpt-oss-20b", ...}'   # warm A
curl -s http://localhost:4174/v1/chat/completions -H 'Authorization: Bearer sk-...' -d '{"model":"gpt-oss-20b", ...}'   # warm B
curl -s http://localhost:4173/project/update -H 'Authorization: Bearer sk-1234' \
  -d '{"project_id":"7629ec45-...","models":["gpt-oss-120b"]}'                                                          # update via A
curl -s -w "\nHTTP %{http_code}\n" http://localhost:4174/v1/chat/completions -H 'Authorization: Bearer sk-...' \
  -d '{"model":"gpt-oss-20b","messages":[{"role":"user","content":"this should be blocked"}]}'                          # immediately via B

Before (70b3ed3, eviction without broadcast), worker B kept serving its stale in-memory copy and the call went through to the provider:

{"id":"chatcmpl-7d4d31f1-2c82-4c32-9ee5-e252a46c43a0","created":1785980831,"model":"gpt-oss-20b","object":"chat.completion", ...}
HTTP 200

After (ad00a37), worker B blocks immediately because the broadcast evicted its local copy:

{"error":{"message":"project not allowed to access model. This project can only access models=['gpt-oss-120b']. Tried to access gpt-oss-20b","type":"project_model_access_denied","param":"model","code":"403"}}
HTTP 403

and the allowed model still completes for real via B:

{"id":"chatcmpl-f8f4a753-0ca6-4358-a8e1-12cbd82444aa", ... "message":{"content":"ALLOWED","role":"assistant", ...}
HTTP 200

The enforcement itself lives in common_checks in the shared auth layer, upstream of any specific LLM endpoint, so the same 403 applies to /v1/chat/completions, /v1/messages, and /v1/responses alike

Type

🐛 Bug Fix

Changes

Auth reads projects through get_project_object, which serves project_id:{id} from user_api_key_cache with a 60s TTL and no freshness check, and nothing ever evicted that entry: neither /project/update nor /project/delete touched the cache. A project cached while its models list was still empty (warmed during key creation or by any completion made before the admin set the allowlist) kept an empty allowlist in cache, so _run_project_checks skipped can_project_access_model and a project-bound key could call team models outside the project allowlist until the TTL expired. This is the bypass reported in the ticket. Blocked status and budget fields had the same staleness window, and a deleted project stayed enforceable from cache

The fix adds delete_cached_project_object next to get_project_object in auth_checks.py, shares the cache-key derivation between the two via _project_cache_key so they can't drift, and calls the eviction after the DB write in update_project and delete_project. UserApiKeyCache is a DualCache, so the eviction clears both the in-memory layer and Redis where configured

Since DualCache reads return local in-memory hits before consulting Redis, evicting locally plus Redis still leaves every other worker serving its own stale copy. The new auth_cache_invalidation_pubsub module closes that: eviction publishes the cache key on a coordination Redis channel (litellm_proxy.auth_cache_invalidation, namespace-aware, following the existing config_sync_pubsub pattern), and an AuthCacheInvalidationSubscriber on every worker deletes the local in-memory entry so the next auth read refetches from the DB. The subscriber starts whenever a coordination Redis is configured, independent of store_model_in_db, and deployments without Redis fall back to the TTL exactly as before

Eviction and publish are both best-effort: the DB write has already committed when they run, so a cache backend error logs a warning instead of turning a successful update into a 500 or aborting the remaining ids in /project/delete

Tests: two endpoint regressions prove update and delete evict the cached project (both fail without the fix), the enforcement test the ticket asked for proves a key with models: [] passes the key layer unrestricted while _run_project_checks returns 403 project_model_access_denied, pub/sub tests cover publish (channel, namespacing, no-redis no-op, error swallowing) and the subscriber (deletes the local entry on message, ignores malformed messages), and endpoint tests prove update survives a failing cache backend and eviction publishes the invalidation

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR invalidates cached project objects after project updates and deletions, and broadcasts invalidations so each worker removes its local cached copy

• Shares project cache-key construction between reads and eviction
• Makes post-mutation cache eviction and publication best-effort
• Adds a coordination Redis subscriber with startup, reconnect, and shutdown handling
• Adds endpoint, authorization, and pub/sub regression coverage

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename	Overview
enterprise/litellm_enterprise/proxy/management_endpoints/project_endpoints.py	Project update and delete now evict the affected cached project after the database mutation
litellm/proxy/auth/auth_checks.py	Project cache keys are centralized and project eviction tolerates cache failures before broadcasting invalidation
litellm/proxy/common_utils/auth_cache_invalidation_pubsub.py	Adds namespaced best-effort invalidation publication and worker-local cache eviction through Redis pub/sub
litellm/proxy/proxy_server.py	Integrates the invalidation subscriber with proxy startup and shutdown
tests/enterprise/litellm_enterprise/proxy/management_endpoints/test_project_endpoints_prisma.py	Covers update and delete eviction, cache-backend failure tolerance, and invalidation publication
tests/test_litellm/proxy/auth/test_auth_checks.py	Verifies project-level model restrictions remain enforced when the key allowlist is empty
tests/test_litellm/proxy/common_utils/test_auth_cache_invalidation_pubsub.py	Covers publication, namespacing, malformed messages, Redis failures, and subscriber-driven local eviction

_{Reviews (3): Last reviewed commit: "fix(lint): sort auth cache invalidation ..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 83.82353% with 22 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...oxy/common_utils/auth_cache_invalidation_pubsub.py	86.53%	14 Missing ⚠️
litellm/proxy/auth/auth_checks.py	36.36%	7 Missing ⚠️
litellm/proxy/proxy_server.py	95.23%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[ryan-crabbe-berri]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit_3803_project_cache_invalidation (709ecc1) with litellm_internal_staging (e1717c5)}

[ryan-crabbe-berri]

@greptileai re review update score