perf(auth): read user, team, membership, org, project and spend counters in one MGET, one query and one pipeline

[devin-ai-integration]

TLDR

Problem this solves:

• Auth does one Redis GET per object it reads (user, team, membership, org, project, budget) and a Postgres query each when cold
• Spend admission checks do one more Redis GET per entity counter (key, user, team, member, org, end user)
• Cold key: 43 Redis round trips and 18 Postgres statements in auth alone; hot key: 6 Redis round trips
• Prod traces showed these serial round trips as most of the pre-LLM latency

How it solves it:

• One Redis MGET for every object the key references, one raw SQL join for whatever missed, one pipeline write-back
• Hot path: one MGET for all admission spend counters, shared with the reseed path
• The per-object getters stay the readers and the enforcement point; the prefetch only warms the cache and falls back on any failure
• Denied requests pay nothing new: the prefetch runs only after the key passes the model access check

User Flow

Before: a developer's first request on a virtual key waits on a chain of cache and DB lookups before the model is even called

1. They send POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5.4-mini", "messages": [...]} and a virtual key that belongs to a user, a team, an org and has an end user
2. The gateway checks the key, then the user, then the team, then the team membership, then the org, then each budget, then each spend counter, each as its own network hop
3. The response arrives with HTTP 200 and the usual chatcmpl-... id, tens of milliseconds later than the model call alone would take
4. The next request on the same key on a warm proxy still spends six sequential Redis hops in auth

After: the same request does the same checks in the same order, with the reads batched

1. They send the same POST https://litellm-domain/v1/chat/completions with the same key
2. The gateway checks the key, then fetches the user, team, membership, org, project and budgets in one batch (and one DB query if none were cached), then all spend counters in one batch
3. The response arrives with HTTP 200 and the same chatcmpl-... shape; a key without access to the model still gets the identical HTTP 403 body it got before
4. The next request on the same key on a warm proxy spends one Redis hop in auth instead of six

Relevant issues

Part of the request I/O work tracked in #40539

Linear ticket

Resolves LIT-7634

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Design

litellm/proxy/auth/auth_object_prefetch.py is new. AuthObjectRefs.from_token(valid_token) lists the ids the checks will read. prefetch_auth_objects then does at most three I/Os for all of them:

missing  = entries not already in the in-memory user_api_key_cache
found    = redis.async_batch_get_cache(sorted cache keys of missing)      # 1 MGET
rows     = prisma.db.query_first(_SQL, user_id, team_id, user_id, org_id, project_id)   # 1 query, only for what still missed
objects  = validate each row into the same pydantic model the getter would build (LiteLLM_UserTable, LiteLLM_TeamTableCachedObj, ...)
cache.redis_cache.async_set_cache_pipeline_with_ttls(objects with their own TTLs)     # 1 pipeline

_SQL is one statement with five scalar subselects, each building the object as jsonb with its budget, model table and object permission nested the way the existing per-object queries return them. Nullable scalar lists are coalesced to [] because prisma reads a NULL array as an empty list and to_jsonb reads it as null. Cache keys and TTLs are the ones the getters already use (team_id:{id}, org_id:{id} and org_id:{id}:with_budget, project_id:{id}, team_membership:{team}:{user} at 5 s, the membership reservation key, and the plain user id), so the getters hit the in-memory cache on the very next line and nothing about invalidation or the management team freshness check changes. The prefetch is best effort: any Redis or DB failure logs a warning and the getters fetch exactly as they do today, so enforcement never depends on it

_prefetch_referenced_auth_objects is called from _user_api_key_auth_builder right after _enforce_key_and_fallback_model_access, so a key that cannot call the requested model is denied before any of this runs

litellm/proxy/spend_tracking/spend_counter_batch.py is new. spend_counter_batch_scope(redis) opens a task-local SpendCounterBatch for the auth span. bind_admission_counter_keys(token, end_user_id) registers the key, user, team, team member, org and end user counter keys. The first reader triggers one MGET for all bound keys; keys bound later join the next MGET. async_batch_get_cache maps a clean miss to None and drops a key only when Redis failed, so the batch treats a present None as an authoritative miss and an absent key as "read Redis yourself". SpendCounterReseed.coalesced and proxy_server.read_spend_counter_cache_value consult the active batch before issuing their own GET, which is what removes the per-counter GETs from the hot path

RedisCache.async_set_cache_pipeline_with_ttls is a small addition next to async_set_cache_pipeline: the same non-transactional pipeline, but each entry carries its own TTL, since the objects above have different lifetimes

Tests: tests/test_litellm/proxy/auth/test_auth_object_prefetch.py counts Redis round trips and DB queries per cache regime with fakes (cold, Redis warm, hot, partial hit, invalidation, missing column, Redis failure, no Redis, no DB). tests/proxy_behavior/auth/test_auth_object_prefetch.py runs the raw join on the real Postgres of the proxy-behavior CI shard: a user in two teams with different member budgets gets the requested team's membership row, and the getters are then served from cache with the DB mocked dead. tests/test_litellm/proxy/spend_tracking/test_spend_counter_batch.py covers the batch (one MGET, incremental binding, authoritative miss, Redis failure fallback, task isolation) and test_user_api_key_auth.py asserts the prefetch does not run for a key denied on model access

Benchmarks

Local rig: one proxy process per run, Postgres with log_statement=all (statements attributed by application_name), Redis MONITOR (commands grouped into round trips by an 80 µs gap). Before is litellm_internal_staging at the branch point (1be9306), after is this branch. Model calls in this matrix are mocked so the numbers isolate gateway I/O. Everything below is single process on one box and is not a production fleet number

Regimes: cold = Redis flushed and fresh proxy; hot = second request on the same proxy; idle = the same key after the in-memory TTL lapsed; expired = after the Redis entries expired; denied = a key that may not call the requested model. "rt" = estimated round trips

chat (Rust admission off), before -> after

regime	status	Redis rt auth	pre-call	post-call	db writer	Redis rt total	Redis cmds	PG auth	pre-call	post-call	db writer	PG total	latency ms
cold	200	43 -> 24	25	30	1	99 -> 80	151 -> 135	18 -> 11	0	3 -> 4	15 -> 14	36 -> 29	142.1 -> 131.9
hot	200	6 -> 1	22	25	1	54 -> 49	90 -> 85	0	0	2 -> 1	15	17 -> 16	26.6 -> 27.4
idle	200	20 -> 4	22	27 -> 24	1	70 -> 51	112 -> 88	10 -> 1	0	3	15	28 -> 19	78.0 -> 36.3
expired	200	21 -> 8	22	24	1	68 -> 55	107 -> 92	8 -> 1	0	2 -> 3	15	25 -> 19	49.9 -> 50.6
denied	403	3	0	0	1 -> 0	4 -> 3	4 -> 3	1	0	1	11 -> 0	13 -> 2	18.9 -> 17.8

Concurrent load, 8 in flight, 200 requests, before -> after: p50 116.8 -> 118.0 ms, p95 139.4 -> 146.7 ms, p99 484.3 -> 555.4 ms, 60.5 -> 58.6 rps, Redis commands/request 86.4 -> 81.4, PG statements/request 0.15 -> 0.17

messages (Rust admission off), before -> after

regime	status	Redis rt auth	pre-call	post-call	db writer	Redis rt total	Redis cmds	PG auth	pre-call	post-call	db writer	PG total	latency ms
cold	200	43 -> 23	25	20	1	89 -> 69	136 -> 118	18 -> 11	0	3 -> 2	15	36 -> 28	155.2 -> 142.8
hot	200	6 -> 1	22	20 -> 21	1	49 -> 45	79 -> 76	0	0	1 -> 2	15	16 -> 17	24.7 -> 25.4
idle	200	12 -> 4	22	21 -> 20	1	56 -> 47	85 -> 79	6 -> 1	0	4	16 -> 15	26 -> 20	42.0 -> 36.3
expired	200	17 -> 8	22	21	1	61 -> 52	90 -> 85	6 -> 1	0	4 -> 3	16 -> 15	26 -> 19	46.8 -> 42.6
denied	403	3	0	0	1	4	4	1	0	0 -> 1	12	13 -> 14	19.0 -> 17.7

Concurrent load, 8 in flight, 200 requests, before -> after: p50 112.7 -> 110.5 ms, p95 143.1 -> 141.0 ms, p99 496.6 -> 464.5 ms, 61.1 -> 63.7 rps, Redis commands/request 77.4 -> 72.4, PG statements/request 0.14 -> 0.1

chat (Rust admission on), before -> after

regime	status	Redis rt auth	pre-call	post-call	db writer	Redis rt total	Redis cmds	PG auth	pre-call	post-call	db writer	PG total	latency ms
cold	200	43 -> 24	25 -> 26	33 -> 30	1	102 -> 81	155 -> 135	18 -> 11	0	3	14	35 -> 28	439.4 -> 460.0
hot	200	6 -> 1	22	25	1	54 -> 49	90 -> 85	0	0	2 -> 3	15 -> 16	17 -> 19	25.8 -> 30.0
idle	200	12 -> 4	22	26 -> 27	1	61 -> 54	96 -> 92	6 -> 1	0	4 -> 3	16 -> 15	26 -> 19	48.7 -> 44.3
expired	200	17 -> 8	22	25 -> 24	1	65 -> 55	99 -> 92	6 -> 1	0	4 -> 2	16 -> 15	26 -> 18	46.6 -> 44.4
denied	403	3	0	0 -> 1	0 -> 1	3 -> 5	3 -> 6	1	0	0 -> 2	0 -> 12	1 -> 15	17.7 -> 18.5

Concurrent load, 8 in flight, 200 requests, before -> after: p50 118.7 -> 131.5 ms, p95 170.4 -> 160.2 ms, p99 492.2 -> 579.8 ms, 58.8 -> 53.3 rps, Redis commands/request 86.5 -> 81.3, PG statements/request 0.17 -> 0.1

messages (Rust admission on), before -> after

regime	status	Redis rt auth	pre-call	post-call	db writer	Redis rt total	Redis cmds	PG auth	pre-call	post-call	db writer	PG total	latency ms
cold	200	43 -> 24	25	20	1	89 -> 70	136 -> 120	18 -> 11	0	3 -> 2	15 -> 14	36 -> 27	446.7 -> 430.9
hot	200	6 -> 1	22	21	1	50 -> 45	81 -> 76	0	0	2	15	17	61.7 -> 27.1
idle	200	12 -> 4	28 -> 22	15 -> 20	1	56 -> 47	85 -> 79	6 -> 1	0	4	16	26 -> 21	199.2 -> 38.6
expired	200	17 -> 8	22	23 -> 22	1	63 -> 53	92 -> 87	6 -> 1	0	2 -> 4	15	23 -> 20	48.3 -> 58.5
denied	403	3	0	0	0	3	3	1	0	2	0	3	17.3 -> 19.3

Concurrent load, 8 in flight, 200 requests, before -> after: p50 115.2 -> 113.0 ms, p95 296.8 -> 136.8 ms, p99 532.9 -> 496.0 ms, 57.2 -> 62.3 rps, Redis commands/request 77.6 -> 72.3, PG statements/request 0.34 -> 0.09

How to read it: the auth columns are the ones this PR touches and they move the same way in every leg (Redis 43 -> 24 cold, 6 -> 1 hot, 12/20 -> 4 idle, 17/21 -> 8 expired; Postgres 18 -> 11 cold, 6 to 10 -> 1 idle and expired). The remaining cold auth I/O is the key lookup itself, the end user, the model access group budget and the spend counter reseeds, which are separate call sites (next PR). Pre-call and post-call columns are unchanged by design. The "db writer" column is the background spend writer flushing whatever happened to land in the capture window and is noise for the denied rows. Single-request latency on a warm box is dominated by the mocked model call plus Python work, so it moves by a few ms at most except where the removed I/O was serial and cold (idle 78 -> 36 ms). The 8-in-flight p50/p95 vary between runs by more than the auth savings on a single box, so this PR claims fewer round trips and statements, not a fixed latency win

Auth-phase Redis commands by call site, cold, chat, Rust on (before -> after):

call site	command	before	after
auth:user	GET / SET	1 / 1	0 / 1
auth:team	GET / SET / DEL	1 / 1 / 1	0 / 1 / 0
auth:team_membership	GET / MGET / SET	2 / 0 / 2	0 / 1 / 2
auth:org	GET / SET	1 / 1	0 / 2
spend_counter	GET / MGET / SET	11 / 0 / 4	1 / 1 / 5
auth:key_object	GET / SET	2 / 1	2 / 1
auth:end_user	GET / SET	3 / 2	3 / 2
auth:tag_or_access_group	GET / SET	2 / 1	2 / 1

Auth-phase Postgres statements, cold, chat, Rust on (before -> after): the one combined LiteLLM_UserTable + TeamTable + TeamMembership + OrganizationTable + OrganizationMembership + ProjectTable + BudgetTable + ModelTable + ObjectPermissionTable select replaces SELECT LiteLLM_UserTable (1 -> 0), SELECT LiteLLM_TeamTable (2 -> 1), SELECT LiteLLM_TeamMembership (3 -> 2), SELECT LiteLLM_OrganizationTable (2 -> 1), SELECT LiteLLM_OrganizationMembership (1 -> 0) and SELECT LiteLLM_BudgetTable (4 -> 1)

Screenshots / Proof of Fix

Live proxy on the same rig (Postgres + Redis in docker, LITELLM_RUST unset), real provider calls to openai/gpt-5.6 (alias real-chat) and anthropic/claude-sonnet-5 (alias real-claude). Fixture: an org (live-org), a team in it with a team budget, a user with a max budget who is a team member with a member budget, an end user with its own budget, and a key $KEY bound to that user and team. $KEY_DENIED is a second key on the same user and team allowed only real-chat. Redis is flushed and the proxy restarted before each side's first request. The third line of every case is what Redis MONITOR and Postgres log_statement=all recorded for that one request, split by phase

Before (19c8553)

/v1/chat/completions, cold caches (Redis flushed, fresh proxy)

1. curl -s http://127.0.0.1:20900/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "real-chat", "messages": [{"role": "user", "content": "hi"}], "user": "authio-enduser"}'
2. HTTP 200, 1315.1 ms: {"id":"chatcmpl-EN9...","created":1789185...,"model":"real-chat","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi! How can I help?","role":"assistant",...
3. Redis MONITOR for this request: 128 commands in about 94 round trips (auth 43, pre-call 19, post-call 31, db writer 1); Postgres log_statement=all: 34 statements (auth 18, pre-call 0, post-call 2, db writer 14)

/v1/chat/completions, second request on the same proxy

1. same curl as above
2. HTTP 200, 961.8 ms, same response shape
3. Redis MONITOR: 70 commands in about 48 round trips (auth 6, pre-call 17, post-call 24, db writer 1); Postgres: 28 statements (auth 0, pre-call 13, post-call 0, db writer 15)

/v1/messages, first request for that route on the same proxy

1. curl -s http://127.0.0.1:20900/v1/messages -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "real-claude", "max_tokens": 16, "messages": [{"role": "user", "content": "hi"}], "metadata": {"user_id": "authio-enduser"}}'
2. HTTP 200, 1139.7 ms: {"model":"real-claude","id":"msg_011Cex...","type":"message","role":"assistant","content":[{"type":"text","text":"Hi there! How can I help you today?"}],...
3. Redis MONITOR: 80 commands in about 56 round trips (auth 12, pre-call 16, post-call 27, db writer 1); Postgres: 24 statements (auth 6, pre-call 0, post-call 3, db writer 15)

/v1/messages, second request

1. same curl as above
2. HTTP 200, 1306.4 ms, same response shape
3. Redis MONITOR: 69 commands in about 47 round trips (auth 6, pre-call 16, post-call 24, db writer 1); Postgres: 17 statements (auth 0, pre-call 0, post-call 2, db writer 15)

key without access to the model (denied)

1. curl -s http://127.0.0.1:20900/v1/chat/completions -H "Authorization: Bearer $KEY_DENIED" -H 'Content-Type: application/json' -d '{"model": "real-claude", "messages": [{"role": "user", "content": "hi"}], "user": "authio-enduser"}'
2. HTTP 403, 18.4 ms: {"error":{"message":"key not allowed to access model. This key can only access models=['real-chat']. Tried to access real-claude","type":"key_model_access_denied","param":"model","code":"403"}}
3. Redis MONITOR: 3 commands in about 3 round trips (auth 3, pre-call 0, post-call 0, db writer 0); Postgres: 4 statements (auth 1, pre-call 0, post-call 1, db writer 2)

After (e75b541)

/v1/chat/completions, cold caches (Redis flushed, fresh proxy)

1. curl -s http://127.0.0.1:20900/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "real-chat", "messages": [{"role": "user", "content": "hi"}], "user": "authio-enduser"}'
2. HTTP 200, 2555.2 ms: {"id":"chatcmpl-ENE8zvBFcIj2mJ5xQKtOIPwy5wLy0","created":1789204621,"model":"real-chat","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi! How can I help?","role":"assistant","provider_specific...
3. Redis MONITOR for this request: 116 commands in about 77 round trips (auth 22, pre-call 19, post-call 30, db writer 2); Postgres log_statement=all: 27 statements (auth 11, pre-call 0, post-call 0, db writer 14)

/v1/chat/completions, second request on the same proxy

1. curl -s http://127.0.0.1:20900/v1/chat/completions -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "real-chat", "messages": [{"role": "user", "content": "hi"}], "user": "authio-enduser"}'
2. HTTP 200, 1685.5 ms: {"id":"chatcmpl-ENE9PplIBYBvXSP3nQ8uF3p6zj0hb","created":1789204647,"model":"real-chat","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hi! How can I help?","role":"assistant","provider_specific...
3. Redis MONITOR for this request: 66 commands in about 44 round trips (auth 1, pre-call 16, post-call 22, db writer 1); Postgres log_statement=all: 18 statements (auth 0, pre-call 1, post-call 0, db writer 14)

/v1/messages, first request for that route on the same proxy

1. curl -s http://127.0.0.1:20900/v1/messages -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "real-claude", "max_tokens": 16, "messages": [{"role": "user", "content": "hi"}], "metadata": {"user_id": "authio-enduser"}}'
2. HTTP 200, 1112.2 ms: {"model":"real-claude","id":"msg_011CeyJ3xgn3KFKGMzXZZutx","type":"message","role":"assistant","content":[{"type":"text","text":"Hi there! How can I help you today?"}],"container":null,"stop_reason":"end_turn","stop_sequence":null,"stop_det...
3. Redis MONITOR for this request: 72 commands in about 45 round trips (auth 4, pre-call 16, post-call 22, db writer 1); Postgres log_statement=all: 20 statements (auth 2, pre-call 1, post-call 0, db writer 14)

/v1/messages, second request

1. curl -s http://127.0.0.1:20900/v1/messages -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model": "real-claude", "max_tokens": 16, "messages": [{"role": "user", "content": "hi"}], "metadata": {"user_id": "authio-enduser"}}'
2. HTTP 200, 1161.8 ms: {"model":"real-claude","id":"msg_011CeyJ5pZgG7pHHUaJ3A63U","type":"message","role":"assistant","content":[{"type":"text","text":"Hi there! How can I help you today?"}],"container":null,"stop_reason":"end_turn","stop_sequence":null,"stop_det...
3. Redis MONITOR for this request: 64 commands in about 42 round trips (auth 1, pre-call 16, post-call 22, db writer 1); Postgres log_statement=all: 15 statements (auth 0, pre-call 0, post-call 0, db writer 14)

key without access to the model (denied)

1. curl -s http://127.0.0.1:20900/v1/chat/completions -H "Authorization: Bearer $KEY_DENIED" -H 'Content-Type: application/json' -d '{"model": "real-claude", "messages": [{"role": "user", "content": "hi"}], "user": "authio-enduser"}'
2. HTTP 403, 24.3 ms: {"error":{"message":"key not allowed to access model. This key can only access models=['real-chat']. Tried to access real-claude","type":"key_model_access_denied","param":"model","code":"403"}}
3. Redis MONITOR for this request: 6 commands in about 6 round trips (auth 5, pre-call 0, post-call 0, db writer 1); Postgres log_statement=all: 16 statements (auth 3, pre-call 0, post-call 0, db writer 12)

Same rig, same fixture, same five requests as the Before block, re-captured after the last runtime commit (e75b541). The auth column is what this PR changes and it matches the earlier capture at 897d561 exactly (cold chat 22 Redis round trips and 11 statements, hot chat 1 and 0, messages 4 and 2 then 1 and 0). Latency differs run to run because the provider and the shared box do; the denied request in this capture also shows the pre-existing organization budget check refreshing org_id:live-org:with_budget (2 extra Redis commands, 2 extra statements) because its in-memory entry had lapsed, which the base branch does on the same schedule

Tests and gates

State at e75b541:

• make check: PASS at e75b541 (ruff, ruff strict gate, type-discipline gate, basedpyright gate, test-quality gate, OpenAPI/dashboard type sync); no budget file touched
• CI: 86 check runs at e75b541, 85 success and 1 skipped (the skipped one is a path-filtered job), including codecov/patch at 97.68% of the diff against an 80.69% target, CodSpeed "Performance Gate Passed" and buildkite e2e; nothing pending or failed
• Greptile: Confidence Score 5/5, "no outstanding correctness, security, or repository-rule findings", last reviewed commit e75b541 (re-triggered with @greptileai after the final push, summary refreshed 09:16 UTC)
• Bugbot: "found no new issues", review posted for commit e75b541 (triggered once per push from the reviewer's account, never twice on one SHA). Veria: no check run or comment from it exists on this PR, so there is no Veria result to report
• Bugbot history: its first pass found three things in the raw join. Nested NULL arrays becoming JSON null was fixed in 897d561. Team model_aliases lost through to_jsonb (the physical column is aliases, mapped by prisma) and the user row hydrating budgets and permissions that get_user_object does not load were marked resolved by the bot but were still in the code; both are fixed in e75b541, with a real-Postgres regression test (test_join_reads_team_model_aliases_from_the_mapped_column) that fails on the previous SQL and passes now
• Mutation check: 20 hand-written mutants, each applied to the working tree and run against the focused tests, all 20 killed. Prefetch: hot regime always MGETs (4 tests fail), Redis hits not copied into memory (3), cold rows never written back (8), write-back skips the pipeline (4), Redis miss never falls through to the DB (8), invalid row cached anyway (1), membership join ignores the requested team (2, real Postgres), prefetch moved before the model access check (2), alias column left as aliases (1, real Postgres), model_aliases filled from NULL (1, real Postgres), memberships subselect dropped (3, real Postgres). Spend batch: closed batch still answers reads (2), late-bound keys never join the MGET (9), fetched keys re-read on every access (4), MGET failure propagates instead of handing the read back (3), release does not close the batch (2), org counter dropped from the admission keys (4), reseed ignores the batched value (1), reseed re-reads Redis after a batched clean miss (2). Redis: pipeline writes drop the TTL (1)
• Concernloop: every bot thread is fixed or answered as above, Greptile and Bugbot were re-triggered on the tip after the last push; /qa is the real-provider before/after run above; /live-pr-risk is the section below and its checkbox at the end of this description

Live risk (what else depends on the changed code, and what was driven live)

Changed contracts: RedisCache gains async_set_cache_pipeline_with_ttls next to async_set_cache_pipeline; nothing existing changes signature. _user_api_key_auth_builder gains one awaited call after _enforce_key_and_fallback_model_access whose every failure path is caught and logged, so a denied key, a key with no team or user, a proxy without Redis and a proxy without a DB all take the same code path as before. The getters (get_user_object, get_team_object, get_team_membership, get_org_object, get_project_object) are untouched and remain the enforcement point; the prefetch writes the same cache keys with the same TTLs they use, so key, team, user and org invalidation (/key/update, /team/update, /user/update, deletion) keeps working because it deletes those same keys. The user row is now shaped exactly like get_user_object's include={"organization_memberships": True} result, and the team row carries litellm_model_table.model_aliases, so team_model_aliases(team_object) (the JWT auth path) and any reader of team.litellm_model_table get the same value from a prefetched team as from a getter-fetched one (the virtual key path reads team_model_aliases from the key's own combined_view query, which is untouched). read_spend_counter_cache_value and SpendCounterReseed.coalesced consult the task-local batch first and fall back to their existing Redis read when there is no batch or the batch has no answer, so the post-call spend path and the background reseed job, which run outside the auth span, are unchanged

Driven live on both sides (base 19c8553 vs head e75b541, one proxy process per side on the same Postgres and Redis, real OpenAI and Anthropic calls): /v1/chat/completions and /v1/messages cold and on a second request, a key denied on model access (identical HTTP 403 body on both sides), the org, team, membership, user, end-user and member budgets all present on the key, and the spend flush of the db writer. Statuses and bodies match between sides; only the auth-phase Redis and Postgres counts differ, as tabulated above. The synthetic matrix adds Rust admission on and off, the idle (in-memory TTL lapsed) and expired (Redis TTL lapsed) regimes and 8-way concurrency

Covered by tests but not driven live: a Redis outage during the MGET or the write-back pipeline (getters fetch as before, test_auth_object_prefetch.py), a prefetched row that fails pydantic validation (not cached, getter fetches it, same file), a proxy with no Redis or no DB, a user in two teams (the join returns the requested team's membership, tests/proxy_behavior/auth), a team with model aliases (same file), and an MGET failure inside the spend batch (test_spend_counter_batch.py). Not verified: JWT and custom-auth paths do not call _user_api_key_auth_builder's prefetch branch and were not exercised; a Redis deployment that rejects pipeline(transaction=False) (redis-py has supported it since 2.x). No lockfile, pyproject.toml, Dockerfile or schema.prisma change, so there is no packaging or migration leg

Type

🆕 New Feature

Caveats (if any)

Medium

• The raw join has to undo prisma's @map renames by hand: LiteLLM_ModelTable.aliases is exposed as model_aliases. Any future @map on a table the join reads needs the same treatment or the value is dropped at validation and the getter fetches the row the old way
• The prefetch warms the shared cache rather than handing typed objects to common_checks; the getters remain the single reader. Passing an immutable bundle through common_checks would save no further I/O and is left out to keep the diff reviewable
• The raw join duplicates the column-to-model mapping the per-object queries do implicitly. A new nullable array column on the user, team, org, project, budget or object permission table must be added to the COALESCE list or that row fails validation and the getter fetches it the old way (logged at warning, no behavior change)
• Concurrent p50/p95 on the single-box rig vary more between runs than the auth savings; treat the round trip and statement columns as the measurement

Low

• The spend counter batch is scoped to the auth span only; the post-call warm checks and increments still do one GET per counter and are the subject of the next PR

• The remaining cold auth I/O (key lookup, end user, model access group budget, counter reseed) is untouched here

• e75b541 passes /live-pr-risk

---

Note

Medium Risk
Touches the auth and budget-admission read path and changes spend-counter cache miss semantics when the batch is active; enforcement still relies on existing getters, but incorrect prefetch SQL or batch behavior could affect latency or spend checks until fallback runs.

Overview
Batches proxy auth cache I/O so a single request warms user, team, membership, org, and project entries with one Redis MGET, one combined Postgres query when cold, and one pipeline SET (per-key TTLs via new async_set_cache_pipeline_with_ttls). Prefetch runs only after the key passes model access; failures are swallowed and existing per-object getters still enforce.

Spend admission reads during auth are collapsed into a task-local SpendCounterBatch (one MGET for key/user/team/member/org/end-user counters). read_spend_counter_cache_value, SpendCounterReseed.coalesced, and common_checks use the batch scope so batched Redis misses stay authoritative and reseed skips redundant GETs.

^{Reviewed by Cursor Bugbot for commit e75b541. Bugbot is set up for automated code reviews on this repo. Configure here.}

Link to Devin session: https://app.devin.ai/sessions/bbe614d754bf4e779ed46e0d131c407e
Open in Devin Desktop: https://app.devin.ai/desktop/session/bbe614d754bf4e779ed46e0d131c407e?variant=devin
Requested by: @yassin-berriai

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_auth_single_io_lookup (e75b541) with litellm_internal_staging (9071ca5)}

[greptile-apps]

Greptile Summary

This PR reduces authentication-path Redis and PostgreSQL round trips by batching referenced auth-object reads, cache writes, and spend-counter reads

• Adds best-effort auth-object prefetching while retaining existing getters as the enforcement and fallback path
• Adds request-scoped spend-counter batching with authoritative Redis-miss handling
• Adds a Redis pipeline operation supporting per-entry TTLs
• Adds fake-backed unit coverage and real-PostgreSQL proxy behavior coverage
• The changes since the previous review align prefetched user relations with the established getter and correctly expose team model aliases

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule findings

The latest changes bring prefetched user data in line with the established getter and correctly map team model aliases, while all previous findings were resolved or conceded

Important Files Changed

Filename	Overview
litellm/proxy/auth/auth_object_prefetch.py	Adds best-effort batched cache and database warming, with the latest changes matching established user and team model relation shapes
litellm/proxy/auth/user_api_key_auth.py	Integrates auth-object and spend-counter batching after model access enforcement and scopes it to authentication
litellm/proxy/spend_tracking/spend_counter_batch.py	Implements task-local spend-counter MGET batching with clean-miss and Redis-failure distinction
litellm/proxy/db/spend_counter_reseed.py	Reuses active batched counter reads before performing per-key Redis or database reseeding
litellm/caching/redis_cache.py	Adds non-transactional pipelined writes with independent TTLs for each cache entry
tests/proxy_behavior/auth/test_auth_object_prefetch.py	Verifies the combined SQL against PostgreSQL, including membership selection, nullable lists, and team model aliases

_{Reviews (6): Last reviewed commit: "fix(auth): map the model table's aliases..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 97.68340% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/caching/redis_cache.py	68.75%	5 Missing ⚠️
litellm/proxy/auth/auth_object_prefetch.py	99.20%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[yassin-berriai]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[devin-ai-integration]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e75b541. Configure here.}