fix(cache): write DualCache's redis tier with default_redis_ttl

[apex-mochen]

Fixes #43187.

Signed-off-by: ding 2756823972@qq.com

Problem and scope

DualCache inherited the in-memory default TTL for Redis writes even when a separate Redis default was configured (#43187). Resolve each tier's default independently while retaining an explicitly supplied TTL. Preserve auth-cache mirroring and remove the cache-only setting before constructing the Redis client. The current diff includes regression coverage for direct and batched writes and the auth-cache configuration path.

2026-10-05 compatibility evidence

PR head aab86f42ee4e0ec91128b3975b9fc9e7bd154afa was combined with upstream main 1238cfe90f1029bf33e31ab40a805e7ecabfa1ff in an isolated local integration worktree without conflicts. The published branch is unchanged. The auth-cache tests have moved under tests/unit/proxy/ on current main.

/home/dcl/venvs/litellm/bin/python -m pytest tests/unit/caching/test_dual_cache.py tests/unit/caching/test_request_redis_batch_pre_call.py tests/unit/caching/test_request_redis_batch_post_call.py tests/unit/proxy/test_redis_auth_cache_flag.py tests/unit/proxy/common_utils/test_user_api_key_cache.py -q --tb=short
# 165 passed, 2 warnings in 8.20s

The initial attempt with the old test path ran no tests; the result above is the successful run with the current paths. This is local compatibility evidence, not a new approval or a performance measurement.

The September 30 approvals refer to an earlier SHA and came from a community reviewer, who explicitly noted that a BerriAI maintainer approval is still needed. Current-head review remains outstanding; this PR is not being represented as merge-approved. AI assistance was used in this validation and description update.

Additional real Redis validation (2026-10-05)

A standalone integration probe drove the real DualCache/RedisCache paths against Redis 7.0.15 on loopback in Ubuntu 24.04/WSL. No Redis methods were mocked. Each run used a fresh isolated instance with persistence disabled and terminated it afterward.

At PR head aab86f42ee4e0ec91128b3975b9fc9e7bd154afa, 12/12 checks passed: sync write, async write, async pipeline, and request-scoped deferred batch, each with Redis default TTL 3600 (memory TTL 300), explicit TTL 7, and memory-TTL fallback 300 when the Redis default is unset. Redis TTL was read back directly; cached values matched in every case.

Negative control at main commit 1238cfe90f1029bf33e31ab40a805e7ecabfa1ff produced 8/12 passes: all four Redis-default cases incorrectly read back 300 rather than 3600; explicit and fallback controls passed. The baseline tree and PR branch were not modified for this experiment.

This verifies single-node Redis command/TTL behavior, not actual wall-clock expiry, cluster/sentinel behavior, fault injection, or performance. No throughput improvement is claimed. Previous community approvals apply to their reviewed SHAs; maintainer approval of the current head is still needed.

[greptile-apps]

Confidence Score: 4/5

[Medium risk] Fixes cache TTL configuration for Redis writes.

The PR appears safe to merge, with a non-blocking request for Redis TTL regression coverage.

Findings

1.  Redis TTL lacks test coverage ▶

Summary

The PR separates in-memory and Redis TTL arguments for DualCache’s sync, async, and pipeline writes so Redis uses its configured default rather than inheriting the in-memory default.

• The changed Redis TTL behavior lacks a targeted regression test.

_{Reviews (1) · Last reviewed commit: "fix(cache): DualCache uses default_redis..."}

[greptile-apps]

Redis TTL lacks test coverage

This fix changes the Redis TTL passed by the sync, async, and pipeline writes, but the existing DualCache tests check only in-memory expiry. A test with different defaults for each tier should assert the Redis TTL in all three paths; without it, the original bug could return without a test failing.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing apex-mochen:fix/dual-cache-redis-ttl (aab86f4) with main (fe9b6fd)¹}

Footnotes

1. No successful run was found on main (615ed79) during the generation of this report, so fe9b6fd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[lets-order-some-fries]

Both proxy-infra failures are the #43187 behaviour, pinned on purpose for the auth cache, so they need a decision rather than a test edit.

tests/test_litellm/proxy/common_utils/test_user_api_key_cache.py:142-160 and :179-190 (at 14f4c34c61) build UserApiKeyCache(redis_cache=fake, default_in_memory_ttl=60) (and 45) with no Redis TTL and assert the Redis write gets the in-memory value. At this head the fake gets no ttl at all: assert None == 60 and assert None == 45 in the job log, 2 failed / 12731 passed. Locally that file is 32 passed on 14f4c34c61 and 2 failed / 30 passed on 2a2e9af8c6.

Both tests came in with #26202, which put user_api_key_cache on Redis. The docstring says "Redis should receive the same TTL as memory (matches proxy defaults: enum 60s)", and the same PR wrote proxy_server.py:6400-6402: mirror the TTL on Redis, "otherwise DualCache misses in memory fall back to a key that outlasts user_api_key_cache_ttl". It also added the default_redis_ttl=litellm.default_redis_ttl at proxy_server.py:4745-4748, but DualCache never read that value, so what actually held is what the tests and that comment describe: Redis mirrors the in-memory TTL.

Loading each config with cache: true (type: redis) and enable_redis_auth_cache: true, and recording what a real RedisCache sends:

config	Redis EX on main	Redis EX at this head
defaults	60	60
litellm_settings.default_redis_ttl: 3600	60	3600
general_settings.user_api_key_cache_ttl: 300	300	300

That is for auth-cache writes without an explicit ttl, e.g. the team-membership row at litellm/proxy/auth/auth_checks.py:2357-2361 and the SSO user row at litellm/proxy/management_endpoints/ui_sso.py:1930-1934. Key and team objects go through _cache_management_object, which passes ttl=get_management_object_ttl(...) (auth_checks.py:2779-2784), so they keep the in-memory TTL either way. The defaults row only matches because RedisCache has its own 60s fallback (litellm/caching/redis_cache.py:671-674, litellm/caching/base_cache.py:23). The 3600 row needs default_redis_ttl listed before cache: true, because Redis is attached to the auth cache while cache is processed; listed after it, both columns read 60.

One way to keep the auth-cache contract without special-casing DualCache: UserApiKeyCache falls back to its in-memory TTL for Redis, and the proxy stops passing the global on attach.

# litellm/proxy/common_utils/user_api_key_cache.py, UserApiKeyCache.__init__
redis_ttl: Final = default_redis_ttl if default_redis_ttl is not None else default_in_memory_ttl
# pass redis_ttl as default_redis_ttl to super().__init__() and to key_object_cache

# litellm/proxy/proxy_server.py:4745-4748
user_api_key_cache.attach_redis_cache(redis_cache)

On top of this head that gives 60 / 60 / 300 for the three rows, both tests pass unchanged, and tests/unit/caching/test_dual_cache.py, test_user_api_key_cache.py and test_redis_auth_cache_flag.py pass (85, including the test below). If maintainers would rather the auth cache follow default_redis_ttl, updating the two tests is the other route, but that should be their call, stated in the PR.

The fix itself has no test (Greptile flagged this too). This fails on 14f4c34c61 (the three tier_defaults cases: Redis gets 60, not 3600) and passes on this head. It fits beside the other TTL tests in tests/unit/caching/test_dual_cache.py, which already has every import it needs:

async def _write_through(dual_cache: DualCache, write_path: str, **kwargs) -> None:
    if write_path == "set_cache":
        dual_cache.set_cache("ttl_key", "v", **kwargs)
    elif write_path == "async_set_cache":
        await dual_cache.async_set_cache("ttl_key", "v", **kwargs)
    else:
        await dual_cache.async_set_cache_pipeline([("ttl_key", "v")], **kwargs)


@pytest.mark.asyncio
@pytest.mark.parametrize("write_path", ["set_cache", "async_set_cache", "async_set_cache_pipeline"])
@pytest.mark.parametrize(
    ("kwargs", "memory_ttl", "redis_ttl"),
    [({}, 60, 3600), ({"ttl": 99}, 99, 99)],
    ids=["tier_defaults", "explicit_ttl"],
)
async def test_dual_cache_writes_each_tier_with_its_own_default_ttl(write_path, kwargs, memory_ttl, redis_ttl):
    """
    Regression for #43187: the Redis tier was written with default_in_memory_ttl,
    so a configured default_redis_ttl never took effect. An explicit ttl still
    reaches both tiers unchanged.
    """
    in_memory_cache = InMemoryCache(default_ttl=600)
    mock_redis = MagicMock()
    mock_redis.async_set_cache = AsyncMock()
    mock_redis.async_set_cache_pipeline = AsyncMock()
    dual_cache = DualCache(
        in_memory_cache=in_memory_cache,
        redis_cache=mock_redis,
        default_in_memory_ttl=60,
        default_redis_ttl=3600,
    )

    before = time.time()
    await _write_through(dual_cache, write_path, **kwargs)
    after = time.time()

    assert getattr(mock_redis, write_path).call_args.kwargs["ttl"] == redis_ttl
    expiry = in_memory_cache.ttl_dict["ttl_key"]
    assert before + memory_ttl <= expiry <= after + memory_ttl

Validate PR title fails on the subject: .github/workflows/conventional-commits.yml:45 requires a lowercase first character, e.g. fix(cache): write DualCache's redis tier with default_redis_ttl.

[apex-mochen]

@lets-order-some-fries Thanks for the thorough review. Addressed both points in f01a77cf (rebased onto latest main d18fcb09):

1. Auth-cache mirror TTL — moved the fallback into UserApiKeyCache.__init__:

   effective_redis_ttl = default_redis_ttl if default_redis_ttl is not None else default_in_memory_ttl

   and passed that as default_redis_ttl to super().__init__() and key_object_cache. DualCache itself stays generic (no auth-cache special-casing). The three-row table now reads 60 / 3600 / 300, and the two existing test_user_api_key_cache.py assertions keep passing unchanged.

2. cache_params startup TypeError — _init_cache now does cache_params.pop("default_redis_ttl") after promoting it to litellm.default_redis_ttl, so redis-py Redis() no longer receives the unexpected kwarg.

3. Regression test — added test_dual_cache_writes_each_tier_with_its_own_default_ttl to tests/unit/caching/test_dual_cache.py, covering set_cache / async_set_cache / async_set_cache_pipeline with both ({}, 60, 3600) tier-defaults and ({"ttl": 99}, 99, 99) explicit-ttl cases.

4. PR title — updated to fix(cache): write DualCache's redis tier with default_redis_ttl to satisfy the conventional-commits check.

Happy to adjust if you'd rather the auth cache follow default_redis_ttl instead of mirroring the in-memory TTL — that would just mean flipping the two test_user_api_key_cache.py assertions, as you noted.

[lets-order-some-fries]

@apex-mochen re-checked at f01a77cfb3. The DualCache change, the new test and the startup fix all check out. Three things are left: lint needs a format pass, the auth cache still takes default_redis_ttl when it is set before Redis is attached, and the pop has a side effect on DB-stored cache settings.

What I ran:

• test_dual_cache_writes_each_tier_with_its_own_default_ttl passes here (6). With litellm/ reset to the merge base d18fcb09d6 and the tests kept, the three tier_defaults cases fail with assert 60 == 3600. The explicit_ttl cases pass on both.
• test_user_api_key_cache.py is not in the diff and passes (32). Reverting only user_api_key_cache.py brings back None == 60 / None == 45 in the two TTL tests, so the __init__ fallback is what keeps them green. tests/unit/caching/test_dual_cache.py gives 46 passed and test_redis_auth_cache_flag.py 7.
• A config with cache_params: {type: redis, host: ..., port: ..., default_redis_ttl: 3600} raises TypeError: Redis.__init__() got an unexpected keyword argument 'default_redis_ttl' at d18fcb09d6 and loads here.

lint stops at ruff format --check (Would reformat: litellm/proxy/common_utils/user_api_key_cache.py). With ruff 0.15.3, the version pyproject.toml pins and the job installs, this hunk is the whole change:

-        effective_redis_ttl: Final = (
-            default_redis_ttl
-            if default_redis_ttl is not None
-            else default_in_memory_ttl
-        )
+        effective_redis_ttl: Final = default_redis_ttl if default_redis_ttl is not None else default_in_memory_ttl

The later steps in that job haven't run yet. Locally at this head, ruff check on the changed files and the strict-rule, type-discipline and test-quality gates (against d18fcb09d6) pass.

On your question: I'd keep the mirror, but at this head it breaks when default_redis_ttl is set before Redis is attached. _attach_redis_usage_cache still hands the global to the auth cache (proxy_server.py:4744-4748), and attach_redis_cache replaces the fallback with any non-None value (dual_cache.py:105-106). The two unit tests build UserApiKeyCache directly, so they never reach that path. I loaded each config with cache: true (type: redis) and enable_redis_auth_cache: true, then recorded the EX that a real RedisCache sends for a write without ttl, as the membership row at auth_checks.py:2357-2361 does:

config	d18fcb09d6	this head	head + attach change below
defaults	60	60	60
litellm_settings.default_redis_ttl: 3600, before cache	60	3600	60
same, after cache	60	60	60
cache_params.default_redis_ttl: 3600	TypeError	3600	60
general_settings.user_api_key_cache_ttl: 300	300	300	300

So at this head the result also depends on key order in the YAML. A write with ttl=get_management_object_ttl(...), as key and team objects use, got 60 (300 in the last row) in every run that loaded.

Why mirror:

• Key and team objects are written with an explicit ttl= through _cache_management_object (auth_checks.py:2779-2784), so this setting never reaches them, and revoked or updated keys behave the same either way.
• What it does reach are the writes without ttl: the membership row above, the user row written by _sync_user_role_from_jwt_role_map (ui_sso.py:1930-1934), and the proxy-wide spend the max_budget check caches through cache=user_api_key_cache (user_api_key_auth.py:748-751, stored at cache_coordinator.py:148; with default_redis_ttl: 3600 before cache, that write gets EX 3600 at this head and 60 at d18fcb09d6).
• Evicting those is best-effort: on a cache error evict_and_broadcast logs that "a stale object may be served until its TTL expires" (auth_cache_invalidation_pubsub.py:138-143). If the Redis delete is what failed, every instance reads the stale copy back from Redis on its next in-memory miss (dual_cache.py:255-261). With the mirror that window is the auth cache's own TTL (60s, or user_api_key_cache_ttl); without it, it is default_redis_ttl. The comment at proxy_server.py:6404-6406 asks for the mirror for the same reason.

A longer auth-cache TTL is still available through user_api_key_cache_ttl, which sets both tiers.

The change for the last column, the attach half of my earlier suggestion:

# litellm/proxy/proxy_server.py:4744-4748
    if enable_redis_auth_cache is True:
        user_api_key_cache.attach_redis_cache(redis_cache)

Below is a test that fails at this head (assert 3600 == 60) and passes with the change. It fits at the end of tests/test_litellm/proxy/test_redis_auth_cache_flag.py, which already imports litellm, patch, ps and DualCache and defines _FakeRedisCache:

def test_auth_cache_redis_tier_keeps_in_memory_ttl_when_default_redis_ttl_is_set(monkeypatch):
    """
    #43187: user_api_key_cache mirrors its in-memory TTL on Redis. A global
    default_redis_ttl, from litellm_settings or cache_params, must not override
    that when Redis is attached.
    """
    from types import SimpleNamespace

    from litellm.proxy.common_utils.user_api_key_cache import UserApiKeyCache

    monkeypatch.setattr(litellm, "default_redis_ttl", 3600)
    auth_cache = UserApiKeyCache(default_in_memory_ttl=60)
    with (
        patch.object(ps, "user_api_key_cache", auth_cache),
        patch.object(ps, "spend_counter_cache", DualCache()),
        patch.object(ps, "cli_sso_session_cache", DualCache()),
        patch.object(ps, "litellm_config_cache", SimpleNamespace(redis_cache=None)),
    ):
        ps._attach_redis_usage_cache(_FakeRedisCache(), enable_redis_auth_cache=True)

    assert auth_cache.redis_cache is not None
    assert auth_cache.default_redis_ttl == 60
    assert auth_cache.key_object_cache.default_redis_ttl == 60

cache_params.pop changes the caller's dict. init_cache_settings_in_db calls _init_cache(cache_params=cache_params) and then stores cache_params.copy() as _last_cache_params, so it can skip the rebuild when the DB row hasn't changed (cache_settings_endpoints.py:388-401). The update endpoint does the same at :645-651. After the pop that copy has no default_redis_ttl, so the next reload compares unequal and rebuilds litellm.cache. That happens on every add_deployment run (30s by default). I ran two polls over one stored row, {type: local, default_redis_ttl: 3600}. At d18fcb09d6 the cache is built once and an entry written between the polls survives. Here it is built twice and the entry is gone. With type: redis, the base raises the TypeError on each poll and this head builds a new RedisCache on each poll. Reading the key instead of popping it builds once in both cases and still avoids the TypeError:

        if "default_redis_ttl" in cache_params:
            litellm.default_redis_ttl = cache_params["default_redis_ttl"]

        litellm.cache = Cache(**{k: v for k, v in cache_params.items() if k != "default_redis_ttl"})

With the format fix, both changes and that test, test_dual_cache.py, test_user_api_key_cache.py and test_redis_auth_cache_flag.py give 86 passed, test_cache_settings_endpoints.py 50, and the redis or cache subset of test_proxy_server.py 46.

[apex-mochen]

Thanks for the detailed re-check. Addressed all three remaining points in f11995d:

1. Ran the pinned Ruff 0.15.3 formatter on litellm/proxy/common_utils/user_api_key_cache.py. The only hunk is exactly what you flagged — effective_redis_ttl folds onto one line.
2. Kept auth-cache Redis TTL mirrored to its own in-memory TTL by no longer passing the global default_redis_ttl during auth-cache Redis attachment (_attach_redis_usage_cache now calls user_api_key_cache.attach_redis_cache(redis_cache)). Since DualCache.attach_redis_cache only overrides default_redis_ttl when the kwarg is non-None, the auth cache's constructor-set TTL (mirror of default_in_memory_ttl) survives. Added the regression test you supplied at the end of tests/test_litellm/proxy/test_redis_auth_cache_flag.py (test_auth_cache_redis_tier_keeps_in_memory_ttl_when_default_redis_ttl_is_set).
3. Removed the cache_params.pop() mutation in _init_cache. It now reads litellm.default_redis_ttl = cache_params["default_redis_ttl"] without touching the caller's dict and filters the key out only when constructing litellm.cache = Cache(**{k: v for k, v in cache_params.items() if k != "default_redis_ttl"}), so _last_cache_params snapshots match across polls and the cache is not rebuilt every 30s.

I kept the diff tight: no reformatting of test_dual_cache.py or other pre-existing lines (I reverted those the formatter wanted to collapse), no changes to DualCache.attach_redis_cache itself, and no other Redis-TTL contract changes.

Local test run: I wasn't able to bring up the full litellm test env in this sandbox (editable install hit a metadata-generation error and pulling the whole transitive dep tree would not finish in the window), so I'd appreciate CI running the focused suites you enumerated — tests/unit/caching/test_dual_cache.py, tests/test_litellm/proxy/common_utils/test_user_api_key_cache.py, tests/test_litellm/proxy/test_redis_auth_cache_flag.py, tests/test_litellm/proxy/test_cache_settings_endpoints.py, plus the redis/cache subset of test_proxy_server.py. The three code changes are mechanical and match your suggested diffs line-for-line; the new test is verbatim from your comment.

New commit: f11995dfb6cd864265ec8818120be9f5a805df66 on fix/dual-cache-redis-ttl.

[apex-mochen]

Follow-up: rebased onto latest main (f4308bc1) and squashed to a single commit 3e891a3d. Main had moved _init_cache to a non-pop read already, so the Cache() now filters default_redis_ttl at construction time (keeping main's new default_in_memory_ttl handling intact). The attach-arg removal and the new regression test are unchanged. mergeable_state is now clean.

[lets-order-some-fries]

@apex-mochen CI did run them at 3e891a3d94: proxy-infra ran test_redis_auth_cache_flag.py, including test_auth_cache_redis_tier_keeps_in_memory_ttl_when_default_redis_ttl_is_set (13002 passed), and every test job is green. The attach change and the non-mutating filter read as intended. Two mechanical things are left:

1. lint fails at ruff format --check with Would reformat: litellm/proxy/proxy_server.py. Ruff 0.15.3 wants the Cache(...) call on one line:

   -        litellm.cache = Cache(
   -            **{k: v for k, v in cache_params.items() if k != "default_redis_ttl"}
   -        )
   +        litellm.cache = Cache(**{k: v for k, v in cache_params.items() if k != "default_redis_ttl"})

2. The squash converted litellm/proxy/common_utils/user_api_key_cache.py and tests/test_litellm/proxy/test_redis_auth_cache_flag.py to CRLF line endings: every line of both files now ends in \r, and none did on main or at f01a77cf. That is why the diff shows both files rewritten (+689/−597 overall); with git diff --ignore-cr-at-eol against main the PR is +111/−19. ruff format keeps existing line endings, so lint won't catch it. Converting the two files back to LF (dos2unix, or sed -i 's/\r$//') and amending brings the diff back to the real change.

[apex-mochen]

Thanks for the re-check. Addressed both mechanical points in 66cd3f9a:

1. Collapsed the Cache(**{...}) call onto one line so ruff format --check is happy.
2. Rewrote user_api_key_cache.py and test_redis_auth_cache_flag.py from the LF versions on main (the previous push had picked up CRLF from a Windows working tree). The PR diff is now +109/-19 across the five files, no whole-file rewrites.

Rebased onto latest main (f4308bc1). All changes are still the minimal semantic set: per-tier TTL split in DualCache, the effective_redis_ttl fallback in UserApiKeyCache.__init__, dropping the global default_redis_ttl from the auth-cache attach call, the non-mutating cache_params filter at Cache(...) construction, and the two regression tests.

[lets-order-some-fries]

@apex-mochen re-checked at 66cd3f9a: both files are LF again, the diff is back to the five files you listed, and lint is green along with every test job. The attach change, the non-mutating filter and the regression test are all in. Thanks for turning this around so quickly. LGTM.

[apex-mochen]

Rebased onto main as requested. New head: 38615ef282.

main had moved 110 commits ahead of the old merge base and had refactored the same three files, so the branch was conflicting. It is now a single commit whose parent is 04fa760bf2 (current main), and compare/main...head is ahead=1, behind=0.

The only conflict was append-only in tests/unit/caching/test_dual_cache.py. main had added six test_shared_batch_read_* tests at EOF while this PR added _write_through plus the per-tier TTL regression at EOF. I kept both; no code existed on both sides of the marker. The test file now has 43 test functions against main's 42 — exactly the one this PR adds. The added test block is byte-identical to the version you approved, and all six of main's new tests are present.

The three production files auto-merged, and I checked the change survived rather than assuming it did: every added line from the approved revision is present verbatim in the rebased tree (24/24 in dual_cache.py, 6/6 in user_api_key_cache.py, 13/13 in proxy_server.py) and every removed line is still absent. All five files are LF.

What I ran locally at 38615ef282:

• pytest tests/unit/caching/test_dual_cache.py — 54 passed
• pytest tests/test_litellm/proxy/test_redis_auth_cache_flag.py — 8 passed
• pytest tests/test_litellm/proxy/common_utils/test_user_api_key_cache.py — 32 passed
• ruff format --check on the three changed litellm/ files — clean
• ruff check --config ruff-tests.toml tests — clean, so there is no unused pytest import
• scripts/ruff_strict_gate.py --base main — OK; I ran it against the pre-rebase head too and it passes at both, so the resolve introduced no lint delta

One thing worth flagging rather than leaving for CI: tests/test_litellm/proxy/common_utils has 11 failures here (test_encrypt_decrypt_utils.py, test_callback_utils.py, test_registry_read_through.py). I ran the same directory on unmodified main and got the identical 11 failures and the same 923 passes, so they are inherited from main and unrelated to this diff — not something I should paper over in this PR.

No further changes from me; the branch is yours to re-check.

[apex-mochen]

Rebased and pushed 8fb80bdcc0. Two of the three failures were real and are fixed; the third is inherited from main.

misc / Run tests — inherited from main, not this diff.
job. This job is red at step 11 ("Run tests") on every one of the last 11 commits on main, including today's HEAD: 6bc17f98d7, 314ff111e5, d79600987e, 6cf51383bf, d02ff435bf, ba6d6d1a95, b71f02dbcf, 7d9cc28dce, cd0ac30881, 61a73c59b0, 8afabe81f1 — 11/11 failure. main's own red job is here; the only other check red on main is ui-unit-tests. Rerunning will not clear this one.

lint — real, now fixed. job. It is not a formatting failure: step 14, "Check ruff format", passed (my local ruff is 0.15.3, the version pyproject.toml pins). It stops at step 20, "Check type-discipline budget (delta vs base)": LIT002: total 26731 over limit 26715 (this change added 7). That budget is a whole-tree ceiling, and the per-tier dict(kwargs) copies this PR needs tripped seven mutable-collection construction hits. Fixed by keeping each copy on one line with a # mutable-ok: reason, and by replacing the filtered Cache(**{...}) construction with a copy-then-pop. Both budget gates now report OK against base 04fa760bf2.

caching-local — real, now fixed. job. The failure is test_request_redis_batch_post_call.py::test_a_deferred_response_cache_set_without_a_ttl_expires_in_redis_like_the_direct_path, from the post-call batching work that landed on main after the last review. It passes on a clean main checkout and failed on my branch, so this one is mine to fix rather than an environment difference.

The cause was the semantics, not the test. That test asserts the redis tier is written with TTL 300 when only default_in_memory_ttl=300 is set. My previous revision gave the redis tier default_redis_ttl only, so with no redis-specific TTL configured the tier got no TTL at all. That contradicts the test and the contract this PR already documents for the auth cache ("Redis mirrors in-memory TTL when no explicit Redis TTL is configured").

Fixed by making that fallback the general rule in DualCache: the redis tier takes default_redis_ttl and falls back to the in-memory default when unset. An explicit ttl= still reaches both tiers unchanged, and when both defaults are set each tier keeps its own — so default_redis_ttl now only ever overrides the mirroring, it never removes it. This also makes UserApiKeyCache's effective_redis_ttl consistent with the general rule instead of relying on it.

Verification at 8fb80bdcc0 (base 04fa760bf2):

• scripts/type_discipline_gate.py --base 04fa760bf2 -> OK (was +7 over the ceiling)
• scripts/ruff_strict_gate.py --base 04fa760bf2 -> OK
• ruff format --check on the two changed litellm/ files -> clean
• pytest tests/unit/caching/test_request_redis_batch_post_call.py -> 22 passed
• pytest tests/unit/caching/test_dual_cache.py tests/test_litellm/proxy/test_redis_auth_cache_flag.py tests/test_litellm/proxy/common_utils/test_user_api_key_cache.py -> 94 passed

Negative control, because a local failure is easy to misattribute: the whole tests/unit/caching/ tree fails 13 tests on this branch and the same 13 on a clean checkout of main (the semantic-cache and redis breaker/pool tests that need services this box doesn't have). No failure on this branch is absent on clean main.

I'm happy to adjust if you'd rather default_redis_ttl not fall back and the post-call test be updated instead — that's the product decision you flagged, and the fallback is the reading that keeps both behaviours consistent.

[apex-mochen]

Update on the post-fix CI run (8fb80bdcc) — both regressions are resolved:

• LiteLLM Linting / lint → success (was the type-discipline budget: LIT002 pushed the whole-tree total over its ceiling)
• Unit Tests / caching-local → success (was test_a_deferred_response_cache_set_without_a_ttl_expires_in_redis_like_the_direct_path)

The one remaining red check, misc / Run tests, has an outcome identical to main's:

	failing	passed
this PR @ 8fb80bdcc	5	6875
main @ 6bc17f98d7 (current HEAD)	5	6875

The 5 failing tests are the same on both sides, and none of them are under the files this PR touches:

• tests/unit/interactions/test_openapi_compliance.py (4 tests)

• tests/unit/test_utils.py::test_aaamodel_prices_and_context_window_json_is_valid

• this PR's job: https://github.com/BerriAI/litellm/actions/runs/36705349782/job/109854173229

• main's job: https://github.com/BerriAI/litellm/actions/runs/36697630376/job/109829293463

misc / Run tests has been red on main for every commit since 74cad08997 (2026-09-28T03:31:22), with no green run since — 112 red / 180 green across the last 292 commits, so it was intermittently flaky before it went permanently red.

Local checks at 8fb80bdcc: the type-discipline gate passes, ruff format --check is clean on both changed litellm/ files, and tests/unit/caching/ gives 13 failed / 516 passed — those same 13 fail on a clean main worktree here (they need services not available on this box), with none unique to this branch.

@lets-order-some-fries could you confirm the remaining misc red is not a blocker here? Happy to rebase or adjust if you see it differently.

[apex-mochen]

@lets-order-some-fries a nudge on this one, with the evidence collected in a single place.

The remaining red check is main's, not this diff's. misc / Run tests fails identically on both sides:

	failing	passed
this PR @ 8fb80bdcc0	5	6875
main @ 6bc17f98d7 (current HEAD)	5	6875

The 5 failing tests are the same set on both sides — tests/unit/interactions/test_openapi_compliance.py (4 tests) plus tests/unit/test_utils.py::test_aaamodel_prices_and_context_window_json_is_valid. None of them sit under a file this PR changes. misc / Run tests has been red on main for every commit since 74cad08997 (2026-09-28T03:31:22) with no green run since, so a re-run will not clear it.

Current state of this PR: 1 approval, 100 checks, 99 green, and misc is the only red one.

Could you either confirm that misc is not a blocker here, or tell me what you'd like adjusted? If you want a rebase onto current main, say the word and I'll do it immediately.

One note so this doesn't need repeating: I have already posted the same finding on #43577, #43292 and #42892. If you confirm here, I'll point those three at this thread rather than re-argue it on each — the question and the evidence are identical on all four.

[lets-order-some-fries]

@apex-mochen misc / Run tests isn't a required check on main, and main at b370996b9d fails the same five tests, so it isn't from this PR. It isn't mine to waive, here or on the other three: merging needs a BerriAI maintainer's approval, and mine doesn't count toward it.

I'd keep the fallback. In #43187 I suggested falling through to the backend's default_ttl, but main already sends default_in_memory_ttl to Redis when no ttl is passed, so with the fallback the PR only changes writes where default_redis_ttl is set. Without it, those writes would get RedisCache's default_ttl (60s by default), not no TTL; only the test's fake records none.

One path isn't covered yet, new on main since my approval: _set_on_batch (dual_cache.py:560), behind async_set_cache_pre_call and async_set_cache_post_call, still sends the in-memory TTL to Redis. A completion's response-cache SET inside a proxy request goes that way, so #43187's own config (60 / 3600) still gets EX 60. That's from reading the code; I didn't run the branch. Applied to main, this gives 3600 for that case and keeps the #43779 test green:

        if ttl is None and self.default_redis_ttl is not None:
            return batch.set(key, value, self.default_redis_ttl)
        return batch.set(key, value, effective_ttl)

My approval is on 66cd3f9a. I'll look again once this and a default_redis_ttl=3600 case beside the #43779 test are in.

[apex-mochen]

@lets-order-some-fries Thanks for the detailed checklist. Both points are addressed:
_set_on_batch (pre/post-call pipeline path) — now writes the Redis tier with default_redis_ttl, keeping the same chain fallback (default_redis_ttl → default_in_memory_ttl) used by set_cache / async_set_cache / async_set_cache_pipeline; the in-memory tier still uses default_in_memory_ttl. The default_redis_ttl-less behavior is unchanged (falls back to in-memory TTL), so the original #43779 test stays green.
Test — added test_a_deferred_response_cache_set_without_a_ttl_uses_default_redis_ttl_when_configured right beside the #43779 post-call test, asserting the pipeline SET carries ttl=3600 when default_redis_ttl=3600.
Branch is rebased onto current main (9dda4d8). The misc / Run tests red check is pre-existing on main (b370996 fails the same 5 tests) and unrelated to this PR. Could you take another look? And since your approval doesn't count toward merge, if it looks good could you also help route it to a maintainer with write access? Happy to adjust if anything else comes up.

[lets-order-some-fries]

Approving at d7462ad2. 322b0952 is 8fb80bdc rebased with no change (git range-diff shows =), and d7462ad2 makes _set_on_batch resolve the Redis TTL in the same order as the direct paths: explicit ttl, then default_redis_ttl, then default_in_memory_ttl. Locally the three test files this PR touches pass (85 tests), and with only the _set_on_batch hunk reverted the new test fails with ('SET', 'deferred', 300) == ('SET', 'deferred', 3600), so it covers the gap. The required checks are green. misc fails the same four test_openapi_compliance.py tests as main at 9dda4d89 (#43764 fixed the fifth), and osv-scan flags urllib3 in uv.lock and next in the dashboard's package-lock.json, which this PR doesn't touch, and fails the same way on other open PRs.

On routing: I don't have write access, so this review is as far as I can take it. The litellm-engineering team is already a requested reviewer.

[apex-mochen]

Thanks for the thorough review! Could someone from @BerriAI/litellm-engineering take a look when available? The latest revision has been approved, with 85 relevant tests passing locally. Happy to address any remaining feedback.

[apex-mochen]

Hi @litellm-engineering — rebased #43293 onto latest main and resolved the test-file conflict; mergeable is back to true and the earlier approvals (lets-order-some-fries, 85 tests verified) still stand. Could this be merged when CI is green? Happy to make any further adjustments.

[apex-mochen]

CI triage for the four red checks on 7001066203:

Inherited from main — not this PR

• proxy-behavior: red on main itself. It is failure at ef6aa4ad66 (this branch's merge-base) and at 6d7d183a80, while caching-local / Run tests is success at the same SHAs. Neither commit here touches tests/proxy_behavior/.
• mcp-integration / Run tests: also red on main at those same two SHAs. The single failure is test_optional_discovery_rejects_incomplete_walks[False-method_not_found-resources/templates/list] (assert 0 == 2, behind a TimeoutError against https://example.com/mcp) in tests/unit/experimental_mcp_client/, which this PR does not touch.

Mine — fixed in 76bf500021

• lint: reportUnknownArgumentType was 44811 against a 44802 limit (base 44804), i.e. +7 blamed on this diff. The per-tier dict(kwargs) copies rewrote each call site as **per_tier_kwargs, and unpacking an unknown-typed mapping is exactly what that rule reports. Replaced with a ttl_given flag that keeps the base's **kwargs call shape. Re-measured against ef6aa4ad66: +0 for that rule. ruff_strict_gate, type_discipline_gate, test_quality_gate, ruff check on both trees and the diff-scoped ruff format --check all pass locally.
• caching-local / Run tests: 2 failed, 566 passed — my two new cases called DualCache.async_set_cache_post_call, which does not exist. Dropped them; the post-call batch path stays covered by the existing perf(proxy): one post-call Redis pipeline per backend for spend, rate-limit, routing and response-cache writes #43779 test, and the [Bug]: DualCache writes Redis entries with default_in_memory_ttl, so a configured default_redis_ttl never takes effect #43187 regression moves to test_request_redis_batch_pre_call.py beside it.

Full affected files now pass (121 passed): tests/unit/caching/test_dual_cache.py, tests/unit/caching/test_request_redis_batch_pre_call.py, tests/unit/caching/test_request_redis_batch_post_call.py, tests/test_litellm/proxy/test_redis_auth_cache_flag.py.

I also re-ran the negative control: the same test files staged onto ef6aa4ad66 (unfixed source) fail exactly the four regression cases and pass the three explicit-ttl guard cases, so the new tests bind to the fix rather than to the fixture.

No Postgres job was red on the previous head — the four above were the only failures.

[apex-mochen]

Follow-up, because "red on main too" undersold it. proxy-behavior has no test failure at all:

=============== 929 passed, 15670 warnings in 110.34s (0:01:50) ================
...
==> Verifying GPG signature integrity
gpg:                using RSA key 27034E7FDB850E0BBC2C62FF806BB28AED779869
gpg: Can't check signature: No public key
==> Could not verify signature. Please contact Codecov if problem continues
    Exiting...
##[error]Process completed with exit code 1.
##[end-action id=__codecov_codecov-action.__run_8;outcome=failure;conclusion=failure;duration_ms=535]

The suite passes; the red is the codecov/codecov-action step, whose freshly downloaded CLI fails signature verification in the runner. The identical step fails identically on main at ef6aa4ad66, so this check carries no signal about a diff — mine or anyone else's.

The run on 76bf500021 finished at 97 success / 2 skipped / 1 failure, and the single failure is the proxy-behavior codecov step above. The checks that were red before are now green: lint and caching-local / Run tests, alongside mcp-integration / Run tests, unit-test, code-quality, schema-migration and the three */ Run tests shards that touch this area (misc, proxy-server, core-utils). So nothing here is left to act on except the codecov signature problem, which is repo-wide.

[apex-mochen]

Heads-up on the approval SHA: the branch moved twice after the 09-30 LGTM, so the approved commit is no longer the head.

• approved: d7462ad2ab (2026-09-30 15:30)
• head: 76bf500021, pushed 2026-10-01 05:50

76bf500021 changes the shape of what was reviewed:

• litellm/caching/dual_cache.py — the per-tier dict(kwargs) copies are gone. A single ttl_given = "ttl" in kwargs is captured before either tier writes, and the tiers then share the call's own kwargs. The resolution order you verified is unchanged: explicit ttl, else default_in_memory_ttl for the memory tier and default_redis_ttl (falling back to default_in_memory_ttl) for Redis. This was forced by the type-discipline budget, not by behaviour.
• tests/unit/caching/test_dual_cache.py — test_dual_cache_writes_each_tier_with_its_own_default_ttl is replaced by two named cases covering the same two branches, and a pipelined case moved into test_request_redis_batch_pre_call.py.

The branch is still six files, +159/-10 against main, and no test file was dropped. I am not asking for anything with this comment — flagging it only so the merge decision is not made on a stale SHA.

[apex-mochen]

@lets-order-some-fries thanks for the thorough check — glad the five failing tests are pre-existing on main and not from this PR. The DualCache redis-tier fix and tests are ready on our side. Is there anything else needed before this can merge?

[apex-mochen]

@lets-order-some-fries rebased once more onto the latest main — the test-file conflict from the last 80+ commits is resolved (head aab86f4). The change itself is unchanged from your last review; CI is re-running. Should be ready to merge once checks pass.

[apex-mochen]

Gentle bump for merge:

• head aab86f42 (rebased onto latest main on 10-02, conflict fix only), mergeable=true, no conflicts, all runnable checks green;
• approved by lets-order-some-fries on 09-30 (two rounds); the rebase kept the code change identical.

The branch moved after that approval, so a quick re-approve at aab86f42 may be all that's missing — otherwise just a maintainer merge. Could someone from @litellm-engineering take it home? Thanks!

[apex-mochen]

Hi @litellm-engineering, gentle follow-up on this one. #43293 received two approvals from @lets-order-some-fries (Sep 29 & 30), and I've since rebased onto the latest main (head aab86f42); mergeable is true, and the remaining red checks were confirmed pre-existing on main. The change stays scoped to the DualCache redis-tier write path plus its regression test. When you have a moment, could this be merged? Thanks!

[apex-mochen]

Hi team, #43293 has two approvals and green CI — could a maintainer merge it?

[apex-mochen]

Hi maintainers — just a friendly bump: both review approvals are in and CI is fully green (104 checks). This fixes a real TTL-mirroring bug for the auth cache behind enable_redis_auth_cache. Could this be merged when convenient? Happy to rebase if anything landed since. Thanks!

[apex-mochen]

Maintainer review request: this is ready from the implementation/testing side.

The current version covers both the direct DualCache write path and the deferred batch path, preserves the existing in-memory fallback behavior, and includes regression coverage. I also validated the current patch against recent main and a real Redis 7.0.15 instance.

The community review has been positive, but the reviewer correctly noted that a BerriAI maintainer approval is still required.

@BerriAI/litellm-engineering could someone take a look when bandwidth allows? Happy to address any remaining concerns.

[apex-mochen]

Hi @lets-order-some-fries — thanks again for your earlier review and approval!

The HEAD has moved from the previously approved d7462ad to aab86f4. The subsequent changes are limited to CI/lint and test adjustments; the core per-tier Redis TTL behavior remains unchanged.

The current HEAD is CI-green, and the remaining historical review feedback has been addressed.

Would you be willing to give the current revision a quick re-review to confirm that your earlier approval still holds?

Once confirmed, I’d appreciate a final merge decision from the LiteLLM maintainers.

Thanks again for your help!

[lets-order-some-fries]

Re-checked at aab86f42 against d7462ad2. The ttl_given rewrite in dual_cache.py is safe: the in-memory tier is written first, from an unpacked copy, so the Redis TTL can't leak into it. I compared the four write paths from both heads over 152 combinations of the two defaults, caller ttl (absent, None, 99), local_only and an extra kwarg: identical. The #43187 script gets ttl=60 on all seven writes on main (63a4f3f2) and 3600 here; ttl=99 passes through unchanged. Across four proxy configs the auth-cache EX matches main. Locally, tests/unit/caching/ plus the two auth-cache test files: 614 passed, 1 skipped; merged into main, 696 passed.

This commit also drops the _init_cache filter, so cache_params: {type: redis, default_redis_ttl: 3600}, which loaded at d7462ad2, raises TypeError again, as on main. Contrary to the commit message, the base didn't tolerate it, and the body's "remove the cache-only setting before constructing the Redis client" no longer holds. That's outside #43187. My approval holds at this head.

[apex-mochen]

Thanks @lets-order-some-fries for the thorough review and approval! Really appreciate the extensive regression testing.

I’ll keep the approved head unchanged. The _init_cache issue appears separate from this PR’s scope and can be addressed independently.

Would a maintainer be able to take a look at the final merge readiness? Happy to help with anything remaining.

[apex-mochen]

Hi @ishaan-berri @yuneng-berri,

Could someone from the LiteLLM engineering team help with the final merge review for this PR?

@lets-order-some-fries has thoroughly reviewed and approved the latest commit (aab86f4), including 152 TTL combinations and 696 passing tests against main.

The remaining merge blockers, if any, would benefit from a maintainer decision. Happy to address anything needed.

Thanks for your time!

[apex-mochen]

Thanks again for the approvals on #43293 — all 100 checks are green. Do you happen to know which BerriAI maintainer could give the merge a nudge, or is there anything else you'd want changed first?