fix(caching): inject default_in_memory_ttl in DualCache async_set_cache and async_set_cache_pipeline

[pnookala-godaddy]

DualCache.async_set_cache and async_set_cache_pipeline were missing the default_in_memory_ttl injection that the sync set_cache method has. This caused InMemoryCache to fall back to its own default_ttl (600s) instead of using DualCache's configured default_in_memory_ttl (typically 60s).

This is particularly impactful for end-user budget enforcement in the proxy, where cached spend values could remain stale for 10 minutes instead of 1 minute, allowing users to exceed their budgets.

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🐛 Bug Fix

Changes

• litellm/caching/dual_cache.py: Added the same default_in_memory_ttl injection guard to async_set_cache and async_set_cache_pipeline that already existed in the sync set_cache method. If ttl is not
  provided and self.default_in_memory_ttl is set, the TTL is now forwarded to the underlying InMemoryCache call in both async paths.

Test

• tests/test_litellm/caching/test_dual_cache.py: Added 4 unit tests covering the fixed behavior:
  • test_dual_cache_async_set_cache_injects_default_in_memory_ttl — verifies that async_set_cache forwards default_in_memory_ttl to InMemoryCache when no explicit TTL is given
  • test_dual_cache_async_set_cache_respects_explicit_ttl — verifies that an explicitly passed TTL takes precedence over default_in_memory_ttl
  • test_dual_cache_async_set_cache_pipeline_injects_default_in_memory_ttl — verifies the same injection behavior for async_set_cache_pipeline
  • test_dual_cache_sync_and_async_set_cache_use_same_ttl — verifies that the sync and async code paths produce identical TTL behavior

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 26, 2026 11:57pm

[greptile-apps]

Greptile Summary

This PR fixes a parity bug between the sync and async cache write paths in DualCache. The sync set_cache method already injected default_in_memory_ttl into kwargs when no explicit TTL was provided, but both async_set_cache and async_set_cache_pipeline were missing this guard. As a result, async writes to InMemoryCache fell back to the cache's own default_ttl (600s) instead of DualCache's configured default_in_memory_ttl (typically 60s). This is especially impactful for end-user budget enforcement in the proxy, where stale cached spend values could persist for 10 minutes instead of 1 minute.

• Added 2-line TTL injection guard to async_set_cache (matching existing sync set_cache pattern)
• Added 2-line TTL injection guard to async_set_cache_pipeline (same pattern)
• 4 new unit tests verify the fix, including sync/async parity, explicit TTL override, and pipeline behavior
• No network calls in tests; all tests use local InMemoryCache objects

Confidence Score: 5/5

• This PR is safe to merge — it's a minimal, well-tested fix that adds missing TTL injection logic to async paths, matching existing sync behavior.
• The change is only 4 lines of production code, replicating an already-proven pattern from the sync set_cache method. The fix is straightforward, well-scoped, and accompanied by 4 comprehensive unit tests that verify both the fix and edge cases. No new dependencies, no architectural changes, and no risk of regression.
• No files require special attention.

Important Files Changed

Filename	Overview
litellm/caching/dual_cache.py	Adds default_in_memory_ttl injection guard to async_set_cache and async_set_cache_pipeline, matching the existing sync set_cache pattern. Changes are minimal (4 lines total) and correctly replicate the existing logic.
tests/test_litellm/caching/test_dual_cache.py	Adds 4 well-structured unit tests covering the TTL injection fix. Tests use real InMemoryCache objects (no network calls) and verify both the default injection and explicit TTL override behaviors, as well as sync/async parity.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["DualCache.async_set_cache / async_set_cache_pipeline called"] --> B{"'ttl' in kwargs?"}
    B -- Yes --> D["Use explicit TTL"]
    B -- No --> C{"default_in_memory_ttl set?"}
    C -- Yes --> E["Inject kwargs['ttl'] = default_in_memory_ttl"]
    C -- No --> F["No TTL injected — InMemoryCache uses its own default_ttl"]
    E --> G["InMemoryCache.async_set_cache with injected TTL"]
    D --> G
    F --> G
    G --> H{"redis_cache present and not local_only?"}
    H -- Yes --> I["RedisCache.async_set_cache with same kwargs"]
    H -- No --> J["Done"]
    I --> J

Loading

_{Last reviewed commit: 9766b52}

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}