Skip to content

refactor(cache): remove dead Cache._native_cache runtime path - #44785

Merged
yujonglee-berri merged 1 commit into
mainfrom
litellm_remove_native_cache_runtime
Oct 6, 2026
Merged

yujonglee-berri merged 1 commit into
mainfrom
litellm_remove_native_cache_runtime

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Summary

Cache._native_cache has had no assigner since #43601, and the V2 cache (litellm/_v2/cache plus python-bridge/src/cache/native/v2.rs and NativeCacheHandle) replaced it. Everything reachable only through that slot is removed

Python: the _native_cache attribute and its branches in get_cache, async_get_cache, add_cache, async_add_cache and async_add_cache_pipeline, the _native_response, _native_request and _stamp_semantic_similarity helpers, litellm/rust_bridge/response_cache.py, and the _ResponseCacheRuntime stub in _native.pyi. Python cache reads and writes keep going through self.cache / dynamic_cache_object exactly as before

Rust (python-bridge/src/cache/): runtime.rs (_ResponseCacheRuntime, CacheBinding), future.rs, python/callback.rs, and native/{activation,backend,config,embedder,facade,identity,request,semantic}.rs, plus the _ResponseCacheRuntime registration in lib.rs. native/v2.rs used super::request::now(), so it now has its own now(). selection.rs, python/{host,service}.rs and native/v2.rs stay

Deps dropped from python-bridge only: litellm-cache-{azure-blob,s3,gcs,disk,redis-semantic,qdrant-semantic,valkey-semantic}, qdrant-client, url, and the sha2 dev-dependency. The crates themselves and their tests are unchanged

Tests: tests/test_litellm_rust/cache/test_{redis,s3,gcs,azure_blob,disk,redis_semantic,valkey_semantic,qdrant_semantic,facade,rollout}.py, support/s3_stub.py with its fixture, and the activate_native / native_runtime / CacheTestResolver helpers in support/cache.py

Verified locally: cargo clippy -p litellm-python-bridge --all-targets -- -D warnings clean, tests/unit/caching 588 passed, tests/test_litellm_rust/cache + tests/unit/rust_bridge 1286 passed with LITELLM_RUST=1 on a rebuilt extension, make check PASS

Link to Devin session: https://app.devin.ai/sessions/2ace2cec0f4e485f8cad590dbf4a1473
Open in Devin Desktop: https://app.devin.ai/desktop/session/2ace2cec0f4e485f8cad590dbf4a1473?variant=devin
Requested by: @yujonglee-berri

The native response-cache runtime attached through Cache._native_cache is
unreachable since the V2 cache replaced it. Drop the Python branches and
wrapper, the _ResponseCacheRuntime pyclass and its backend/activation/
semantic modules, the python-bridge deps only they used, and the tests and
fixtures dedicated to that path.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Removes large native cache backend implementation from Rust bridge.

The PR appears safe to merge.

Summary

The PR removes the unused Cache._native_cache runtime path and its Python and Rust bindings, dependencies, and tests. The V2 native cache handle and the existing Python cache fallback remain.

Reviews (1) · Last reviewed commit: "refactor(cache): remove dead Cache._nati..."

@codecov

codecov Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_remove_native_cache_runtime (d36aba7) with main (a983b2a)

Open in CodSpeed

@yujonglee-berri
yujonglee-berri merged commit 5f9eff5 into main Oct 6, 2026
109 of 110 checks passed
@yujonglee-berri
yujonglee-berri deleted the litellm_remove_native_cache_runtime branch October 6, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant