docs(caching): document the semantic cache embedding deadline

[mateo-berri]

TLDR

Documents the deadline that BerriAI/litellm#37742 puts on the semantic cache's prompt embedding call.

That embedding runs inline before every request, so an embedding endpoint that is unreachable or hanging used to park the whole request for minutes. It is now capped at 5 seconds, tunable per cache with semantic_cache_embedding_timeout or globally with SEMANTIC_CACHE_EMBEDDING_TIMEOUT_SECONDS

Changes

A short section in docs/proxy/caching.md explaining what happens when the deadline fires (cache miss, x-litellm-semantic-similarity: 0.0, request continues to the model) and how to raise it, plus the two semantic cache_params in the supported-params block and a row in the environment variable reference table

Note on ordering

litellm's code-quality and documentation jobs check out this repo's default branch, so BerriAI/litellm#37742 stays red on SEMANTIC_CACHE_EMBEDDING_TIMEOUT_SECONDS until this merges

---

Note

Low Risk
Documentation-only; no runtime or security behavior changes.

Overview
Documents the 5-second deadline on the inline prompt-embedding call used by semantic caches. If the embedding endpoint hangs, the lookup is dropped as a miss (x-litellm-semantic-similarity: 0.0) and the request continues to the model.

Shows how to raise the deadline with semantic_cache_embedding_timeout under cache_params or SEMANTIC_CACHE_EMBEDDING_TIMEOUT_SECONDS, lists those params in the supported cache_params block, and adds the env var to the config settings table.

^{Reviewed by Cursor Bugbot for commit 6c67a2f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview	Aug 21, 2026 12:08am

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6c67a2f. Configure here.}