fix(caching): stamp provider on embedding cache-hit spend logs

[devin-ai-integration]

TLDR

Problem this solves:

• Fully-cached embedding requests log provider as None/unknown
• Spend logs and daily-spend rows then miss the provider

How it solves it:

• Forward the resolved custom_llm_provider into the logging obj on the full embedding cache-hit path

Relevant issues

Linear ticket

Resolves LIT-5015

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy with local caching on and an openai/text-embedding-3-small deployment, backed by Postgres, hitting the real OpenAI embeddings API. The same request is sent twice; the second is a full cache hit. We then read the provider straight out of the LiteLLM_SpendLogs table.

Config:

model_list:
  - model_name: text-embedding-3-small
    litellm_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY
litellm_settings:
  cache: true
  cache_params:
    type: local
general_settings:
  master_key: sk-1234

Requests (same for before and after):

BODY='{"model":"text-embedding-3-small","input":"<phrase>"}'
curl -s -X POST localhost:4000/v1/embeddings -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d "$BODY"   # miss
curl -s -X POST localhost:4000/v1/embeddings -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d "$BODY"   # full cache hit

Before (fix line reverted; parent of 15c7d850e5), the cache_hit=True row logs an empty provider:

substring | custom_llm_provider | cache_hit
91d6f382-84db-4fb5-bda | openai | None
9241e9e3-8bd1-4c1c-b4d |        | True     <- provider dropped on cache hit

The same shows up in the daily aggregate (LiteLLM_DailyTeamSpend), where the cache-hit tokens land under a null provider:

model                         | custom_llm_provider | prompt_tokens
openai/text-embedding-3-small |                     | 6              <- cache hit, no provider
openai/text-embedding-3-small | openai              | 30

After (fix at commit 15c7d850e5), the cache_hit=True row records the provider:

substring | custom_llm_provider | cache_hit
956d2009-7a23-4fed-a65 | openai | None
546fccbb-c83c-4d8c-a02 | openai | True     <- provider now stamped on cache hit

Type

🐛 Bug Fix

Changes

LLMCachingHandler._process_async_embedding_cached_response handles the case where every input in an embedding request is a cache hit. It resolves the provider with litellm.get_llm_provider(...), but when it then calls _update_litellm_logging_obj_environment(...) it never forwarded custom_llm_provider, so the arg defaulted to None. That left model_call_details["custom_llm_provider"] unset, so the SpendLogs row (custom_llm_provider column, and the model prefix from reconstruct_model_name) and the daily-spend aggregates recorded provider as None/unknown

Every other cache-hit path (the single-result branch and the sync branch) already passes custom_llm_provider through; this branch was the outlier. The fix forwards the already-resolved value:

self._update_litellm_logging_obj_environment(
    logging_obj=logging_obj,
    model=model,
    kwargs=kwargs,
    cached_result=final_embedding_cached_response,
    is_async=True,
    is_embedding=True,
    custom_llm_provider=custom_llm_provider,
)

Added a regression test asserting the logging obj ends up with custom_llm_provider == "openai" after a full embedding cache hit; it fails before the one-line fix and passes after

QA runbook

• tests/test_litellm/caching/test_caching_handler.py::test_embedding_cache_hit_sets_custom_llm_provider_on_logging_obj - a fully-cached embedding request stamps the resolved provider onto the logging obj so spend logs record it
  • Configure a proxy with an openai/text-embedding-3-small deployment and caching enabled (cache: true, cache_params.type: local)
  • POST the same /v1/embeddings request twice with one string input so the second is a full cache hit
  • Query LiteLLM_SpendLogs (or http://localhost:4000/ui/?page=logs) and confirm the cache_hit=True row shows provider openai, not empty/None
  • Sanity check: this test makes sense to add and is not hand-wavey; it fails before the fix and passes after

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/f17d0ffcdaf242b8ab2b995a061d9b73

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

This PR corrects provider attribution for fully cached asynchronous embedding requests.

• Passes the already-resolved custom_llm_provider into the logging environment update.
• Adds a regression test confirming that an OpenAI embedding cache hit records the provider in logging metadata.

Confidence Score: 5/5

The PR appears safe to merge with no actionable defects identified.

The change forwards an already-resolved provider into existing logging metadata without altering cache responses or provider selection, and the regression test directly verifies the corrected value.

Important Files Changed

Filename	Overview
litellm/caching/caching_handler.py	Forwards the resolved provider through the full async embedding cache-hit logging path, matching the established cache-hit behavior.
tests/test_litellm/caching/test_caching_handler.py	Adds a focused regression test that exercises the changed branch and verifies the provider stored in model-call details.

_{Reviews (1): Last reviewed commit: "fix(caching): stamp provider on embeddin..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_embedding_cache_provider (15c7d85) with litellm_internal_staging (4eecf7a)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (6794755) during the generation of this report, so 4eecf7a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

LGTM