fix(proxy): track spend for OpenAI passthrough /v1/embeddings

[lostmartian]

TLDR

Problem this solves:

• OpenAI passthrough embeddings returned 200 with no key spend
• Budget limits could be bypassed on /openai_passthrough/v1/embeddings

How it solves it:

• Bill OpenAI passthrough /v1/embeddings like other supported OpenAI routes
• Stop Cohere /v1/embed matching from stealing OpenAI /v1/embeddings

User Flow

Before: a developer embeds via OpenAI passthrough and their key spend never moves, so budgets are under-enforced

1. They send POST http://localhost:4000/openai_passthrough/v1/embeddings with {"model":"text-embedding-3-small","input":"PROOF_SENTINEL_TEXT"} using a virtual key
2. The response is HTTP 200 with usage.prompt_tokens: 6
3. GET http://localhost:4000/key/info for that key still shows the same info.spend after flush
4. Another caller can keep embedding on that route without the key budget catching the usage

After: the same passthrough embedding increases key spend by the embedding cost

1. They send the same POST http://localhost:4000/openai_passthrough/v1/embeddings with {"model":"text-embedding-3-small","input":"PROOF_SENTINEL_TEXT"}
2. The response is still HTTP 200 with usage.prompt_tokens: 6
3. After spend flush, GET http://localhost:4000/key/info shows info.spend increased by 1.2e-7
4. That usage now counts against the key budget

Relevant issues

Fixes #36646

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Proxy:

python litellm/proxy/proxy_cli.py \
  --config litellm/proxy/dev_config.yaml \
  --detailed_debug --reload --use_v2_migration_resolver

Before (first live passthrough run on this branch while the proxy was still serving pre-fix behavior; HEAD f64479e74d)

Translated control billed correctly:

curl -i -X POST "http://localhost:4000/v1/embeddings" \
  -H "Authorization: Bearer $VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"text-embedding-3-small","input":"PROOF_SENTINEL_TEXT"}'

Result:

HTTP/1.1 200 OK
x-litellm-response-cost: 1.2e-07

Passthrough returned success but did not bill the key:

curl -i -X POST "http://localhost:4000/openai_passthrough/v1/embeddings" \
  -H "Authorization: Bearer $VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"text-embedding-3-small","input":"PROOF_SENTINEL_TEXT"}'

Result:

HTTP/1.1 200 OK

Key spend did not increase until the proxy was restarted onto the fix. Missing x-litellm-response-cost on passthrough is expected on this path and is not the success signal for this bug

After (commit 1eaca98690acfefe9f0640d5940b6b4a0324b530)

echo "commit=$(git rev-parse HEAD)"
echo "spend_before=$(curl -s http://localhost:4000/key/info -H "Authorization: Bearer $VIRTUAL_KEY" | jq '.info.spend')"

curl -s -D - -o /tmp/emb_pt_body.json -X POST "http://localhost:4000/openai_passthrough/v1/embeddings" \
  -H "Authorization: Bearer $VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"text-embedding-3-small","input":"PROOF_SENTINEL_TEXT"}' \
  | head -n 20

jq '{model, usage}' /tmp/emb_pt_body.json
sleep 90
echo "spend_after=$(curl -s http://localhost:4000/key/info -H "Authorization: Bearer $VIRTUAL_KEY" | jq '.info.spend')"

Result:

commit=1eaca98690acfefe9f0640d5940b6b4a0324b530
spend_before=0.0
HTTP/1.1 200 OK
usage: {"prompt_tokens": 6, "total_tokens": 6}
spend_after=1.2E-7

Translated control on the same key/model still works: HTTP 200 with x-litellm-response-cost: 1.2e-07

A second live run on the same fix also showed spend move 2.4e-7 -> 3.6e-7 (+1.2e-7)

Type

🐛 Bug Fix
✅ Test

Caveats (if any)

• Classic Azure /openai/deployments/.../embeddings is out of scope
• Passthrough often omits x-litellm-response-cost because cost is computed after headers are sent; key spend after flush is the success signal for this bug

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

The PR adds spend accounting for OpenAI passthrough embedding requests and prevents the Cohere embed matcher from intercepting that route.

• Recognizes OpenAI-compatible /v1/embeddings requests in passthrough dispatch.
• Converts embedding responses and calculates their token-based cost.
• Refines Cohere route matching and adds focused regression coverage.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/pass_through_endpoints/llm_provider_handlers/openai_passthrough_logging_handler.py	Adds OpenAI embedding-response normalization and cost calculation to the existing passthrough logging handler.
litellm/proxy/pass_through_endpoints/llm_provider_handlers/cohere_passthrough_logging_handler.py	Excludes OpenAI’s plural embeddings route from Cohere’s singular embed handling.
litellm/proxy/pass_through_endpoints/success_handler.py	Dispatches supported OpenAI embedding requests correctly while refining Cohere route detection.
tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_openai_passthrough_logging_handler.py	Adds coverage for OpenAI passthrough embedding recognition, conversion, and cost propagation.
tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_cohere_passthrough_logging_handler.py	Adds regression coverage ensuring OpenAI embeddings do not enter Cohere embed processing.

_{Reviews (3): Last reviewed commit: "fix(proxy): drop unreachable embeddings ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[lostmartian]

@greptileai please rereview

[lostmartian]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing lostmartian:litellm_openai_passthrough_embeddings_spend (d5a5724) with litellm_internal_staging (0ca0fa2)}

[mateo-berri]

Just tested it. Works on a DB-backed proxy: /openai_passthrough/v1/embeddings now writes a spend log and bills the key exactly (6 tokens, $0.00000012), while before this pr, we bill nothing

[mateo-berri]

LGTM. Thanks for the contribution!