refactor(s3_vectors): embed search queries through the shared vector store executor by mateo-berri · Pull Request #39474 · BerriAI/litellm

mateo-berri · 2026-09-03T02:27:59Z

TLDR

Problem this solves:

S3 Vectors embedded search queries with its own Router lookup, not the shared executor
That embedding call never carried the caller's team or key, so its spend went unattributed
Every vector store transform carried a router kwarg that only S3 Vectors read
The shared executor routed a store that carried no embedding configuration at all straight to the Router, so it raised "no healthy deployments" instead of falling back to the SDK

How it solves it:

S3 Vectors now subclasses BaseQueryEmbeddingVectorStoreConfig and embeds through the executor
The Router executor forwards the request's team and key metadata on the embedding call
embedding_model stays accepted as an alias of litellm_embedding_model
The router kwarg is gone from the handler and all provider transforms
The executor is back to one rule, route when the Router serves the model and embed through the SDK otherwise, with the request metadata attached either way
_embedding_router.py stays: the redis, qdrant, and valkey semantic caches still use it

User Flow

Before: a developer on a team key searches an S3 Vectors store whose embedding model is a Router alias; the search works, but its query embedding never shows up in the team's spend logs, or anyone's

The admin adds qa-team-embeddings (OpenAI text-embedding-3-small) to model_list and starts the proxy
A developer on a team key sends POST https://litellm-domain/v1/rag/ingest with a document and ingest_options naming custom_llm_provider: "s3_vectors", the bucket, the region, and embedding_model: "qa-team-embeddings"; 200 with a vector_store_id like lit6750-qa-vectors:litellm-index-39e9592a
The developer sends POST https://litellm-domain/v1/vector_stores/{vector_store_id}/search with the team key and {"query": "What must be drained before flipping DNS?"}; 200 with the matching chunk
The developer sends POST https://litellm-domain/v1/chat/completions with vector_store_ids: ["<that id>"] and the same question; 200 with "The Osaka queue must be drained before flipping DNS."
The admin opens GET https://litellm-domain/spend/logs/ui?team_id=: the search and the chat completion are listed, but there is no embedding row for the query, on that team or any other

After: the same search embeds through the Router as the developer's team and key, so the query embedding shows up under the team's spend

The admin adds qa-team-embeddings (OpenAI text-embedding-3-small) to model_list and starts the proxy
A developer on a team key sends POST https://litellm-domain/v1/rag/ingest with a document and ingest_options naming custom_llm_provider: "s3_vectors", the bucket, the region, and embedding_model: "qa-team-embeddings"; 200 with a vector_store_id like lit6750-qa-vectors:litellm-index-39e9592a
The developer sends POST https://litellm-domain/v1/vector_stores/{vector_store_id}/search with the team key and {"query": "What must be drained before flipping DNS?"}; 200 with the matching chunk
The developer sends POST https://litellm-domain/v1/chat/completions with vector_store_ids: ["<that id>"] and the same question; 200 with "The Osaka queue must be drained before flipping DNS."
The admin opens GET https://litellm-domain/spend/logs/ui?team_id=: next to the search there is now an aembedding row for qa-team-embeddings carrying the team id and the team key's hash

Relevant issues

Follow-up to #34788 and #38936

Linear ticket

Resolves LIT-6750

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup: two proxy processes with two uvicorn workers each on one Postgres, model_list carrying gpt-5.4-mini (openai/gpt-5.4-mini) and qa-team-embeddings (openai/text-embedding-3-small), store_model_in_db: true, a real S3 Vectors bucket lit6750-qa-vectors in eu-central-1, and a one-line document lit6750-doc.txt ("The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl."). Real OpenAI and AWS calls, no mocks. Search, chat, and rag/query answer from the document on both sides; the difference is in the last two spend-log cases: Before has no embedding row anywhere for the four query embeddings, After has one aembedding row per direct search and per chat completion, all carrying the team id and the team key hash (rag/query bills its search into its own aquery row by design, so it has no separate embedding row on either side)

Before (`9e8e486`)

team + team key on p1 (port 34446)

curl -s http://127.0.0.1:34446/team/new -H 'Authorization: Bearer <master key>' -d '{"team_alias":"lit6750-team-before"}'
```
{"team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "team_alias": "lit6750-team-before"}
```

curl -s http://127.0.0.1:34446/key/generate -H 'Authorization: Bearer <master key>' -d '{"team_id":"c05b3300-af28-4e8d-ab88-9af8de59205a","key_alias":"lit6750-key-before"}'

{"key": "sk-...(team key, alias lit6750-key-before)", "token": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a"}

ingest on p1 with the team key, embedding model = bare Router alias qa-team-embeddings

curl -s http://127.0.0.1:34446/v1/rag/ingest -H 'Authorization: Bearer <team key>' -F file=@lit6750-doc.txt -F 'request={"ingest_options":{"embedding":{"model":"qa-team-embeddings"},"vector_store":{"custom_llm_provider":"s3_vectors","vector_bucket_name":"lit6750-qa-vectors","aws_region_name":"eu-central-1","embedding_model":"qa-team-embeddings"}}}'
```
{"id":"ingest_bf49328c-adbd-4ef0-85e7-e97ef5ff6085","status":"completed","vector_store_id":"lit6750-qa-vectors:litellm-index-39e9592a","file_id":"1b2c983cda254b76a9537aa4800203db.txt"}
```

chat completion with vector_store_ids on p2 (port 31642), then on p1 (port 34446), team key

curl -s http://127.0.0.1:31642/v1/chat/completions -H 'Authorization: Bearer <team key>' -d '{"model":"gpt-5.4-mini","vector_store_ids":["lit6750-qa-vectors:litellm-index-39e9592a"],"messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook? Answer in one sentence."}]}'
```
{"id": "chatcmpl-EJrNWHQUJR3mSQGVGEhe9b50qWSYi", "content": "The Osaka queue must be drained before flipping DNS."}
```
curl -s http://127.0.0.1:34446/v1/chat/completions -H 'Authorization: Bearer <team key>' -d '{"model":"gpt-5.4-mini","vector_store_ids":["lit6750-qa-vectors:litellm-index-39e9592a"],"messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook? Answer in one sentence."}]}'
```
{"id": "chatcmpl-EJrNae86e9o5u3jsAvcAPfByi5KGI", "content": "The Osaka queue must be drained before flipping DNS."}
```

direct search, POST /v1/vector_stores/{id}/search on p2 with the team key

curl -s http://127.0.0.1:31642/v1/vector_stores/lit6750-qa-vectors:litellm-index-39e9592a/search -H 'Authorization: Bearer <team key>' -d '{"query":"What must be drained before flipping DNS?"}'
```
{"object": "vector_store.search_results.page", "top": [[0.498, "The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl."]]}
```

control: POST /v1/rag/query on p1 with the team key

curl -s http://127.0.0.1:34446/v1/rag/query -H 'Authorization: Bearer <team key>' -d '{"model":"gpt-5.4-mini","retrieval_config":{"vector_store_id":"lit6750-qa-vectors:litellm-index-39e9592a"},"messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook? Answer in one sentence."}]}'
```
{"content": "The Osaka queue must be drained before flipping DNS.", "search_results": ["The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl."]}
```

spend attribution of the query embeddings: GET /spend/logs/ui on p2, filtered to the team

curl -s 'http://127.0.0.1:31642/spend/logs/ui?start_date=2026-09-03%2002:20:45&end_date=2026-09-04%2000:00:00&page_size=100&team_id=c05b3300-af28-4e8d-ab88-9af8de59205a' -H 'Authorization: Bearer <master key>'

6 rows for team_id=c05b3300-af28-4e8d-ab88-9af8de59205a
{"call_type": "aingest", "model_group": "", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "api_key": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "spend": 0.0}
{"call_type": "acompletion", "model_group": "gpt-5.4-mini", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "api_key": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "spend": 9.9e-05}
{"call_type": "asearch", "model_group": "gpt-5.4-mini", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "api_key": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "spend": 0.0}
{"call_type": "acompletion", "model_group": "gpt-5.4-mini", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "api_key": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "spend": 9.9e-05}
{"call_type": "asearch", "model_group": "gpt-5.4-mini", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "api_key": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "spend": 0.0}
{"call_type": "avector_store_search", "model_group": "", "team_id": "c05b3300-af28-4e8d-ab88-9af8de59205a", "api_key": "596fd339874c48a1a68383bf08bbf387304c56fe8b903052cfd27185225fe28e", "spend": 0.0}

same window, every embedding row regardless of team (unattributed rows show up here)

curl -s 'http://127.0.0.1:31642/spend/logs/ui?start_date=2026-09-03%2002:20:45&end_date=2026-09-04%2000:00:00&page_size=200' -H 'Authorization: Bearer <master key>'
```
0 embedding rows in the window
```

After (`7f7e0d5`)

team + team key on p1 (port 34446)

curl -s http://127.0.0.1:34446/team/new -H 'Authorization: Bearer <master key>' -d '{"team_alias":"lit6750-team-after2"}'
```
{"team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "team_alias": "lit6750-team-after2"}
```

curl -s http://127.0.0.1:34446/key/generate -H 'Authorization: Bearer <master key>' -d '{"team_id":"ce42fe38-d3ef-44e6-b651-f3bbd5a70f29","key_alias":"lit6750-key-after2"}'

{"key": "sk-...(team key, alias lit6750-key-after2)", "token": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29"}

ingest on p1 with the team key, embedding model = bare Router alias qa-team-embeddings

curl -s http://127.0.0.1:34446/v1/rag/ingest -H 'Authorization: Bearer <team key>' -F file=@lit6750-doc.txt -F 'request={"ingest_options":{"embedding":{"model":"qa-team-embeddings"},"vector_store":{"custom_llm_provider":"s3_vectors","vector_bucket_name":"lit6750-qa-vectors","aws_region_name":"eu-central-1","embedding_model":"qa-team-embeddings"}}}'
```
{"id":"ingest_8b938e58-5b86-431c-8f41-314a9eaccb6b","status":"completed","vector_store_id":"lit6750-qa-vectors:litellm-index-e4e42d8a","file_id":"d3ad3b8417a6471ca79fd81882a6470c.txt"}
```

chat completion with vector_store_ids on p2 (port 31642), then on p1 (port 34446), team key

curl -s http://127.0.0.1:31642/v1/chat/completions -H 'Authorization: Bearer <team key>' -d '{"model":"gpt-5.4-mini","vector_store_ids":["lit6750-qa-vectors:litellm-index-e4e42d8a"],"messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook? Answer in one sentence."}]}'
```
{"id": "chatcmpl-EJtBhBwwnFx2dgP6kscliTDuW99tA", "content": "The Osaka queue must be drained before flipping DNS."}
```
curl -s http://127.0.0.1:34446/v1/chat/completions -H 'Authorization: Bearer <team key>' -d '{"model":"gpt-5.4-mini","vector_store_ids":["lit6750-qa-vectors:litellm-index-e4e42d8a"],"messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook? Answer in one sentence."}]}'
```
{"id": "chatcmpl-EJtBkrNNvdr8ymmaBlEc9A4Lz4jl6", "content": "The Osaka queue must be drained before flipping DNS."}
```

direct search, POST /v1/vector_stores/{id}/search on p2 with the team key

curl -s http://127.0.0.1:31642/v1/vector_stores/lit6750-qa-vectors:litellm-index-e4e42d8a/search -H 'Authorization: Bearer <team key>' -d '{"query":"What must be drained before flipping DNS?"}'
```
{"object": "vector_store.search_results.page", "top": [[0.498, "The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl."]]}
```

control: POST /v1/rag/query on p1 with the team key

curl -s http://127.0.0.1:34446/v1/rag/query -H 'Authorization: Bearer <team key>' -d '{"model":"gpt-5.4-mini","retrieval_config":{"vector_store_id":"lit6750-qa-vectors:litellm-index-e4e42d8a"},"messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook? Answer in one sentence."}]}'
```
{"content": "The Osaka queue must be drained before flipping DNS.", "search_results": ["The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl."]}
```

spend attribution of the query embeddings: GET /spend/logs/ui on p2, filtered to the team

curl -s 'http://127.0.0.1:31642/spend/logs/ui?start_date=2026-09-03%2004:16:51&end_date=2026-09-04%2000:00:00&page_size=100&team_id=ce42fe38-d3ef-44e6-b651-f3bbd5a70f29' -H 'Authorization: Bearer <master key>'

10 rows for team_id=ce42fe38-d3ef-44e6-b651-f3bbd5a70f29
{"call_type": "aingest", "model_group": "", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 0.0}
{"call_type": "acompletion", "model_group": "gpt-5.4-mini", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 9.9e-05}
{"call_type": "asearch", "model_group": "gpt-5.4-mini", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 0.0}
{"call_type": "aembedding", "model_group": "qa-team-embeddings", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 4e-07}
{"call_type": "acompletion", "model_group": "gpt-5.4-mini", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 9.9e-05}
{"call_type": "asearch", "model_group": "gpt-5.4-mini", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 0.0}
{"call_type": "aembedding", "model_group": "qa-team-embeddings", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 4e-07}
{"call_type": "avector_store_search", "model_group": "", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 0.0}
{"call_type": "aembedding", "model_group": "qa-team-embeddings", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 1.6e-07}
{"call_type": "aquery", "model_group": "gpt-5.4-mini", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 9.9e-05}

same window, every embedding row regardless of team (unattributed rows show up here)

curl -s 'http://127.0.0.1:31642/spend/logs/ui?start_date=2026-09-03%2004:16:51&end_date=2026-09-04%2000:00:00&page_size=200' -H 'Authorization: Bearer <master key>'

3 embedding rows in the window
{"call_type": "aembedding", "model_group": "qa-team-embeddings", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 4e-07}
{"call_type": "aembedding", "model_group": "qa-team-embeddings", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 4e-07}
{"call_type": "aembedding", "model_group": "qa-team-embeddings", "team_id": "ce42fe38-d3ef-44e6-b651-f3bbd5a70f29", "api_key": "b9b1942b987b363ddf064222d88ea5ee8758f9074487c68e2a54b3030c8f1859", "spend": 1.6e-07}

Regression guard: a store registered with no embedding model, base vs head

The s3_vectors default embedding model is text-embedding-3-small, and this proxy's model_list does not serve it under that name, so the search has to fall back to the SDK. For this leg only, port 34446 ran the merge base (9e8e486) and port 31642 ran the head (7f7e0d5), sharing the one Postgres and the one S3 Vectors bucket. Both sides answer the same

ingest with NO embedding_model on the vector store registration (search falls back to the s3_vectors default text-embedding-3-small, which model_list does not serve)

curl -s http://127.0.0.1:34446/v1/rag/ingest -H 'Authorization: Bearer <team key>' -F file=@lit6750-doc.txt -F 'request={"ingest_options":{"embedding":{"model":"qa-team-embeddings"},"vector_store":{"custom_llm_provider":"s3_vectors","vector_bucket_name":"lit6750-qa-vectors","aws_region_name":"eu-central-1"}}}'
```
{"id":"ingest_b8683331-0ad1-4f6f-8f1d-ff9ea214fd93","status":"completed","vector_store_id":"lit6750-qa-vectors:litellm-index-4be4fe28","file_id":"006a2bbdd3cb445db579c43a8e5a2f37.txt"}
```

search that store on the base proxy (port 34446)

curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:34446/v1/vector_stores/lit6750-qa-vectors:litellm-index-4be4fe28/search -H 'Authorization: Bearer <team key>' -d '{"query":"What must be drained before flipping DNS?"}'
```
{"http": "200", "top": [[0.498, "The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-ca"]]}
```

search that store on the head proxy (port 31642)

curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:31642/v1/vector_stores/lit6750-qa-vectors:litellm-index-4be4fe28/search -H 'Authorization: Bearer <team key>' -d '{"query":"What must be drained before flipping DNS?"}'
```
{"http": "200", "top": [[0.498, "The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-ca"]]}
```

Type

🐛 Bug Fix
🧹 Refactoring

Caveats (if any)

Low

Team-scoped /model/new aliases were not driven live
- The local enterprise license expired, so the alias sits in model_list
- The unit test covers the team metadata forwarding
Overlaps feat(vector-stores): support Milvus gRPC search #39039 on the Milvus transform signature
- Whichever lands second re-applies a one-line router kwarg removal
The route-or-SDK fallback was proven live on S3 Vectors only
- Milvus, Azure AI, and Valkey share it, covered by executor unit tests
Five tests/vector_store_tests live-API failures also fail at the merge base
- Azure DeploymentNotFound, and a missing Bedrock knowledge base

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Note

Medium Risk
Changes vector store search embedding routing and spend attribution across providers; behavior shifts for unserved embedding models (SDK fallback instead of errors) and any code that passed router into transform signatures.

Overview
Vector store search no longer passes a bare router into provider transforms or the HTTP handler. search() builds a RouterVectorStoreEmbeddingExecutor (with request litellm_metadata) and hands that as embedding_executor, so query embeddings can be attributed to the calling team/key (LIT-6750).

S3 Vectors now subclasses BaseQueryEmbeddingVectorStoreConfig and embeds queries through the same executor path as Azure AI/Milvus instead of its own Router lookup via resolve_embedding_router. It still accepts embedding_model as an alias of litellm_embedding_model and defaults to text-embedding-3-small when unset.

RouterVectorStoreEmbeddingExecutor simplifies to one rule: use the Router when it serves the model, otherwise call the LiteLLM SDK—with caller metadata attached in both cases. Unserved models no longer error; they fall back to SDK embedding (fixes stores with no named embedding model on proxies that don’t list the default).

embed_query / aembed_query on the base query-embedding config only use the injected executor (no router argument at the transform layer).

^{Reviewed by Cursor Bugbot for commit 7f7e0d5. Bugbot is set up for automated code reviews on this repo. Configure here.}

…store executor S3 Vectors now subclasses BaseQueryEmbeddingVectorStoreConfig, so its query embedding runs through the Router executor with the request metadata instead of a private router lookup. embedding_model stays accepted as an alias of litellm_embedding_model. The router kwarg is gone from the search handler and every provider transform now that nothing but the executor fallback read it.

codspeed-hq · 2026-09-03T02:30:16Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_s3_vectors_query_embedding_executor (7f7e0d5) with litellm_internal_staging (291d02f)¹}

No successful run was found on litellm_internal_staging (ff17e8b) during the generation of this report, so 291d02f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

greptile-apps · 2026-09-03T02:30:56Z

Greptile Summary

The PR routes S3 Vectors query embeddings through the shared vector-store executor so Router aliases, SDK fallback, and request metadata use the common path.

Adds S3 Vectors support for the shared query-embedding configuration and executor.
Propagates team and API-key metadata into embedding requests for spend attribution.
Removes the obsolete router argument from vector-store transformation interfaces and provider implementations.
Adds regression coverage for Router alias resolution, SDK fallback, metadata forwarding, and embedding-model aliases.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/base_llm/vector_store/transformation.py	Simplifies the shared executor to route models served by the Router and otherwise fall back to the SDK while preserving request metadata.
litellm/llms/s3_vectors/vector_stores/transformation.py	Migrates S3 Vectors query embedding to the shared executor and retains the configured/default embedding-model behavior.
litellm/vector_stores/main.py	Constructs and forwards the metadata-aware embedding executor through vector-store search handling.
litellm/llms/custom_httpx/llm_http_handler.py	Removes direct Router plumbing from vector-store handlers in favor of the supplied shared embedding executor.
tests/test_litellm/llms/s3_vectors/vector_stores/test_s3_vectors_transformation.py	Removes the previously flagged redundant docstrings without weakening assertions and adds SDK-fallback regression coverage.
tests/router_unit_tests/test_router_embedding_integration.py	Covers Router selection and request metadata forwarding for shared vector-store embeddings.
tests/test_litellm/vector_stores/test_main.py	Verifies executor construction and propagation through the public vector-store search path.

_{Reviews (2): Last reviewed commit: "fix(vector-store): embed through the SDK..." | Re-trigger Greptile}

codecov · 2026-09-03T02:35:55Z

Codecov Report

❌ Patch coverage is 94.73684% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ellm/llms/azure_ai/vector_stores/transformation.py	0.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

…ve the query embedding model The Router executor only routed a query embedding when the vector store carried extra embedding configuration, so a store registered with no embedding model at all always went to the Router and 500'd on the s3_vectors default text-embedding-3-small when no deployment served it. Route on whether the Router serves the model, which is the rule the executor had before, and keep the request metadata on the SDK fallback so the embedding stays attributed either way.

mateo-berri · 2026-09-03T04:32:53Z

@greptileai

mateo-berri · 2026-09-03T05:05:57Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 7f7e0d5. Configure here.}

tin-berri

LGTM

…hook fake handler The fake vector store search handler declared `embedding_executor` twice after BerriAI#39474 landed on staging, which is a syntax error under `ruff check --config ruff-tests.toml tests` and fails the required lint job for every PR targeting the branch. The handler is called by keyword, so keeping the first declaration is behavior-preserving. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LEHLJtc7bYodrmfXSAcBvh

…hook fake handler The fake vector store search handler declared `embedding_executor` twice after BerriAI#39474 landed on staging, which is a syntax error under `ruff check --config ruff-tests.toml tests` and fails the required lint job for every PR targeting the branch. The handler is called by keyword, so keeping the first declaration is behavior-preserving.

…B fake handler Two branches independently added embedding_executor to the same fake search handler in this file, BerriAI#39472 in the middle of the signature and BerriAI#39474 at the end. Neither conflicted with the other, so both edits merged and the function ended up declaring the parameter twice. Python rejects that at compile time, so the whole module fails to import and every test in the file is uncollectable, taking the logging_testing job down on staging. Keep the earlier of the two, which sits where the real handler declares the parameter.

greptile-apps Bot reviewed Sep 3, 2026

View reviewed changes

Comment thread tests/test_litellm/llms/s3_vectors/vector_stores/test_s3_vectors_transformation.py

mateo-berri added the run-ci label Sep 3, 2026

mateo-berri mentioned this pull request Sep 3, 2026

fix(vector-stores): survive a failing vector store search in the chat completions hook #39495

Merged

6 tasks

cursor Bot reviewed Sep 3, 2026

View reviewed changes

mateo-berri requested a review from tin-berri September 3, 2026 05:10

mateo-berri enabled auto-merge September 3, 2026 05:10

tin-berri approved these changes Sep 3, 2026

View reviewed changes

mateo-berri merged commit 66a3d24 into litellm_internal_staging Sep 3, 2026
115 of 117 checks passed

mateo-berri deleted the litellm_s3_vectors_query_embedding_executor branch September 3, 2026 05:23

mateo-berri mentioned this pull request Sep 3, 2026

refactor(typing): cut 1,397 Any errors across 183 backend files #39461

Merged

7 tasks

sowmy86 mentioned this pull request Sep 3, 2026

fix(transformation): merge consecutive system and developer messages for non-OpenAI providers and Responses API #39282

Closed

6 tasks

Navigation Menu

Uh oh!

Uh oh!

refactor(s3_vectors): embed search queries through the shared vector store executor - #39474

refactor(s3_vectors): embed search queries through the shared vector store executor#39474
mateo-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_s3_vectors_query_embedding_executor

mateo-berri commented Sep 3, 2026 •

edited by cursor Bot

Loading

Uh oh!

codspeed-hq Bot commented Sep 3, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 3, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Sep 3, 2026 •

edited

Loading

Uh oh!

mateo-berri commented Sep 3, 2026

Uh oh!

mateo-berri commented Sep 3, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

tin-berri left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

mateo-berri commented Sep 3, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (9e8e486)

team + team key on p1 (port 34446)

ingest on p1 with the team key, embedding model = bare Router alias qa-team-embeddings

chat completion with vector_store_ids on p2 (port 31642), then on p1 (port 34446), team key

direct search, POST /v1/vector_stores/{id}/search on p2 with the team key

control: POST /v1/rag/query on p1 with the team key

spend attribution of the query embeddings: GET /spend/logs/ui on p2, filtered to the team

same window, every embedding row regardless of team (unattributed rows show up here)

After (7f7e0d5)

team + team key on p1 (port 34446)

ingest on p1 with the team key, embedding model = bare Router alias qa-team-embeddings

chat completion with vector_store_ids on p2 (port 31642), then on p1 (port 34446), team key

direct search, POST /v1/vector_stores/{id}/search on p2 with the team key

control: POST /v1/rag/query on p1 with the team key

spend attribution of the query embeddings: GET /spend/logs/ui on p2, filtered to the team

same window, every embedding row regardless of team (unattributed rows show up here)

Regression guard: a store registered with no embedding model, base vs head

ingest with NO embedding_model on the vector store registration (search falls back to the s3_vectors default text-embedding-3-small, which model_list does not serve)

search that store on the base proxy (port 34446)

search that store on the head proxy (port 31642)

Type

Caveats (if any)

Low

Final Attestation

Uh oh!

codspeed-hq Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

greptile-apps Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mateo-berri commented Sep 3, 2026

Uh oh!

mateo-berri commented Sep 3, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

tin-berri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mateo-berri commented Sep 3, 2026 •

edited by cursor Bot

Loading

Before (`9e8e486`)

After (`7f7e0d5`)

codspeed-hq Bot commented Sep 3, 2026 •

edited

Loading

greptile-apps Bot commented Sep 3, 2026 •

edited

Loading

codecov Bot commented Sep 3, 2026 •

edited

Loading