fix(vector_stores): s3 vectors search router bypass + rag query config drop + ui error swallow

[michelligabriele]

TLDR

Problem this solves:

• S3 Vectors stores ingest fine but cannot be queried
• Search-time embeddings bypass the Router, so virtual model names fail
• /v1/rag/query drops store config: "aws_region_name is required"
• UI renders any backend search error as "No results found"

How it solves it:

• Threads the Router down to vector store search transforms
• S3 Vectors resolves query embeddings via Router iff it serves the model
• /v1/rag/query reads the managed-store registry; store config always wins
• Only allowlisted retrieval_config keys reach the search call
• Proxy query surfaces reject caller-picked embedding model/credential keys
• Search region falls back env, then default, matching ingestion
• UI surfaces the backend error message in the test tab

User Flow

Before: an S3 Vectors store that ingests fine cannot be queried through /v1/rag/query, direct search, or the Admin UI test tab

1. The proxy admin ingests a document into a new S3 Vectors store: POST https://litellm-domain/v1/rag/ingest with a file and ingest_options naming custom_llm_provider: "s3_vectors", their bucket, region eu-central-1, and their gateway-served embedding model qa-team-embeddings; 200 with a fresh vector_store_id like my-bucket:litellm-index-5049eabb, and the store shows up on https://litellm-domain/ui/?page=vector-stores with its region and embedding model
2. They query it: POST https://litellm-domain/v1/rag/query with model: "gpt-4o-mini" and retrieval_config: {"vector_store_id": "..."}; 500 with an OpenAI "Invalid 'vector_store_id'" error, because the search went to OpenAI instead of their S3 Vectors store
3. They retry naming custom_llm_provider: "s3_vectors" in retrieval_config; 500 "aws_region_name is required for S3 Vectors" even though the store carries the region
4. They retry again with aws_region_name: "eu-central-1" pasted into retrieval_config; the same 500, the region never reaches the search
5. They try the store directly: POST https://litellm-domain/v1/vector_stores/{id}/search with a query; 400 "LLM Provider NOT provided ... model=qa-team-embeddings", so the ingested data is unreachable everywhere
6. On https://litellm-domain/ui/?page=vector-stores they open the store's Test Vector Store tab and run a search; the panel shows "No results found" even though the search actually errored with that same 400
7. Extra retrieval_config keys (api_base, api_key, embedding_model, litellm_credential_name) change nothing, since every key but the id is dropped on the way to the search, which is also why the region never arrives

After: the same store answers through /v1/rag/query and direct search, and the test tab shows results or the real error

1. The proxy admin ingests a document into a new S3 Vectors store: POST https://litellm-domain/v1/rag/ingest with the same file and ingest_options; 200 with a fresh vector_store_id like my-bucket:litellm-index-5a1ed2ac, and the store shows up on https://litellm-domain/ui/?page=vector-stores with its region and embedding model
2. They query it: POST https://litellm-domain/v1/rag/query with model: "gpt-4o-mini" and retrieval_config: {"vector_store_id": "..."} only; 200 with the matching chunk under choices[0].message.provider_specific_fields.search_results and the model's answer built from it, the store's provider, region, bucket, and embedding model applied automatically
3. Naming custom_llm_provider: "s3_vectors" in retrieval_config gives the same 200
4. Pasting aws_region_name: "eu-central-1" too gives the same 200
5. They try the store directly: POST https://litellm-domain/v1/vector_stores/{id}/search with a query; 200 with the matching chunk and its score
6. On https://litellm-domain/ui/?page=vector-stores the Test Vector Store tab lists the matching chunk with its score, and a search against a store whose index does not exist reads "Search failed: " instead of "No results found"
7. A caller who adds api_base, api_key, or another custom_llm_provider to retrieval_config gets the same 200 from the real store, since the store's own settings win, and one who adds embedding_model, litellm_credential_name, litellm_embedding_model, or litellm_embedding_config to retrieval_config or to the direct search body gets a 400 naming the key, so nobody can point a shared store's search at another host or spend a restricted embedding deployment through it

Relevant issues

Linear ticket

Resolves LIT-6612

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live e2e run against a real S3 Vectors bucket (lit6612-qa-vectors, eu-central-1) with real OpenAI embeddings and chat completions. Each leg is 2 separate proxy processes x 2 uvicorn workers (--num_workers 2) sharing one Postgres, requests alternating between the two processes (ingest on process 1, the store then read back through process 2), so the managed-store registry is proven across processes. Before: process 1 on port 47468, process 2 on 52978. After: process 1 on 51101, process 2 on 54746. AWS_BEARER_TOKEN_BEDROCK is unset in both legs so S3 Vectors signs with SigV4

config.yaml (both legs):

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
  - model_name: qa-team-embeddings
    litellm_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY
general_settings:
  master_key: sk-lit6612

lit6612-doc.txt (the ingested file): The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl.

Admin UI leg: the dashboard dev server (npm run dev in ui/litellm-dashboard) pointed at process 1 of each leg, vector stores page, Test Vector Store tab, query What must be drained before flipping DNS?. The broken-store case is a store registered through /vector_store/new with an index that does not exist (lit6612-qa-vectors:does-not-exist)

Before (695d943)

Ingest and cross-process listing

1. Ingest on process 1

$ curl -s http://127.0.0.1:47468/v1/rag/ingest -H 'Authorization: Bearer sk-lit6612' -F file=@lit6612-doc.txt -F 'request={"ingest_options":{"embedding":{"model":"qa-team-embeddings"},"vector_store":{"custom_llm_provider":"s3_vectors","vector_bucket_name":"lit6612-qa-vectors","aws_region_name":"eu-central-1","embedding_model":"qa-team-embeddings"}}}'
{"id":"ingest_34d0bc86-6e31-45d4-b6b6-005983bd40f9","status":"completed","vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","file_id":"c2ae781e04ee4e7f8bfe833563ed4ccd.txt"}

2. The store is listed with its region and embedding model on process 2

$ curl -s http://127.0.0.1:52978/vector_store/list -H "Authorization: Bearer sk-lit6612" | jq -c '.data[] | select(.vector_store_id=="lit6612-qa-vectors:litellm-index-5049eabb") | {vector_store_id, custom_llm_provider, litellm_params: (.litellm_params | {vector_bucket_name, aws_region_name, embedding_model})}'
{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","custom_llm_provider":"s3_vectors","litellm_params":{"vector_bucket_name":"lit6612-qa-vectors","aws_region_name":"eu-central-1","embedding_model":"qa-team-embeddings"}}

RAG query with bare retrieval_config

1. Only vector_store_id in retrieval_config, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:52978/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb"}}'
{"detail":{"error":"litellm.BadRequestError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Invalid 'vector_store_id': 'lit6612-qa-vectors:litellm-index-5049eabb'. Expected an ID that contains letters, numbers, underscores, or dashes, but this value contained additional characters.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"vector_store_id\",\n    \"code\": \"invalid_value\"\n  }\n}"}}
HTTP 500

RAG query naming the provider and region

1. custom_llm_provider: s3_vectors added, on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:47468/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","custom_llm_provider":"s3_vectors"}}' | cut -c1-400
{"detail":{"error":"litellm.APIConnectionError: aws_region_name is required for S3 Vectors\nTraceback (most recent call last):\n  File \".../litellm/vector_stores/main.py\", line 319, in asearch\n    response = await init_response\n ...
HTTP 500

2. aws_region_name: eu-central-1 pasted in too, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:52978/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","custom_llm_provider":"s3_vectors","aws_region_name":"eu-central-1"}}' | cut -c1-400
{"detail":{"error":"litellm.APIConnectionError: aws_region_name is required for S3 Vectors\nTraceback (most recent call last):\n  File \".../litellm/vector_stores/main.py\", line 319, in asearch\n    response = await init_response\n ...
HTTP 500

RAG query with hostile connection overrides

1. custom_llm_provider: openai, api_base pointing at a dead port, and a bogus api_key in retrieval_config, on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:47468/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","custom_llm_provider":"openai","api_base":"http://127.0.0.1:9/v1","api_key":"sk-attacker"}}' | cut -c1-400
{"detail":{"error":"litellm.BadRequestError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Invalid 'vector_store_id': 'lit6612-qa-vectors:litellm-index-5049eabb'. Expected an ID that contains letters, numbers, underscores, or dashes, but this value contained additional characters.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"vector_store_id\",\n    \"code\": \"invalid_v
HTTP 500

Caller-picked embedding keys

1. embedding_model in retrieval_config, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:52978/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","embedding_model":"text-embedding-3-large"}}' | cut -c1-400
{"detail":{"error":"litellm.BadRequestError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Invalid 'vector_store_id': 'lit6612-qa-vectors:litellm-index-5049eabb'. Expected an ID that contains letters, numbers, underscores, or dashes, but this value contained additional characters.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"vector_store_id\",\n    \"code\": \"invalid_v
HTTP 500

2. litellm_credential_name in retrieval_config, on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:47468/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5049eabb","litellm_credential_name":"someone-elses-aws-creds"}}' | cut -c1-400
{"detail":{"error":"litellm.BadRequestError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Invalid 'vector_store_id': 'lit6612-qa-vectors:litellm-index-5049eabb'. Expected an ID that contains letters, numbers, underscores, or dashes, but this value contained additional characters.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"vector_store_id\",\n    \"code\": \"invalid_v
HTTP 500

3. litellm_embedding_model in the direct search body, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:52978/v1/vector_stores/lit6612-qa-vectors:litellm-index-5049eabb/search -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"query":"Kyoto failover","litellm_embedding_model":"text-embedding-3-large"}' | cut -c1-400
{"error":{"message":"litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=qa-team-embeddings\n Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers","type":null,"param":null,"code":"400"}}
HTTP 400

Direct vector store search

1. Plain search on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:47468/v1/vector_stores/lit6612-qa-vectors:litellm-index-5049eabb/search -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"query":"What must be drained before flipping DNS?"}'
{"error":{"message":"litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=qa-team-embeddings\n Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers","type":null,"param":null,"code":"400"}}
HTTP 400

Admin UI Test Vector Store tab

1. Open the vector stores page, click lit6612-qa-vectors:litellm-index-5049eabb, open the Test Vector Store tab, enter the query, click Search: the panel reads "No results found" while the browser console shows the search request answered HTTP 400 with the "LLM Provider NOT provided ... model=qa-team-embeddings" error above. Screenshot: lit6612-before-ui-working-store.png
2. Same steps on lit6612-qa-vectors:does-not-exist: the panel reads "No results found" again, the 404 from S3 Vectors is swallowed. Screenshot: lit6612-before-ui-broken-store.png

After (8b0441a)

Ingest and cross-process listing

1. Ingest on process 1

$ curl -s http://127.0.0.1:51101/v1/rag/ingest -H 'Authorization: Bearer sk-lit6612' -F file=@lit6612-doc.txt -F 'request={"ingest_options":{"embedding":{"model":"qa-team-embeddings"},"vector_store":{"custom_llm_provider":"s3_vectors","vector_bucket_name":"lit6612-qa-vectors","aws_region_name":"eu-central-1","embedding_model":"qa-team-embeddings"}}}'
{"id":"ingest_680305fd-6de6-4ce2-b1fd-219f65985314","status":"completed","vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","file_id":"15937bc865d6498e99bb6fbee2b37d54.txt"}

2. The store is listed with its region and embedding model on process 2

$ curl -s http://127.0.0.1:54746/vector_store/list -H "Authorization: Bearer sk-lit6612" | jq -c '.data[] | select(.vector_store_id=="lit6612-qa-vectors:litellm-index-5a1ed2ac") | {vector_store_id, custom_llm_provider, litellm_params: (.litellm_params | {vector_bucket_name, aws_region_name, embedding_model})}'
{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","custom_llm_provider":"s3_vectors","litellm_params":{"vector_bucket_name":"lit6612-qa-vectors","aws_region_name":"eu-central-1","embedding_model":"qa-team-embeddings"}}

RAG query with bare retrieval_config

1. Only vector_store_id in retrieval_config, on process 2 (the process that did not ingest)

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:54746/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac"}}'
{"id":"chatcmpl-EJkRtYZZ2GC9tWf5SwrpItm0GwDV3","created":1788375489,"model":"gpt-4o-mini-2024-07-18","object":"chat.completion","system_fingerprint":"fp_7ffe4e4620","choices":[{"finish_reason":"stop","index":0,"message":{"content":"In the Kyoto failover runbook, the Osaka queue must be drained before flipping the DNS.","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields":{"refusal":null,"search_results":{"object":"vector_store.search_results.page","search_query":"What must be drained before flipping DNS in the Kyoto failover runbook?","data":[{"score":0.7760404944419861,"content":[{"text":"The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl.","type":"text"}],"file_id":"s3-vectors-chunk-0","filename":"15937bc865d6498e99bb6fbee2b37d54.txt", ...
HTTP 200

RAG query naming the provider and region

1. custom_llm_provider: s3_vectors added, on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:51101/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","custom_llm_provider":"s3_vectors"}}' | cut -c1-400
{"id":"chatcmpl-EJkRxEtdYyjYAvjO8CaZEXjs7KZJt","created":1788375493,"model":"gpt-4o-mini-2024-07-18","object":"chat.completion","system_fingerprint":"fp_7ffe4e4620","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The Osaka queue must be drained before flipping DNS in the Kyoto failover runbook.","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields"
HTTP 200

2. aws_region_name: eu-central-1 pasted in too, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:54746/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","custom_llm_provider":"s3_vectors","aws_region_name":"eu-central-1"}}' | cut -c1-400
{"id":"chatcmpl-EJkRzV0DI0TrD15BwB7sfKiLznfKr","created":1788375495,"model":"gpt-4o-mini-2024-07-18","object":"chat.completion","system_fingerprint":"fp_7ffe4e4620","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Before flipping DNS in the Kyoto failover runbook, the Osaka queue must be drained.","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields
HTTP 200

RAG query with hostile connection overrides

1. custom_llm_provider: openai, api_base pointing at a dead port, and a bogus api_key in retrieval_config, on process 1: the real store answers, none of the overrides reach the search

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:51101/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","custom_llm_provider":"openai","api_base":"http://127.0.0.1:9/v1","api_key":"sk-attacker"}}' | cut -c1-400
{"id":"chatcmpl-EJkS2cJFfzR35WqQFXlIqVTicPjqJ","created":1788375498,"model":"gpt-4o-mini-2024-07-18","object":"chat.completion","system_fingerprint":"fp_7ffe4e4620","choices":[{"finish_reason":"stop","index":0,"message":{"content":"The Osaka queue must be drained before flipping DNS in the Kyoto failover runbook.","role":"assistant","tool_calls":null,"function_call":null,"provider_specific_fields"
HTTP 200

Caller-picked embedding keys

1. embedding_model in retrieval_config, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:54746/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","embedding_model":"text-embedding-3-large"}}'
{"detail":{"error":"'embedding_model' cannot be set in retrieval_config. Embedding configuration comes from the vector store's server-side registration."}}
HTTP 400

2. litellm_credential_name in retrieval_config, on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:51101/v1/rag/query -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What must be drained before flipping DNS in the Kyoto failover runbook?"}],"retrieval_config":{"vector_store_id":"lit6612-qa-vectors:litellm-index-5a1ed2ac","litellm_credential_name":"someone-elses-aws-creds"}}'
{"detail":{"error":"'litellm_credential_name' cannot be set in retrieval_config. Embedding configuration comes from the vector store's server-side registration."}}
HTTP 400

3. litellm_embedding_model in the direct search body, on process 2

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:54746/v1/vector_stores/lit6612-qa-vectors:litellm-index-5a1ed2ac/search -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"query":"Kyoto failover","litellm_embedding_model":"text-embedding-3-large"}'
{"detail":{"error":"'litellm_embedding_model' cannot be set in the search request body. Embedding configuration comes from the vector store's server-side registration."}}
HTTP 400

Direct vector store search

1. Plain search on process 1

$ curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:51101/v1/vector_stores/lit6612-qa-vectors:litellm-index-5a1ed2ac/search -H "Authorization: Bearer sk-lit6612" -H "Content-Type: application/json" -d '{"query":"What must be drained before flipping DNS?"}'
{"object":"vector_store.search_results.page","search_query":"What must be drained before flipping DNS?","data":[{"score":0.49791377782821655,"content":[{"text":"The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl.","type":"text"}],"file_id":"s3-vectors-chunk-0","filename":"15937bc865d6498e99bb6fbee2b37d54.txt","attributes":{"filename":"15937bc865d6498e99bb6fbee2b37d54.txt","source_text":"The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl.","chunk_index":"0"}}]}
HTTP 200

Admin UI Test Vector Store tab

1. Open the vector stores page, click lit6612-qa-vectors:litellm-index-5a1ed2ac, open the Test Vector Store tab, enter the query, click Search: the panel reads "1 results" with "Result 1 - The Kyoto failover runbook: drain the Osaka queue before flipping DNS, then page the on-call owl...." and "Score: 0.4979". Screenshot: lit6612-after-ui-working-store.png
2. Same steps on lit6612-qa-vectors:does-not-exist: the panel reads Search failed: {"error":{"message":"litellm.NotFoundError: S3_vectorsException - {\"message\":\"The specified index could not be found\"}","type":null,"param":null,"code":"404"}} instead of "No results found". Screenshot: lit6612-after-ui-broken-store.png

QA observations:

• Chat completions vector_store_ids gets no S3 context; pre-existing, left alone
• filters ignored by S3 Vectors search on both surfaces; pre-existing, left alone
• RAG query and direct search use different error envelopes; pre-existing, left alone
• AWS_BEARER_TOKEN_BEDROCK in the env breaks S3 Vectors SigV4; pre-existing, left alone
• Dashboard session token expires after ten minutes; pre-existing, left alone

Type

🐛 Bug Fix

Caveats (if any)

Medium

• litellm.rag.aquery now drops non-allowlisted retrieval_config keys (api_key, api_base, filters) before search
• Chat completions vector_store_ids still resolves S3 Vectors query embeddings without the Router
  • The same store that now works on /v1/rag/query gives no context there, tracked in LIT-6752

Low

• Nine provider search transforms accept a router parameter they ignore
• S3 Vectors search without a region now uses the env, then us-west-2, instead of erroring
• RAG query errors come back as {"detail": {"error": ...}}, direct search as {"error": {...}}

Changes

Router threading (defect 1): search-time query embeddings could not resolve Router-managed (virtual) embedding model names because the S3 Vectors transform called bare litellm.embedding():

• litellm/router.py: _init_vector_store_api_endpoints now receives call_type and injects router=self for avector_store_search only (assignment via kwargs also overrides any client-supplied router key). The sync wrapper matches: model-less sync vector_store_search/vector_store_create go straight to the SDK function (the fallback path requires a model and raised a TypeError), with the router injected for search only
• litellm/vector_stores/main.py: explicit router: Optional["Router"] param on search/asearch, kept out of **kwargs so it never enters GenericLiteLLMParams or logging params (the fix(rag): use router for completion in RAG query pipeline #19550 serialization trap, regression-tested)
• litellm/llms/custom_httpx/llm_http_handler.py + litellm/llms/base_llm/vector_store/transformation.py: router threaded through both search handlers into the search-transform contract
• 9 other provider transforms: signature-only router param (accepted, unused, no behavior change)
• litellm/llms/s3_vectors/vector_stores/transformation.py: resolves the query embedding via the Router iff it serves embedding_model (existing resolve_embedding_router helper, same policy as the semantic caches); bare-call fallback preserved for provider-prefixed models with env creds. Also: get_complete_url resolves the region via get_aws_region_name_for_non_llm_api_calls (dynamic, then AWS_REGION_NAME, then AWS_REGION, then us-west-2), matching the ingestion path instead of raising. Intended behavior change, ingest/query parity

RAG query pipeline (defect 2): /v1/rag/query failed for stores that work via direct /v1/vector_stores/{id}/search:

• litellm/proxy/rag_endpoints/endpoints.py: merges the managed vector store registry into retrieval_config, reusing the store already resolved during the access check (one registry lookup per request). Store-managed keys win on conflict, so a caller cannot override a shared store's provider or credentials from the request body. The debug log names the model and store id instead of dumping the merged config, which can carry resolved credentials
• litellm/proxy/vector_store_endpoints/endpoints.py: the registry merge used by the direct search endpoint is split into a reusable build_request_data_from_managed_vector_store helper (same behavior: embedding config resolved per request, never persisted back to the registry cache)
• litellm/rag/main.py: forwards only an explicit allowlist of retrieval_config keys (region, bucket, embedding model, credential refs) plus the router to vector_stores.asearch, keeping caller-supplied connection overrides like api_base/api_key away from the search call. filters/retrieval_filter stay reserved so this composes with open PR fix(rag): forward retrieval_filter from retrieval_config to vector store search #34427 in either merge order
• litellm/proxy/vector_store_endpoints/endpoints.py + litellm/proxy/rag_endpoints/endpoints.py: both proxy query surfaces (/v1/rag/query retrieval_config and the /v1/vector_stores/{id}/search body) reject caller-supplied embedding_model/litellm_embedding_model/litellm_embedding_config/litellm_credential_name with a 400. Search-time embeddings resolve through the Router with the proxy's credentials without a key-permission check, so letting a caller pick the model was an authorization bypass (review finding); these keys now only come from the store's server-side registration, matching the documented contract and the ingest-side credential blocklist

UI (defect 3):

• networking.tsx: vectorStoreSearchCall throws on non-OK responses instead of returning null (single caller, verified)
• VectorStoreTester.tsx: failed searches render "Search failed: " in history; "No results found" now means an actually-empty result

Tests: new regression tests across all three defects (all verified to fail pre-fix), backward-compat pins (no-router path, router-not-serving fallback, non-search call types, serialization guard, store-wins merge, connection-override allowlist, sync router injection), parametrized 400s for every blocked embedding-selection key on both query surfaces, un-skipped test_transform_search_request placeholder, updated region-fallback tests, UI error-state + empty-results tests:

tests/test_litellm/llms/s3_vectors/vector_stores/test_s3_vectors_transformation.py::TestS3VectorsVectorStoreConfig::test_atransform_search_uses_router_for_virtual_model
tests/test_litellm/rag/test_main.py::test_aquery_forwards_provider_retrieval_config_and_router_to_search
tests/test_litellm/rag/test_main.py::test_aquery_does_not_forward_connection_override_keys_to_search
tests/test_litellm/proxy/rag_endpoints/test_rag_endpoints.py::test_rag_query_merges_managed_store_params
tests/test_litellm/proxy/rag_endpoints/test_rag_endpoints.py::test_rag_query_store_params_win_over_user_retrieval_config
tests/test_litellm/proxy/rag_endpoints/test_rag_endpoints.py::test_rag_query_rejects_caller_embedding_selection_params
tests/test_litellm/proxy/vector_store_endpoints/test_vector_store_endpoints.py::test_vector_store_search_rejects_caller_embedding_selection_params
tests/test_litellm/test_router.py::test_avector_store_search_injects_router
tests/test_litellm/test_router.py::test_vector_store_search_injects_router

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches proxy RAG/search auth (embedding model selection, credential merge, allowlists) and Router injection on search paths; behavior changes are intentional but affect security-sensitive retrieval flows.

Overview
Fixes S3 Vectors and managed-store RAG/search by passing the LiteLLM Router through vector store search (SDK → HTTP handler → provider transforms) so query-time embeddings can use router-managed virtual embedding models; S3 Vectors resolves embeddings via the router when it serves the model and aligns AWS region resolution with ingestion.

/v1/rag/query now merges managed vector store registry params (same helper as direct search), with store values winning over caller retrieval_config. litellm.aquery forwards only an allowlisted set of retrieval keys plus router to search, blocking connection overrides like api_base/api_key. Proxy /v1/rag/query and /vector_stores/{id}/search return 400 if callers set embedding model/credential keys that must come from server-side store registration.

Dashboard vector store tester surfaces backend error text instead of treating failures as empty results (vectorStoreSearchCall throws on non-OK).

^{Reviewed by Cursor Bugbot for commit 8b0441a. Bugbot is set up for automated code reviews on this repo. Configure here.}

• 8b0441a passes /live-pr-risk

[greptile-apps]

Greptile Summary

The PR restores managed S3 Vector Store querying across SDK, proxy RAG, direct-search, and dashboard paths.

• Threads Router context through synchronous and asynchronous vector-store search so Router-managed embedding models resolve correctly.
• Reuses the authorized managed-store lookup in RAG queries and merges server-side store configuration with precedence over caller input.
• Restricts caller-controlled embedding-selection parameters and forwards only approved retrieval configuration.
• Surfaces backend vector-search errors in the dashboard instead of displaying them as empty results.
• Adds regression coverage for Router propagation, managed-store merging, parameter rejection, region fallback, and UI error handling.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the synchronous Router context loss and duplicate managed-store lookup reported previously are both corrected at the current head.

Important Files Changed

Filename	Overview
litellm/router.py	The synchronous and asynchronous model-less vector-store search paths now inject the Router, resolving the previously reported context loss.
litellm/vector_stores/main.py	Search APIs explicitly accept Router context and pass it through to the shared HTTP handler without adding it to generic request parameters.
litellm/llms/custom_httpx/llm_http_handler.py	Both search handlers propagate Router context into provider search transformations.
litellm/llms/s3_vectors/vector_stores/transformation.py	S3 Vectors uses the Router for configured virtual embedding models and aligns search-region fallback with ingestion.
litellm/proxy/rag_endpoints/endpoints.py	RAG queries reuse the authorization lookup and merge managed-store configuration before runtime dispatch.
litellm/proxy/vector_store_endpoints/endpoints.py	Managed-store request construction is reusable and both proxy query surfaces reject caller-selected embedding configuration.
litellm/rag/main.py	The RAG pipeline forwards Router context and an explicit allowlist of store-search configuration.
ui/litellm-dashboard/src/app/(dashboard)/vector-stores/_components/VectorStoreTester.tsx	Search failures are retained in test history and rendered distinctly from successful empty results.
ui/litellm-dashboard/src/components/networking.tsx	Non-successful vector-store search responses now raise their backend error message for the UI to display.

_{Reviews (4): Last reviewed commit: "fix(vector_stores): block caller-supplie..." | Re-trigger Greptile}

[greptile-apps]

Managed store lookup runs twice

_authorize_nested_vector_store_ids already resolves and authorizes this vector store before this helper resolves it and checks access again. This duplicates registry and cache work on every RAG query and repeats the database fallback on a cache miss, adding avoidable request latency and load.

Rule Used: What: Avoid creating new database requests or Rout... (source)

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 3 · PR risk: 0/10

[codecov]

Codecov Report

❌ Patch coverage is 95.08197% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/vector_store_endpoints/endpoints.py	91.66%	2 Missing ⚠️
litellm/router.py	88.88%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_s3_vectors_search (8b0441a) with litellm_internal_staging (8ce02c0)}

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8b5ae3d. Configure here.}

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8b0441a. Configure here.}

[mateo-berri]

LGTM. Thanks!