[Feat] Add new RAG API on LiteLLM AI Gateway

[ishaan-jaff]

[Feat] Add new RAG API on LiteLLM AI Gateway

This PR adds a new /v1/rag/ingest API endpoint for all-in-one document ingestion into vector stores. The pipeline handles: Upload -> Chunk -> Embed -> Vector Store.

The implementation uses a provider-specific class architecture where each vector store provider (OpenAI, Bedrock) can override ingestion behavior. For example, OpenAI handles embedding internally when files are attached to vector stores, while other providers may require explicit embedding calls.

Usage

curl -X POST "http://localhost:4000/v1/rag/ingest" \
    -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d "{
        \"file\": {
            \"filename\": \"test_document.txt\",
            \"content\": \"$(base64 -i test_document.txt)\",
            \"content_type\": \"text/plain\"
        },
        \"ingest_options\": {
            \"name\": \"test-basic-ingest\",
            \"vector_store\": {
                \"custom_llm_provider\": \"openai\"
            }
        }
    }

Response

{
  "id": "ingest_abc123",
  "status": "completed",
  "vector_store_id": "vs_xyz789",
  "file_id": "file_123"
}

Query the Ingested Data

curl -X POST "http://localhost:4000/v1/vector_stores/vs_xyz789/search" \
    -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"query": "What is the main topic?"}'## Architecture

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Nov 26, 2025 1:55am