This API framework first transforms your data in 1536D vectors (as RAGs do), then employs HNSW indexing for efficient information retrieval (again, as RAGs do) using pgvector. In short - it enhances your database with search capabilities before plugging it into further queries. Check it out on https://tuskvector.com
TuskVector is built with:
- Python for the backend (no surprises there)
- pgvector for Postgre DB vector functionality (elephants and vectors, get it?)
- HNSW for fast approximate nearest neighbor search
- OpenAI's ada2 for text embeddings
- GPT 4o for LLM queries
- FastAPI for building APIs
- HTMX as frontend to dodge JavaScript (because apparently, that's a thing now)
It's packaged and uploaded to PyPI! - check out https://pypi.org/project/tuskvector/ and/or use
pip install tuskvector
API Endpoints - also found on https://tuskvector.com/docs
-
Vector Embedding (POST
/api/embed_text
)- Utilizes OpenAI's text-embedding-ada-002 model
- Generates 1536-dimensional embeddings
- Automatically stores embeddings in pgvector-enabled PostgreSQL database
-
Similarity Search (POST
/api/similarity_search
)- Implements cosine similarity metric
- Utilizes HNSW (Hierarchical Navigable Small World) index for approximate nearest neighbor search
- Configurable search parameters:
ef_search
: Controls the trade-off between search speed and accuracy- Distance threshold: Filters results based on maximum allowed cosine distance
-
Context-Aware LLM Queries (POST
/api/query
)- Integrates with OpenAI's GPT models
- Enhances LLM responses with relevant context from the vector database
- Implements a two-stage retrieval process:
- Vector similarity search to find relevant facts
- LLM query augmented with retrieved context
HNSW_M
: Maximum number of connections per layer in HNSW index (we went with 16)HNSW_EF_CONSTRUCTION
: Size of the dynamic candidate list for constructing the HNSW graph (we went with 64)MAX_DISTANCE
: Cosine distance threshold for similarity search (we went with 0.1)