Skip to content

Latest commit

 

History

History
89 lines (59 loc) · 4.48 KB

README.md

File metadata and controls

89 lines (59 loc) · 4.48 KB
Raw

denser logo Denser Retriever

Python Version Dependencies Status

Code style: ruff Security: bandit Pre-commit Semantic Versions License Coverage Report

An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.

📝 Description

Denser Retriever combines multiple search technologies into a single platform. It utilizes gradient boosting ( xgboost) machine learning technique to combine:

  • Keyword-based searches that focus on fetching precisely what the query mentions.
  • Vector databases that are great for finding a wide range of potentially relevant answers.
  • Machine Learning rerankers that fine-tune the results to ensure the most relevant answers top the list.
  • Our experiments on MTEB datasets show that the combination of keyword search, vector search and a reranker via a xgboost model (denoted as ES+VS+RR_n) can significantly improve the vector search (VS) baseline.

mteb_ndcg_plot

  • Check out Denser Retriever experiments using the Anthropic Contextual Retrieval dataset at here.

🚀 Features

The initial release of Denser Retriever provides the following features.

  • Supporting heterogeneous retrievers such as keyword search, vector search, and ML model reranking
  • Leveraging xgboost ML technique to effectively combine heterogeneous retrievers
  • State-of-the-art accuracy on MTEB Retrieval benchmarking
  • Demonstrating how to use Denser retriever to power an end-to-end applications such as chatbot and semantic search

📦 Installation

We recommend installing Python via Anaconda, as we have received feedback about issues with Numpy installation when using the installer from https://www.python.org/downloads/. We are working on providing a solution to this problem. To install Denser Retriever, you can run:

Pip

pip install git+https://github.com/denser-org/denser-retriever.git#main

Poetry

poetry add git+https://github.com/denser-org/denser-retriever.git#main

📃 Documentation

The official documentation is hosted on retriever.denser.ai. Click here to get started.

👨🏼‍💻 Development

You can start developing Denser Retriever on your local machine.

See DEVELOPMENT.md for more details.

🛡 License

License

This project is licensed under the terms of the MIT license. See LICENSE for more details.

📃 Citation

@misc{denser-retriever,
  author = {denser-org},
  title = {An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/denser-org/denser-retriever}}
}