Skip to content

embeddings-benchmark/leaderboard

Repository files navigation

title emoji colorFrom colorTo sdk sdk_version app_file pinned tags startup_duration_timeout fullWidth
MTEB Leaderboard
🥇
blue
indigo
gradio
4.20.0
app.py
false
leaderboard
1h
true

The MTEB Leaderboard repository

This repository contains the code for pushing and updating the MTEB leaderboard daily.

Relevant Links Decription
mteb The implementation of the benchmark. Here you e.g. find the code to run your model on the benchmark.
leaderboard The leaderboard itself, here you can view results of model run on MTEB.
results The results of MTEB is stored here. Though you can publish them to the leaderboard adding the result to your model card.

Developer setup

To setup the repository:

git clone https://github.com/embeddings-benchmark/leaderboard.git
cd leaderboard
# install requirements
pip install -r requirements.txt
# fetch new results
# python refresh.py
# if you'd like to add results to previously cached models, you may have to remove these models in `EXTERNAL_MODEL_RESULTS.json`
# you can also directly delete `EXTERNAL_MODEL_RESULTS.json` and it will recreate it (but be much slower)
# run the leaderboard
python app.py

Releases

No releases published

Packages

No packages published

Languages