Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion mteb/leaderboard/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ def update_description(
n_tasks = len(benchmark.tasks)
n_domains = len(domains)
description += f" - **Number of languages**: {n_languages}\n"
description += f" - **Number of datasets**: {n_tasks}\n"
description += f" - **Number of tasks**: {n_tasks}\n"
description += f" - **Number of task types**: {n_task_types}\n"
description += f" - **Number of domains**: {n_domains}\n"
if str(benchmark.reference) != "None":
Expand Down Expand Up @@ -428,6 +428,18 @@ def filter_models(
Based on community feedback and research findings, This definition could change in the future.
"""
)
with gr.Accordion(
"What does the other columns mean?",
open=False,
):
gr.Markdown(
"""
- **Number of Parameters**: This is the total number of parameters in the model including embedding parameters. A higher value means the model requires more CPU/GPU memory to run; thus, less is generally desirable.
- **Embedding Dimension**: This is the vector dimension of the embeddings that the model produces. When saving embeddings to disk, a higher dimension will require more space, thus less is usually desirable.
- **Max tokens**: This refers to how many tokens (=word pieces) the model can process. Generally, a larger value is desirable.
- **Zero-shot**: This indicates if the model is zero-shot on the benchmark. For more information on zero-shot see the info-box below.
"""
)
with gr.Accordion(
"Why is a model missing or not showing up?",
open=False,
Expand Down