[MAEB] Merge in main 20250503#2635

Merged

isaac-chung merged 438 commits intomaebfrom

maeb-merge-in-main-20250503

May 3, 2025

This pull request is big! We're only showing the most recent 250 commits

Commits on Mar 18, 2025

Update tasks table
github-actions[bot]
committed
1.36.26
github-actions
committed
Pass task name to all evaluators (#2389)
Samoed
authored

Commits on Mar 20, 2025

Commits on Mar 21, 2025

Minor style changes (#2396)

KennethEnevoldsen
and
isaac-chung
authored
Added new dataset and tasks - ClusTREC-covid , clustering of thematic covid related scientific papers (#2302)
katzurik
authored
Update tasks table
github-actions[bot]
committed

Commits on Mar 22, 2025

Commits on Mar 23, 2025

fix: Add option to remove benchmark from leaderboard (#2417)
KennethEnevoldsen
authored
1.36.31
github-actions
committed
fix: Add VDR Multilingual Dataset (#2408)

authored
Update tasks table
github-actions[bot]
committed
1.36.32
github-actions
committed

Commits on Mar 24, 2025

HOTFIX: pin setuptools (#2423)
Samoed
authored

Commits on Mar 27, 2025

Error while evaluating MIRACLRetrievalHardNegatives: 'trust_remote_code' (#2445)
KennethEnevoldsen
authored

Commits on Mar 28, 2025

Feat/searchmap preview (#2420)

Free-tek
and
Samoed
authored

Commits on Mar 29, 2025

Commits on Mar 31, 2025

ci: cache `~/.cache/huggingface` (#2464)
sam-hey
authored

Commits on Apr 3, 2025

Commits on Apr 4, 2025

Commits on Apr 5, 2025

Commits on Apr 6, 2025

Commits on Apr 8, 2025

fix: validate lang code in ModelMeta (#2499)
ayush1298
authored
Update pyproject.toml (#2522)
Samoed
authored
1.36.38
github-actions
committed
Fix leaderboard version (#2524)
Samoed
authored

Commits on Apr 9, 2025

Fix gte-multilingual-base embed_dim (#2526)
tolgayan
authored

Commits on Apr 12, 2025

Commits on Apr 13, 2025

Model conan (#2534)

lllsy12138
and
shyuli
authored

Commits on Apr 14, 2025

fix: Update mteb.get_tasks with an exclude_aggregate parameter to exclude aggregate tasks (#2536)
Sid-MB
authored
1.36.39
github-actions
committed

Commits on Apr 15, 2025

docs: Add MIEB citation in benchmarks (#2544)
isaac-chung
authored
Add 2 new Vietnamese Retrieval Datasets (#2393)
BaoLocPham
authored
Update tasks table
github-actions[bot]
committed
fix: CacheWrapper per task (#2467)

flogrammer
and
Florian Rottach
authored
1.36.40
github-actions
committed
misc: move MMTEB scripts and notebooks to separate repo (#2546)
isaac-chung
authored
fix: Update requirements in JinaWrapper (#2548)
AlexeyVatolin
authored
1.36.41
github-actions
committed
Docs: Add MIEB to README (#2550)
isaac-chung
authored
Add xlm_roberta_ua_distilled (#2547)

panalexeu
and
Samoed
authored

Commits on Apr 17, 2025

feat: UI Overhaul (#2549)

x-tabdeveloping
and
KennethEnevoldsen
authored
1.38.0
github-actions
committed

Commits on Apr 19, 2025

add USER2 (#2560)
Samoed
authored
Fix leaderboard entry for BuiltBench (#2563)

mehrzadshm
and
Mehrzad Shahin-Moghadam
authored

Commits on Apr 20, 2025

fix: jasper models embeddings having nan values (#2481)
yjoonjang
authored
1.38.1
github-actions
committed
fix frida datasets (#2565)
Samoed
authored
Add relle (#2564)

24September
and
Samoed
authored

Commits on Apr 21, 2025

Commits on Apr 22, 2025

Add ModelMeta for CodeSearch-ModernBERT-Crow-Plus (#2570)

Shun0212
and
Samoed
authored

Commits on Apr 23, 2025

Docs: Improve MIEB docs (#2569)
isaac-chung
authored

Commits on Apr 24, 2025

Commits on Apr 26, 2025

Remove the comments from ImageEncoder (#2579)
KennethEnevoldsen
authored

Commits on Apr 27, 2025

fix: Add Encodechka benchmark (#2561)
Samoed
authored
Update tasks table
github-actions[bot]
committed
1.38.2
github-actions
committed

Commits on Apr 28, 2025

Commits on Apr 30, 2025

Commits on May 1, 2025

Commits on May 2, 2025