[MAEB] merge main -> maeb#2471

Merged

isaac-chung merged 236 commits intomaebfrom

isaac/merge-maeb-main

Apr 1, 2025

Commits on Feb 13, 2025

Commits on Feb 14, 2025

Commits on Feb 15, 2025

Commits on Feb 16, 2025

misc: update model names to adjust for adding to results repo (#2074)
isaac-chung
authored

Commits on Feb 17, 2025

misc: Add all image classification descriptive stats (#2073)

isaac-chung
and
Samoed
authored
Update tasks table
github-actions[bot]
committed
ci: Rerun tests that fail due to networking issues. (#2029)
sam-hey
authored
fix: generate metadata (#2063)

authored
1.34.15
github-actions
committed
fix: add missing `e5` training datasets (#2065)
Samoed
authored
1.34.16
github-actions
committed
fix: Ensure voyage model uses different naming scheme (#2083)
KennethEnevoldsen
authored
1.34.17
github-actions
committed
fix: Freeze model/rank columns in leaderboard (#2044)

shikhar1729
and
Shikhar Shiromani
authored
1.34.18
github-actions
committed
fix: Fixed previous incorrect specification of splits for CMTEB ( MTEB(cmn, v1) ) (#2086)
KennethEnevoldsen
authored
1.34.19
github-actions
committed
Remove duplicated string in docstring of TaskMetadata class (#2087)
dantetemplar
authored
fix: Smarter leaderboard caching with cachetools (#2085)

x-tabdeveloping
and
KennethEnevoldsen
authored
fix: Missing fixes for #2086 - change MultilingualSentiment split from test to validation in CMTEB (#2088)
KennethEnevoldsen
authored
1.34.20
github-actions
committed
merge gme models (#2089)
Samoed
authored

Commits on Feb 19, 2025

Commits on Feb 20, 2025

Add model inf-retriever-v1-1.5b (#2106)
SamuelYang1
authored
docs: Fix typos & refine text (#2102)
Muennighoff
authored
misc: Run Zeroshot Classification Descriptive Stats (#2105)
isaac-chung
authored
Update tasks table
github-actions[bot]
committed
fix: add warning about task category conversion (#2108)
isaac-chung
authored
1.34.23
github-actions
committed
fix: Add codesage-large-v2 (#2090)
Aradhye2002
authored
1.34.24
github-actions
committed
fix: add training data to BGE-m3-custom-fr (#2110)
KennethEnevoldsen
authored
1.34.25
github-actions
committed
fix: Upgrade ruff to be gradio compatible (#2111)
KennethEnevoldsen
authored
1.34.26
github-actions
committed
docs: Follow google docstring format (#2115)
KennethEnevoldsen
authored

Commits on Feb 21, 2025

Update leaderboard_refresh.yaml (#2121)
Samoed
authored
fix InstructSentenceTransformer Model name (#2125)
Samoed
authored
fix voyage (#2127)
Samoed
authored
fix: update e5 instruct training data (#2129)
Samoed
authored
1.34.27
github-actions
committed
format
KennethEnevoldsen
committed
Update tasks table
github-actions[bot]
committed
fix: Add 2 new Static Sentence Transformer models (#2112)

tomaarsen
and
Samoed
authored
1.34.28
github-actions
committed
add is_cross_encoder (#1869)

Samoed
and
KennethEnevoldsen
authored

Commits on Feb 23, 2025

Qodo embed 1 1.5 b (#2137)

authored

Commits on Feb 24, 2025

misc: merge summary retrieval into bitext mining (#2140)
isaac-chung
authored
test: fix dataset availability test (#2141)
KennethEnevoldsen
authored
fix: Update NVIDIA-Embed training data (#2143)
KennethEnevoldsen
authored
1.34.29
github-actions
committed
fix: Add annotations for Voyage exp (#2144)
KennethEnevoldsen
authored
1.34.30
github-actions
committed
Fix tokens num in cde models (#2148)
Samoed
authored
feat: Add Qodo-Embed-1-7B model metadata and rename existing model (#2146)

talshef
and
Tal Sheffer
authored
1.35.0
github-actions
committed
misc: add Any2AnyRetrievalDescriptiveStatistics (#2139)
isaac-chung
authored
Update tasks table
github-actions[bot]
committed

Commits on Feb 25, 2025

Commits on Feb 26, 2025

Update FaMTEBRetrieval.py (#2171)
garciasces
authored
Update tasks table
github-actions[bot]
committed
fix: Add Training data annotations (#2173)
KennethEnevoldsen
authored
1.35.2
github-actions
committed

Commits on Feb 27, 2025

Commits on Feb 28, 2025

Commits on Mar 2, 2025

Commits on Mar 3, 2025

Commits on Mar 4, 2025

Commits on Mar 5, 2025

Remove overlapping legends from radar chart (#2195)
ayush1298
authored

Commits on Mar 7, 2025

Commits on Mar 8, 2025

fix: Run remaining MIEB desc stats (#2288)

isaac-chung
and
Isaac Chung
authored
Update tasks table
github-actions[bot]
committed
1.36.11
github-actions
committed

Commits on Mar 9, 2025

fix: Added Filter Modality (#2262)

authored
1.36.12
github-actions
committed
fix: Add `ModelMeta` license & custom validations (#2293)

Samoed
and
isaac-chung
authored
1.36.13
github-actions
committed

Commits on Mar 10, 2025

ci: Add pre-commit hook (#2194)
sam-hey
authored
Update tasks table
github-actions[bot]
committed
fix: bug in voyage implementation (#2304)

KennethEnevoldsen
and
Samoed
authored
1.36.14
github-actions
committed

Commits on Mar 11, 2025

fix: Update voyage name to include Org. (#2322)
KennethEnevoldsen
authored
1.36.15
github-actions
committed
Added VDR Model (#2290)

ayush1298
and
Samoed
authored
fix: Resolve conflicting dependencies (#2323)
KennethEnevoldsen
authored
1.36.16
github-actions
committed
fix: remove SyntaxWarnings in py312 (#2325)
KennethEnevoldsen
authored
1.36.17
github-actions
committed
fix: add annotation models for stella zh (#2277)

KennethEnevoldsen
and
isaac-chung
authored
1.36.18
github-actions
committed
fix: Add ModelMeta rubert-mini-frida, BERTA (#2330)
sergeyz-zh
authored
docs: fix typos
Muennighoff
authored
1.36.19
github-actions
committed

Commits on Mar 12, 2025

fix: Add WebFAQ bitext mining tasks (#2326)
michaeldinzinger
authored
Update tasks table
github-actions[bot]
committed
1.36.20
github-actions
committed

Commits on Mar 13, 2025

Commits on Mar 17, 2025

Correct -1 to No information in Zero shot (#2381)
ayush1298
authored
fix leaderboard (#2385)
KennethEnevoldsen
authored
fix: Reduce logging and Warnings (#2349)

ayush1298
and
Samoed
authored
1.36.23
github-actions
committed
fix: b1ade (#2386)

KennethEnevoldsen
and
w601sxs
authored
1.36.24
github-actions
committed
fix: pin gradio dependency to ensure leaderboards works (#2387)
KennethEnevoldsen
authored
1.36.25
github-actions
committed

Commits on Mar 18, 2025

Commits on Mar 20, 2025

Commits on Mar 21, 2025

Commits on Mar 22, 2025

Commits on Mar 23, 2025

fix: Add option to remove benchmark from leaderboard (#2417)
KennethEnevoldsen
authored
1.36.31
github-actions
committed
fix: Add VDR Multilingual Dataset (#2408)

authored
Update tasks table
github-actions[bot]
committed
1.36.32
github-actions
committed

Commits on Mar 24, 2025

HOTFIX: pin setuptools (#2423)
Samoed
authored

Commits on Mar 27, 2025

Error while evaluating MIRACLRetrievalHardNegatives: 'trust_remote_code' (#2445)
KennethEnevoldsen
authored

Commits on Mar 28, 2025

Feat/searchmap preview (#2420)

Free-tek
and
Samoed
authored

Commits on Mar 29, 2025

Commits on Mar 31, 2025

ci: cache `~/.cache/huggingface` (#2464)
sam-hey
authored