Skip to content

Comments

add results of Querit/Querit#407

Merged
Samoed merged 4 commits intoembeddings-benchmark:mainfrom
youngbeauty250:Querit
Jan 29, 2026
Merged

add results of Querit/Querit#407
Samoed merged 4 commits intoembeddings-benchmark:mainfrom
youngbeauty250:Querit

Conversation

@youngbeauty250
Copy link
Contributor

@youngbeauty250 youngbeauty250 commented Jan 25, 2026

Checklist

  • My model has a model sheet, report, or similar
  • My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
  • The results submitted are obtained using the reference implementation
  • My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
  • I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

@github-actions
Copy link

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: Querit/Querit
Tasks: AlloprofReranking, AskUbuntuDupQuestions, MindSmallReranking, RuBQReranking, T2Reranking, VoyageMMarcoReranking, WebLINXCandidatesReranking, WikipediaRerankingMultilingual

Results for Querit/Querit

task_name Querit/Querit google/gemini-embedding-001 intfloat/multilingual-e5-large Max result Model with max result In Training Data
AlloprofReranking 0.7756 0.8177 0.6944 0.8540 bflhc/Octen-Embedding-8B False
AskUbuntuDupQuestions 0.6243 0.6424 0.5924 0.7528 IEITYuan/Yuan-embedding-2.0-en False
MindSmallReranking 0.3181 0.3295 0.3024 0.3437 Kingsoft-LLM/QZhou-Embedding False
RuBQReranking 0.703 0.7384 0.756 0.8051 ai-sage/Giga-Embeddings-instruct False
T2Reranking 0.6805 0.6795 0.6632 0.7315 tencent/Youtu-Embedding False
VoyageMMarcoReranking 0.6054 0.6673 0.6821 0.7351 jinaai/jina-reranker-v3 False
WebLINXCandidatesReranking 0.1193 0.1097 0.0778 0.1792 Bytedance/Seed1.6-embedding-1215 False
WikipediaRerankingMultilingual 0.8706 0.9224 0.8981 0.9308 jinaai/jina-reranker-v3 False
Average 0.5871 0.6134 0.5833 0.6665 nan -

@Samoed Samoed merged commit 8bb7eaa into embeddings-benchmark:main Jan 29, 2026
2 of 3 checks passed
@youngbeauty250 youngbeauty250 deleted the Querit branch January 29, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants