Skip to content

add results of ListConRanker#213

Merged
Samoed merged 4 commits intoembeddings-benchmark:mainfrom
tutuDoki:add_listconranker
Jun 30, 2025
Merged

add results of ListConRanker#213
Samoed merged 4 commits intoembeddings-benchmark:mainfrom
tutuDoki:add_listconranker

Conversation

@tutuDoki
Copy link
Contributor

@tutuDoki tutuDoki commented Jun 4, 2025

Checklist

  • My model has a model sheet, report or similar
  • My model has a reference implementation in mteb/models/ this can be as an API. Instruction on how to add a model can be found here
  • The results submitted is obtained using the reference implementation
  • My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
  • I solemnly swear that for all results submitted I have not on the evaluation dataset including training splits. If I have I have disclosed it clearly.

@KennethEnevoldsen
Copy link
Contributor

@Samoed is reviewing the implementation. So will let him review this one as well. Feel free to @ me if needed.

@KennethEnevoldsen KennethEnevoldsen added the waiting for review of implementation This PR is waiting for an implementation review before merging the results. label Jun 16, 2025
@github-actions
Copy link

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: ByteDance/ListConRanker
Tasks: CMedQAv1-reranking, CMedQAv2-reranking, MMarcoReranking, T2Reranking

Results for ByteDance/ListConRanker

task_name ByteDance/ListConRanker google/gemini-embedding-001 intfloat/multilingual-e5-large Max result
CMedQAv1-reranking 0.91 nan 0.68 0.01
CMedQAv2-reranking 0.89 nan 0.67 0.01
MMarcoReranking 0.44 nan 0.29 0
T2Reranking 0.69 0.68 0.66 0.01
Average 0.73 0.68 0.57 0.01

@tutuDoki
Copy link
Contributor Author

@KennethEnevoldsen @Samoed Hi, since the implementation of ListConRanker has been merged, could you please review this PR? Thanks!

@Samoed Samoed merged commit 90fa256 into embeddings-benchmark:main Jun 30, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting for review of implementation This PR is waiting for an implementation review before merging the results.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants