Skip to content

Conversation

@Shun0212
Copy link
Contributor

Model Submission

Model Name: CodeSearch-ModernBERT-Crow-Plus
Model Author: Shuu12121
GitHub Repository (Results): https://github.com/Shun0212/results
Hugging Face Model Page: https://huggingface.co/Shuu12121/CodeSearch-ModernBERT-Crow-Plus

This PR adds evaluation results for the CodeSearch-ModernBERT-Crow-Plus model to the MTEB benchmark.

  • All required metadata and evaluation result files are located under results/CodeSearch-ModernBERT-Crow-Plus/
  • The git_commit_sha corresponds to the version of the model available on Hugging Face
  • All tests were passed successfully using make test (5285 passed, 827 skipped)

Please let me know if any changes or additions are required. Thank you for your time and consideration!

Adds evaluation results and metadata for the CodeSearch-ModernBERT-Crow-Plus model. Evaluated on CodeSearchNetRetrieval and COIRCodeSearchNetRetrieval.
@Samoed
Copy link
Member

Samoed commented Apr 21, 2025

You should add implementation of your model to main mteb repo

@Shun0212 Shun0212 closed this Apr 21, 2025
@Shun0212 Shun0212 deleted the codemodern-crow branch April 21, 2025 20:05
@Shun0212 Shun0212 restored the codemodern-crow branch April 22, 2025 13:23
@Shun0212 Shun0212 reopened this Apr 22, 2025
@Shun0212
Copy link
Contributor Author

Thanks again for your earlier feedback!

As requested, I have already added the implementation of the model to the main mteb repository via PR #2570, which has been successfully merged. Therefore, the model Shuu12121/CodeSearch-ModernBERT-Crow-Plus is now fully integrated into the MTEB framework.

This current PR (#185) adds the corresponding evaluation results for that model. All tests have passed (make test) and the file size checks succeeded. If everything looks good, I would greatly appreciate it if you could review and approve this PR.

Let me know if anything else is needed—I’m happy to make any changes.
Thank you for your time and support!

Copy link
Member

@Samoed Samoed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@Samoed Samoed merged commit 6022ac7 into embeddings-benchmark:main Apr 22, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants