Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 2 new Static Sentence Transformer models #2112

Merged
merged 2 commits into from
Feb 21, 2025

Conversation

tomaarsen
Copy link
Member

Fixes #2109

This adds ModelMeta's for:

I'm running evaluations for these for https://github.com/embeddings-benchmark/results now.

Code Quality

  • Code Formatted: Format the code using make lint to maintain consistent style.

Documentation

  • Updated Documentation: Add or update documentation to reflect the changes introduced in this PR.

Testing

  • New Tests Added: Write tests to cover new functionality. Validate with make test-with-coverage.
  • Tests Passed: Run tests locally using make test or make test-with-coverage to ensure no existing functionality is broken.

Adding a model checklist

  • I have filled out the ModelMeta object to the extent possible
  • I have ensured that my model can be loaded using
    • mteb.get_model(model_name, revision) and
    • mteb.get_model_meta(model_name, revision)
  • I have tested the implementation works on a representative set of tasks.

  • Tom Aarsen

@tomaarsen tomaarsen added the new model Questions related to adding a new model to the benchmark label Feb 20, 2025
Copy link
Collaborator

@Samoed Samoed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

Co-authored-by: Roman Solomatin <[email protected]>
@Samoed
Copy link
Collaborator

Samoed commented Feb 20, 2025

Is this ready to be merged? Just wanted to clarify.

@tomaarsen
Copy link
Member Author

Yes! This is good to go. I've just published the companion PR on the results repo as well: embeddings-benchmark/results#128

  • Tom Aarsen

@KennethEnevoldsen KennethEnevoldsen merged commit e7735b2 into main Feb 21, 2025
9 checks passed
@KennethEnevoldsen KennethEnevoldsen deleted the new_models/sentence-transformers-static branch February 21, 2025 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new model Questions related to adding a new model to the benchmark
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add sentence-transformers/static-similarity-mrl-multilingual-v1
3 participants