-
Notifications
You must be signed in to change notification settings - Fork 125
added w601sxs__b1ade-embed model results and removed KD versions #154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We will need a revision as well as an updated model meta
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this just the output of ModelMeta.dict() ?
{'name': 'b1ade-embed',
'revision': '3bdac13927fdc888b903db93b2ffdbd90b295a69',
'release_date': '2025-03-10',
'languages': ['eng-Latn'],
'loader': None,
'n_parameters': 335000000,
'memory_usage_mb': 1278.0,
'max_tokens': 4096.0,
'embed_dim': 1024,
'license': 'mit',
'open_weights': True,
'public_training_code': None,
'public_training_data': None,
'framework': ['Sentence Transformers', 'PyTorch'],
'reference': 'https://huggingface.co/w601sxs/b1ade-embed',
'similarity_fn_name': 'cosine',
'use_instructions': False,
'training_datasets': {'MSMARCO': []},
'adapted_from': None,
'superseded_by': None,
'is_cross_encoder': None,
'modalities': ['text']}
would that do? In that case I will use this revision id
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, added based on another folder
|
I see |
|
You should put your results in the folder with revision of your model. |
|
^ had to remove .ipynb_checkpoint, but looks like checks passed now. Sorry for the sequence of commits |
|
Does the leaderboard refresh every week/ every day? Wondering why the model does not show up on the leaderboard @KennethEnevoldsen 🔢 |
|
I think it was added in embeddings-benchmark/mteb#2386 with incorrect name (witout user/org prefix). I'll fix that |
|
Now it's showing correctly |
Checklist
make test.make pre-push.Adding a model checklist
mteb/models/directory. Instruction to add a model can be found here in the following PR