-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Some questions about the checkpoint #23
Comments
Thanks for your interest in our work! |
Thank you for your explanation. I did use the same script. I got MRR@10 0.393 when I directly used RetroMAE_MSMARCO_finetune model for testing, but I can't get the same number when I use the checkpoint. |
Hi, I run the same command with one GPU (set num_train_epochs=1 for efficiency), and get MRR@10=0.383. Larger num_train_epochs may further improve the result. |
Hello,
I tried to use your checkpoint to finetune the RetroMAE_MSMARCO model, but the result is lower than the number in your paper(e.g. the MRR@10 is 0.393 in the paper, but I got 0.373). I used the hard_negs in the script, and used the same hyper-parameters. The only difference is that I use one V100 to finetune.
The text was updated successfully, but these errors were encountered: