Linear evaluation of MoCo on CUB

[mprzewie]

Hi! Thank you for a great paper and for sharing the code!

I'm looking to reproduce your results on the MoCo model, especially for transferring it to the CUB dataset.

The command I'm running is:

CUDA_VISIBLE_DEVICES=0 python transfer_linear_eval.py \
    --pretrain-data imagenet100 \
    --ckpt $CHECKPOINT_PATH \
    --model resnet50 \
    --dataset cub200 \
    --datadir $CUB_DIR \
    --metric top1

However, the model achieves accuracy lower than the one reported in the paper (37.0, as reported in Tab. 3):

For MoCo baseline (checkpoint shared by you) I got test acc=0.2575
and for MoCo_augself (checkpoint shared by you) I got test acc=0.3224
For MoCo pretrained by myself, I got test acc=0.3309

Thus, I think the issue may be in linear evaluation.

Oddly enough, I roughly reproduced your results on the CIFAR-10 and CIFAR-100 dataset (with ~0.5% difference), so maybe the issue is with CUB only.

Could you kindly provide some guidance on whether I got the hyperparameters right? Alternatively, how did you set up the CUB files - did you use the default train / test / split?

Best regards!