Experimental settings different?

[ruiyuanlu]

Hi, why the training experimental settings are different from the original papers. E.g. for 《Simple baselines for human pose estimation and tracking》training 210 epochs with different learning rate and different lr scheduler rather 140 epochs. Can you provide baseline models exactly matched the original papers? thanks anyway.

[innerlee]

Hi, the provided configs together with checkpoints might offer higher accuracies. What's the use case of exact replication btw, considering that some of the original choices were not very good? If it was for the reported numbers, maybe just cite their paper is okay?

[innerlee]

@jin-s13 may explain the implementation differences other than configs?

[jin-s13]

Another consideration is that "we want to make most methods/models to share the same set of training/evaluation hyper-parameters (such as lr and training epoch)". This will help us to directly compare different methods/models in the same training/evaluation settings.

[jin-s13]

Different papers may have different training settings.
The following settings may be different (NOT sure if they are exactly the same as the original papers):

1. environment: 1) number of gpus 2) pytorch version ...
2. training hyper-parameters: 1) learning rate 2) batchsize 3) number of epoch 4) warm-up 5) learning rate decay policy ....
3. augmentation hyper-parameters: 1) rotation factor 2) scale factor 3) the probability of using some types of data augmentation.
4. some implementation details of the training pipelines (data augmentation functions) may also have some differences..