About pretrain time

[1787648106]

Thank you very much for your excellent work. May I ask, what is your experimental hardware configuration? For example, the model of the GPU; In addition, how long did you complete the pre training on imagenet1k?

[wwqq]

Hi @1787648106 , V100*8. Tiny:~1.5 days; Small:~1.8 days; Base:~2.2 days Large:~2.5days

[1787648106]

thx for your response ~

[1787648106]

Hi @1787648106 , V100*8. Tiny:~1.5 days; Small:~1.8 days; Base:~2.2 days Large:~2.5days

hello~I pretrained the model on ImageNet1k, but the training time kept about ~5 days. A800 * 8, base model version, amp on. Are there some accelerating skills used ? Thanks again

[1787648106]

Hi @1787648106 , V100*8. Tiny:~1.5 days; Small:~1.8 days; Base:~2.2 days Large:~2.5days

hello~I pretrained the model on ImageNet1k, but the training time kept about ~5 days. A800 * 8, base model version, amp on. Are there some accelerating skills used ? Thanks again

@wwqq hello, I'm so sorry to bother you again. About the accelerating in pretrain.

[wwqq]

@1787648106 Hello, try setting the batch size and learning rate to twice their original values.