Training Time

[mapengsen]

Dear Authors,

when i use one gpu to train the model :
python -m torch.distributed.launch --nproc_per_node=1 scripts/image_train.py

and use 4 gpu to train the model:
python -m torch.distributed.launch --nproc_per_node=4 scripts/image_train.py

The number of training steps is the same under the same training time (the final modelXXX.pt obtained by 4 GPUs will not be more/faster than that obtained by 1 GPU). Is this normal?

[mapengsen]

if i need set "--world_size 4"?