How to add DDP with val loader?

[cswwp]

How to add DDP with val evaluation? Is it same with train? @yangkky

[yangkky]

You'd have to find a way to compile the validation statistics onto one process.

…

On Fri, Oct 23, 2020 at 6:07 AM cswwp ***@***.***> wrote: How to add DDP with val evaluation? Is it same with train? @yangkky <https://github.com/yangkky> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADEMNWF465QKJEIFUHTKMDTSMFIVVANCNFSM4S4LGKEQ> .

[rancheng]

This code is pretty clear:
https://github.com/rwightman/pytorch-image-models/blob/master/train.py#L730-L732

            reduced_loss = reduce_tensor(loss.data, args.world_size)
            acc1 = reduce_tensor(acc1, args.world_size)
            acc5 = reduce_tensor(acc5, args.world_size)