multi-gpu inference during training

[AljoSt]

❓ Questions and Help

What is your question?

During training, I need to run all the data through my model from time to time. Currently, I do this during the on_batch_end hook. But if I just call the model's forward function, it will only use one GPU. I tried to wrap the model into a nn.Dataparallel before inferencing, but that doesn't seem to work. Any idea what I can do?

[rmrao]

I think you can get the wrapped dataparallel model with self.trainer.model (which, if you're using the dataparallel backend is not the same as self) and run self.trainer.model(batch). Slightly better I think is self.trainer.model(self.trainer.transfer_batch_to_gpu(copy.copy(batch), 0)). The relevant section is in the training_loop.py file.

[Ir1d]

+1 here, in testing it can only work with one gpu. I tried wrapping with DataParallel and LightningDataParallel but I get things like this:

AttributeError: 'DataParallel' object has no attribute 'configure_optimizers'

Any ideas @williamFalcon

[williamFalcon]

.test() uses whatever you set up (dp, ddp, etc). no need to do anything yourself.

hard to tell what you need without code

[Ir1d]

Something is going wrong in my case, I've opened a new issue to track what's going on