Half-precision
Using half-precision (i.e. torch.float16 instead of torch.float32) greatly reduces the memory consumption, while the necessary code modifications are minor!
Using half-precision (i.e. torch.float16 instead of torch.float32) greatly reduces the memory consumption, while the necessary code modifications are minor!