Releases: ImahnShekhzadeh/lstm_vision
Releases · ImahnShekhzadeh/lstm_vision
1.4.0
- Fix DDP (on 2 RTXA4000 Ada, runtime compared to a single RTX 4090 is decreased by at least
$25 %$ . - Use
hydra
both as orchestration tool and for setting up the logs. - Remove setup of manual logging, now that
hydra
is used. - Simplify calculation of losses, use as loss summed categorical cross entropy and then take mean for backprop.
1.3.3
Add logging of energy consumption of entire training.
Fix Worker Seeds in DataLoader
- Fix seeds in data loaders (https://pytorch.org/docs/stable/notes/randomness.html#dataloader) for reproducibility.
v1.3.1
Use of wandb
for logging now possible
1.3.0
New feature: Training via DDP possible now. If flag --use_ddp
is specified and there are at least two GPUs, then DDP will automatically be used.
torch.compile()
- Highlight: Optimized (compiled) LSTM can be used by using the flag
--compile_mode
and specifying a compilation mode. - Other changes include refactoring of the entire code base (e.g. by moving the code for training and validation into a separate function).
- It is also way easier now to use another CV dataset than MNIST, check the README for this!
Half-precision
Using half-precision (i.e. torch.float16
instead of torch.float32
) greatly reduces the memory consumption, while the necessary code modifications are minor!
LSTM on MNIST
This code contains the code to train a bidirectional LSTM on the MNIST data. With some simple modifications, any dataset can be taken.