Downloading MNIST fails due to HTTP Error 503

[akihironitta]

🐛 Bug

http://yann.lecun.com/exdb/mnist/index.html often gets unavailable, and because of the issue, some of the tests fail in Bolts due to 503. This is a known issue and its patch will be included in torchvision==0.9.1.

See pytorch/vision#3549

To Reproduce

https://dev.azure.com/PytorchLightning/lightning%20Bolts/_build/results?buildId=7496

[akihironitta]

As far as I investigated, there are two known issues about MNIST downloading:

• 403 (Downloading MNIST dataset with torchvision gives HTTP Error 403 pytorch/vision#1938)
• 503 (Downloading MNIST dataset with torchvision gives HTTP Error 503 pytorch/vision#3549)

For 403 error, we can fix it by setting the user agent to avoid the cloudflare filter as done in PL: Lightning-AI/pytorch-lightning#6357

For 503 error, we need to use a different URL than http://yann.lecun.com/exdb/mnist/ because the website is very unstable and its patch is included in torchvision>=0.9.1.

[oke-aditya]

The patch is already in stable torchvision, it redirects to a new mirror.

Here is the new mirror from torchvision datasets

Here is the mirror link

https://ossci-datasets.s3.amazonaws.com/mnist/

I think the bug is fixed from torchvision side. The recommended fix is to use torchvision==0.9.1

[akihironitta]

@oke-aditya Yes, I'm aware of the mirror link. However, I don't think fixing the version of torchvision at 0.9.1 only for this issue would be a reasonable fix as it requires users to update their torchvision.

[oke-aditya]

Yep I don't think so that's the way to fix for a single use case / issue.

What possibly can be done is to copy paste dataset.mnist from torchvision, and just use it.

Maybe something like

pl_bolts.datasets.mnist instead from torchvision.