-
Notifications
You must be signed in to change notification settings - Fork 7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
datasets: Fallback to our own mirrors for mnist #3544
Conversation
We are experiencing 403s when trying to download from the main mnist site so lets fallback to our own mirror on failure. Signed-off-by: Eli Uriegas <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor nitpick below. Still, could you give an example where you are experiencing an 403
error? Our download test pass for MNIST
(EMNIST
fails but with an unrelated issue) and running
from torchvision import datasets
datasets.MNIST(".", download=True)
also works for me locally.
finally: | ||
print() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These seem like debug statements, no?
finally: | |
print() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this might be here to avoid the download bar stopping at the middle of the screen for some reason. I'd be happy to remove it in a follow-up commit if this is unnecessary, but I'll be moving forward with merging this as this is blocking a few other things
@Zethson The requests fail with an |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @seemethere !
Let's get this merged now to unblock, and maybe revisit @pmeier comments in a follow-up PR
finally: | ||
print() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this might be here to avoid the download bar stopping at the middle of the screen for some reason. I'd be happy to remove it in a follow-up commit if this is unnecessary, but I'll be moving forward with merging this as this is blocking a few other things
This regressed the download of |
@pmeier let's fix the MNIST variants. We got permission to mirror if the download to the original server fails, so we shouldn't just remove the original links |
We are experiencing 403s when trying to download from the main mnist site so lets fallback to our own mirror on failure. Signed-off-by: Eli Uriegas <[email protected]> Co-authored-by: Francisco Massa <[email protected]>
* datasets: Fallback to our own mirrors for mnist (#3544) We are experiencing 403s when trying to download from the main mnist site so lets fallback to our own mirror on failure. Signed-off-by: Eli Uriegas <[email protected]> Co-authored-by: Francisco Massa <[email protected]> * Fix (Fashion|K)MNIST download and MNIST download test (#3557) * add mirrors to (Fashion|K)MNIST * fix download tests for MNIST Co-authored-by: Eli Uriegas <[email protected]> Co-authored-by: Francisco Massa <[email protected]>
Summary: We are experiencing 403s when trying to download from the main mnist site so lets fallback to our own mirror on failure. Signed-off-by: Eli Uriegas <[email protected]> Reviewed By: fmassa Differential Revision: D27127998 fbshipit-source-id: 552a022845eae39e9a7cc3255bf8b6f8eb2c07e7 Co-authored-by: Francisco Massa <[email protected]>
We are experiencing 403s when trying to download from the main mnist
site so lets fallback to our own mirror on failure.
Re-do of #1940
We have now gotten permission to mirror the MNIST dataset
Signed-off-by: Eli Uriegas [email protected]