Datasets download files are incomplete！

[123xiaochen]

Hello, I am using datasets to download large quantities of files, after many attempts finally succeeded, but there are some files that are incomplete.

I'm using the--dehydrated mode in datasets.

But when I download directly through ftp, the file is larger than that downloaded using datasets.

What is the reason? This kind of file accounts for about one hundred of the total number of files. Thanks.

[ericcox1]

Hi @123xiaochen,

Thank you for opening this issue. Could you please share the exact command, including any identifiers, that you used so that we can try to reproduce this bug?

Best,
Eric