http.client.IncompleteRead: IncompleteRead(1932289 bytes read, 14877546 more expected)

[woshixiaozhou]

Dear author, thank you for sharing.
In the process of downloading, I encountered the following problems. How can I solve this problem?
Thank you very much for your reply

[royorel]

Hello @woshixiaozhou

This error seems like some sort of a random network connection issue

Can you try to rerun the download_ffhq_aging.py with the --check_invalid_images flag and see if the problem still exists?

Keep in mind that the initial setup time will be slower because the script will check the validity of every downloaded image

[woshixiaozhou]

你好@woshixiaozhou

此错误似乎是某种形式的随机网络连接问题

您可以尝试重新运行download_ffhq_aging.py带有--check_invalid_images标志的，看看问题是否仍然存在吗？

请记住，初始设置时间会变慢，因为脚本会检查每个下载图像的有效性

Thank you for your reply
I tried your method and it did not solve the problem. Is there any other way to solve this problem?

[royorel]

@woshixiaozhou,

Try to see if it happens with the same image every time or is it a different image each time.
You would have to change the code a little bit for that and print out the image number.

FFHQ-Aging-Dataset/download_ffhq_aging.py

Lines 223 to 241 in 2ecdcd2

	def _download_thread(spec_queue, exception_queue, stats, dst_dir, output_size, drive, download_kwargs):
	with requests.Session() as session:
	while not spec_queue.empty():
	spec = spec_queue.get()
	try:
	if drive != None:
	pydrive_utils.pydrive_download(drive, spec['file_url'], spec['file_path'])
	else:
	download_file(session, spec, stats, **download_kwargs)
	if spec['file_path'].endswith('.png'):
	align_in_the_wild_image(spec, dst_dir, output_size)
	os.remove(spec['file_path'])
	except:
	exception_queue.put(sys.exc_info())
	with stats['lock']:
	stats['files_done'] += 1

Add this line below line 237
print(os.path.basename(spec['file_path']))

That might help a little bit to understand what's going on

[woshixiaozhou]

@woshixiaozhou,

Try to see if it happens with the same image every time or is it a different image each time.
You would have to change the code a little bit for that and print out the image number.

FFHQ-Aging-Dataset/download_ffhq_aging.py

Lines 223 to 241 in 2ecdcd2

	def _download_thread(spec_queue, exception_queue, stats, dst_dir, output_size, drive, download_kwargs):
	with requests.Session() as session:
	while not spec_queue.empty():
	spec = spec_queue.get()
	try:
	if drive != None:
	pydrive_utils.pydrive_download(drive, spec['file_url'], spec['file_path'])
	else:
	download_file(session, spec, stats, **download_kwargs)
	if spec['file_path'].endswith('.png'):
	align_in_the_wild_image(spec, dst_dir, output_size)
	os.remove(spec['file_path'])
	except:
	exception_queue.put(sys.exc_info())
	with stats['lock']:
	stats['files_done'] += 1

Add this line below line 237
print(os.path.basename(spec['file_path']))

That might help a little bit to understand what's going on

Thank you for your reply
The problem is not on the same picture
Is this due to my network instability or the server?

[royorel]

I think that might be the case. If that's indeed the issue, then even though that's not an elegant solution, I would suggest running the script with the --check_invalid_images flag several times until all images are downloaded