Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Gelbooru] Trying to download a tag with a lot of images caps out at 20 thousand images #2232

Closed
Kelhsy opened this issue Jan 27, 2022 · 1 comment

Comments

@Kelhsy
Copy link

Kelhsy commented Jan 27, 2022

Trying to download a tag from gelbooru but the download fails after 20000 images are downloaded, here's the --verbose output:

[gelbooru][error] An unexpected error occurred: AttributeError - 'dict' object has no attribute 'attrib'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[gelbooru][debug]
Traceback (most recent call last):
File "requests\models.pyc", line 910, in json
File "json_init_.pyc", line 348, in loads
File "json\decoder.pyc", line 337, in decode
File "json\decoder.pyc", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "gallery_dl\extractor\gelbooru.pyc", line 40, in _pagination
File "gallery_dl\extractor\gelbooru.pyc", line 25, in _api_request
File "requests\models.pyc", line 917, in json
requests.exceptions.JSONDecodeError: [Errno Expecting value] Too deep! Pull it back some. Holy fuck.: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "gallery_dl\job.pyc", line 79, in run
File "gallery_dl\extractor\booru.pyc", line 29, in items
File "gallery_dl\extractor\gelbooru.pyc", line 46, in _pagination
AttributeError: 'dict' object has no attribute 'attrib'

@mikf mikf added the bug label Jan 27, 2022
mikf added a commit that referenced this issue Jan 27, 2022
Use 'id:<POSTID' as a tag instead of going through pages with 'pid'.

Something similar was already implemented in 93cef78,
but that got broken again in 3085aac.
@mikf
Copy link
Owner

mikf commented Jan 27, 2022

Fixed in e2be199.

As the commit message says, the 20k limit was already fixed in a previous commit, but I managed to break that again when updating to Gelbooru's changed API responses.

@mikf mikf closed this as completed Jan 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants