[Gelbooru] Missing Favorites Extractor

[9696neko]

Hello!

When I run gallery-dl "https://gelbooru.com/index.php?page=favorites&s=view&id=1111111" --write-metadata -o skip=true where 1111111 is my user ID, I get:
gallery-dl: Unsupported URL 'https://gelbooru.com/index.php?page=favorites&s=view&id=1111111

From looking at gallery-dl --list-extractors I can discern only "Pool, Post, and Tag" are defined for Gelbooru.

Reading the code in gallery_dl/extractor/ it looks like there are Favorites Extractors defined for GelbooruV01 and V02. It looks like a favorite extractor section is missing, that would probably use V01/V02's, in the main gelbooru.py file?

I would like to request this Favorites addition to the Gelbooru Extractors. Thanks!

[mikf]

Favorites listings like https://gelbooru.com/index.php?page=favorites&s=view&id=1111111 seem to require login / redirect to the login page.

Would it be OK to internally use the results from https://gelbooru.com/index.php?page=post&s=list&tags=fav:1111111, which for some reason does not require a login, or are its results somehow different?

[9696neko]

The results from your link format are publically accessible but they differ from logged-in favorites in two ways:

1. [Important] They are sorted by descending image id compared to the logged-in view's chronological (newest added to oldest). This would affect people who want to run subsequent update runs where gallery-dl aborts after say five duplicates instead of scanning every favorite.
2. By default Gelbooru hides fringe results behind a setting that can be set without an account called "Display all site content." If not set, some images will be excluded whether logged in or not.

So I believe it is not OK unless this is the only method we can rely on due to sorted by id difference.

[9696neko]

Great stuff. Thank you very much!

I will wait for next release before testing it.

[9696neko]

I ended up testing this out and it works! Build: 1.25.0-dev

Some nitpicks:

• It initially found no images by using Gelbooru API key but then I remembered it needed login cookies. Maybe the error message could mention this?
• I did notice you maybe forgot to update --list-extractors with this new supported format?

[mikf]

I found way to access favorites without cookies, API key, or any form of authentication: dcb8af6.

What I haven't found is a way to have them returned in descending order; &order=desc has no effect. The current code has to do a bit extra to reverse them and --range does not work as expected

---

--list-extractors skips extractors without docstring, which was the case for the favorite extractor. Fixed in b756dc1.

[9696neko]

Very nice work.

I originally tried to find the source code for *booru's (to grab the GET params API) but had no luck.
From looking at the commits, you reverse the favorites before querying (which I guess are chronologically ascending). If so, the current code seems fine as a workaround until we find another way.

Regarding --range you may want to disable it for this extractor unless we can figure something out. I think unexpected results may be worse then nothing at all because people may mistakenly think their range flag was successful.

Sorry I can't contribute more because my Python skills are rusty.