Skip to content

Commit

Permalink
[danbooru] extend 'metadata' option
Browse files Browse the repository at this point in the history
make it possible to specify a custom list of metadata includes
  • Loading branch information
mikf committed Jan 13, 2023
1 parent 26c3292 commit c87bd1a
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 10 deletions.
11 changes: 10 additions & 1 deletion docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1027,13 +1027,22 @@ Description
extractor.danbooru.metadata
---------------------------
Type
``bool``
* ``bool``
* ``string``
* ``list`` of ``strings``
Default
``false``
Example
* ``replacements,comments,ai_tags``
* ``["replacements", "comments", "ai_tags"]``
Description
Extract additional metadata
(notes, artist commentary, parent, children, uploader)

It is possible to specify a custom list of metadata includes.
See `available_includes <https://github.com/danbooru/danbooru/blob/2cf7baaf6c5003c1a174a8f2d53db010cf05dca7/app/models/post.rb#L1842-L1849>`__
for possible field names. ``aibooru`` also supports ``ai_metadata``.

Note: This requires 1 additional HTTP request per post.


Expand Down
25 changes: 16 additions & 9 deletions gallery_dl/extractor/danbooru.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-

# Copyright 2014-2022 Mike Fährmann
# Copyright 2014-2023 Mike Fährmann
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
Expand Down Expand Up @@ -40,7 +40,17 @@ def __init__(self, match):

self.ugoira = self.config("ugoira", False)
self.external = self.config("external", False)
self.extended_metadata = self.config("metadata", False)

metadata = self.config("metadata", False)
if metadata:
if isinstance(metadata, (list, tuple)):
metadata = ",".join(metadata)
elif not isinstance(metadata, str):
metadata = "artist_commentary,children,notes,parent,uploader"
self.metadata_includes = metadata
else:
self.metadata_includes = None

threshold = self.config("threshold")
if isinstance(threshold, int):
self.threshold = 1 if threshold < 1 else threshold
Expand Down Expand Up @@ -99,13 +109,10 @@ def items(self):
url = post["large_file_url"]
post["extension"] = "webm"

if self.extended_metadata:
template = (
"{}/posts/{}.json?only=artist_commentary,children,notes,"
"parent,uploader"
)
resp = self.request(template.format(self.root, post["id"]))
post.update(resp.json())
if self.metadata_includes:
meta_url = "{}/posts/{}.json?only={}".format(
self.root, post["id"], self.metadata_includes)
post.update(self.request(meta_url).json())

if url[0] == "/":
url = self.root + url
Expand Down

0 comments on commit c87bd1a

Please sign in to comment.