Some way to use the same archive for multiple sites?

[KonoVitoDa]

I would like to use the same file for Sankaku Complex, Gelbooru and Danbooru, so if I download a file from one site, if I try to download the same file (same md5) from another site, the file will be skipped.

[nisehime]

One of the possible configs:

{
  "extractor": {
     "booru": {
       "archive": "./booru_db.sqlite3",
       "archive-format": "{md5}"
    }
}

But it won't work because gallery-dl automatically adds category name at the beginning of each string. So, without code modification there's no easy way.

[mikf]

Added an archive-prefix option to control what gets placed before each archive ID, if anything.

     "booru": {
       "archive": "./booru_db.sqlite3",
       "archive-format": "{md5}",
       "archive-prefix": ""
    }

[nisehime]

@mikf Isn't it basically a duplicate option? There's no difference if I set {category} in the archive-format like {category}{id}_{filename} (assuming there's no prefix) or split it to archive-prefix and archive-format like {category}+{id}_{filename}

[thatfuckingbird]

@mikf Isn't it basically a duplicate option? There's no difference if I set {category} in the archive-format like {category}{id}_{filename} (assuming there's no prefix) or split it to archive-prefix and archive-format like {category}+{id}_{filename}

This way it doesn't break existing configs.

[Hrxn]

@mikf Isn't it basically a duplicate option? There's no difference if I set {category} in the archive-format like {category}{id}_{filename} (assuming there's no prefix) or split it to archive-prefix and archive-format like {category}+{id}_{filename}

But it won't work because gallery-dl automatically adds category name at the beginning of each string. So, without code modification there's no easy way.

That's a contradiction, by the way.

Should've looked at the commit first..

[Skyofflad]

Wait. You can use md5 for archive? Or is it a theoretical parameter?
When i tried using the example above - "archive-format": "{md5}" gallery-dl exited with an error.

[nisehime]

Only for the sites which have such keyword, I guess it's only boorus. Check with the -K

[Skyofflad]

Only for the sites which have such keyword, I guess it's only boorus. Check with the -K

Shame, it would be very useful in kemonoparty extractor.