Deviantart problem: The image at $FOO does not exist

[arisboch]

I updated the program a few minutes ago (from the GitHub source) and now I can't download any Deviantart post (I didn't use the program to download Deviantart material for a few days, though, so I can't be sure when it did break and why). Here's what the program says (-v flag used, of course):

[gallery-dl][debug] Version 1.10.2-dev
[gallery-dl][debug] Python 3.7.3 - Linux-5.0.0-25-generic-x86_64-with-Ubuntu-19.04-disco
[gallery-dl][debug] requests 2.21.0 - urllib3 1.24
[gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/kleoite/art/Best-boi-of-the-month-810164338'
[gallery-dl][debug] Updating urllib3 ciphers
[deviantart][debug] Using DeviantartDeviationExtractor for 'https://www.deviantart.com/kleoite/art/Best-boi-of-the-month-810164338'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/kleoite HTTP/1.1" 429 105
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /kleoite/art/Best-boi-of-the-month-810164338 HTTP/1.1" 200 None
[deviantart][error] The image at 'https://www.deviantart.com/kleoite/art/Best-boi-of-the-month-810164338' does not exist

The same goes for any Deviantart post and changing the user agent in the .conf file didn't help. Maybe an implementation of a log-in mechanism would help?

Edit: The download feature of JDownloader seems to be still working, so you maybe could look at the JDownloader source for “inspiration”?

[mikf]

The CI build from yesterday finished without error, so this change happened at some point since then.

DeviantArt seems to have finally replaced its old layout with the new one for everyone, and gallery-dl relied on it to map numerical IDs like 810164338 to their UUID counterpart used by the old OAuth API. Those UUIDs are no longer available on the new site, so a rewrite might be in order.

Entire galleries like https://www.deviantart.com/kleoite should still work, but single deviations and scraps are broken.

[kattjevfel]

While the latest commit (30d6e28) makes single deviations downloadable, target[filename] is not present, only filename, which is not very useful.

You can still view the "proper" name by getting the entire gallery, but that's hardly ideal.

target[filename]
  coffee_drunk_toddler_part_one_by_tugscarebear-dde2qc9

filename
  dde2qc9-0fc874cb-f923-4797-a8cb-9e37e68891f6

A solid way I've found to be able to get filenames again is to look at files[][src]
after every strp/ is the filename along with -150.*filetype*?

coffee_drunk_toddler_part_one_by_tugscarebear_dde2qc9-150.jpg?
best_boi_of_the_month_by_kleoite_ddecmsy-150.jpg?
open__eeveelution_inspired_mascots__by_plushpon_ddek6xu-150.png?

[mikf]

target[filename] came from the same source as filename did: the "filename" part of the download URL and those changed a bit for original files. And there were also lots of cases where you'd get the useless filename before the change on DeviantArt. This deviation for example had d5zx18-58d380af-9e09-4c71-985b-28a9f6ff6c0e as target[filename] on Juli 1st.

Anyway, I implemented your suggestion and found that GIFs and SWF files don't have a "proper" filename anywhere in their metadata. All files now have both a target[filename] and a filename (they are the same) taken from their thumbnail URL, but for those two file types they will still be "useless".

[kattjevfel]

Yeah I know it wasn't perfect, but hell deviantart is pretty far from perfect right now. Thanks for implementing it, at least it's better than having no filename.

Also nitpicking but could you forcefully change the last _ to a - (like they used to be before all this mess), I have a script for this that I typically run but it'd be nice to have it built in.
Of course up to you, since it technically does change the name from what deviantart hands out.
best_boi_of_the_month_by_kleoite_ddecmsy --> best_boi_of_the_month_by_kleoite-ddecmsy

Edit: Saw the commit implementing that as well, you're an absolute saint <3

[arisboch]

Now it works without problems, thanks! 👍🏻

Edit: typo

[mikf]

The original issue has been resolved and all DeviantArt media types now have a "proper" filename field (efb64ad), so I'm closing this.