Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patreon not downloading multiple attachments #693

Closed
calboi91 opened this issue Apr 15, 2020 · 0 comments
Closed

Patreon not downloading multiple attachments #693

calboi91 opened this issue Apr 15, 2020 · 0 comments
Labels

Comments

@calboi91
Copy link

Hi,

There is a post with multiple attachments (.rar files in this case), and only the first attachment is downloaded. The others are skipped. Here is the debug output:

[gallery-dl][debug] Version 1.13.4
[gallery-dl][debug] Python 3.8.2 - Windows-10-10.0.18362
[gallery-dl][debug] requests 2.23.0 - urllib3 1.25.8
[gallery-dl][debug] Starting DownloadJob for 'https://www.patreon.com/posts/march-patreon-in-17996054'
[patreon][debug] Using PatreonPostExtractor for 'https://www.patreon.com/posts/march-patreon-in-17996054'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.patreon.com:443
[patreon][debug] ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) (1/5)
[urllib3.connectionpool][debug] Starting new HTTPS connection (2): www.patreon.com:443
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /posts/march-patreon-in-17996054 HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://www.patreon.com:443 "GET /api/user/5235188 HTTP/1.1" 200 None
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): c10.patreonusercontent.com:443
[downloader.http][warning] ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) (1/5)
[urllib3.connectionpool][debug] Starting new HTTPS connection (2): c10.patreonusercontent.com:443
[urllib3.connectionpool][debug] https://c10.patreonusercontent.com:443 "GET /3/e30%3D/patreon-media/p/post/17996054/651fe9ad6dbf446f9c8ee54d44453f13/1.jpg?token-time=1588126345&token-hash=fYS9CSlTKI44d2zdxy41LbgiEJbVnQQmbEf66QHipBc%3D HTTP/1.1" 200 296228
* test\patreon\Yasume Yukito\17996054_Marc...reon Reward Kai - Blonde in Twintail_01.jpg
[patreon][debug] skipping https://c10.patreonusercontent.com/3/eyJhIjoxLCJwIjoxfQ%3D%3D/patreon-media/p/post/17996054/651fe9ad6dbf446f9c8ee54d44453f13/1.jpg?token-time=1587003145&token-hash=wevRvQa0CxCVkYw1yj5O6vdk2wMkGML2yHbYtzCZJuk%3D (651fe9ad6dbf446f9c8ee54d44453f13 image)
[urllib3.connectionpool][debug] https://www.patreon.com:443 "HEAD /file?h=17996054&i=2046107 HTTP/1.1" 302 0
[urllib3.connectionpool][debug] https://c10.patreonusercontent.com:443 "GET /3/eyJhIjoxLCJwIjoxfQ%3D%3D/patreon-media/p/post/17996054/f64b2cdb6df840f8b9d03a48f66b574d?token-time=1587175957&token-hash=OPb6JysEHqiX3RibLFtoeothLrICdd8EoUEls75Bq0o%3D HTTP/1.1" 200 3320955
* test\patreon\Yasume Yukito\17996054_Marc...reon Reward Kai - Blonde in Twintail_02.rar
[urllib3.connectionpool][debug] https://www.patreon.com:443 "HEAD /file?h=17996054&i=2046110 HTTP/1.1" 302 0
[patreon][debug] skipping https://c10.patreonusercontent.com/3/eyJhIjoxLCJwIjoxfQ%3D%3D/patreon-media/p/post/17996054/76f25ed36b47468eba35d02824dcc756?token-time=1587175958&token-hash=Zj3k2uaYVWCysQMUOHxgZB_4l8wT0RNXip1zljUduUE%3D (17996054 attachment)
[urllib3.connectionpool][debug] https://www.patreon.com:443 "HEAD /file?h=17996054&i=2046243 HTTP/1.1" 302 0
[patreon][debug] skipping https://c10.patreonusercontent.com/3/eyJhIjoxLCJwIjoxfQ%3D%3D/patreon-media/p/post/17996054/acefc5a79a7f42a4a35c0692093c4724?token-time=1587175958&token-hash=3m13SEMwj0VV9Rb2UpAvHMrK3R5CP8gtQj5X3PvgE1c%3D (17996054 attachment)
[urllib3.connectionpool][debug] https://www.patreon.com:443 "HEAD /file?h=17996054&i=2046245 HTTP/1.1" 302 0
[patreon][debug] skipping https://c10.patreonusercontent.com/3/eyJhIjoxLCJwIjoxfQ%3D%3D/patreon-media/p/post/17996054/af2b42b7a9ad4dd1afff994ffadb7272?token-time=1587175958&token-hash=TRtE1DK8v4iFjK8HUaWftRp9DyPVmib_6ptX0xfDZO4%3D (17996054 attachment)

It's weird because the downloads have different names and hashes, but they are skipped after the first one. I read in another issue #590 that the hash is used to detect duplicates, so it is weird that duplicates are detected when the hashes are different. These files are also skipped when setting the skip to false or enumerate.

@mikf mikf added the bug label Apr 15, 2020
@mikf mikf closed this as completed in 08674a9 Apr 17, 2020
mikf added a commit that referenced this issue Apr 29, 2020
Instead of accessing a specific part of a download URL, potentially
causing an exception if it doesn't exist, we're now searching through
all parts for a potential MD5 hash without ever raising an exception.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants