[BUG][Weibo] 403 Forbidden Failed to download

[pxssy]

Noticed this today

gallery-dl --dest M:/ https://weibo.com/1907840615/MbvNTd5Eh
* M:\gallery-dl\weibo\1907840615 啊板的猫\4827438653119981_01.jpg
[30][WARNING] <downloader.http> <http.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\downloader\http.py> <'403 Forbidden' for 'https://wx1.sinaimg.cn/large/71b75667ly1h7e37z7n45j21o02801kz.jpg'>
[40][ERROR] <download> <job.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\job.py> <Failed to download 4827438653119981_02.jpg>
* M:\gallery-dl\weibo\1907840615 啊板的猫\4827438653119981_03.jpg
[30][WARNING] <downloader.http> <http.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\downloader\http.py> <'403 Forbidden' for 'https://wx2.sinaimg.cn/large/71b75667ly1h7e380lpbaj21o0280kig.jpg'>
[40][ERROR] <download> <job.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\job.py> <Failed to download 4827438653119981_04.jpg>
[30][WARNING] <downloader.http> <http.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\downloader\http.py> <'403 Forbidden' for 'https://wx4.sinaimg.cn/large/71b75667ly1h7e381cba9j21fp1snhdt.jpg'>
[40][ERROR] <download> <job.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\job.py> <Failed to download 4827438653119981_05.jpg>
[30][WARNING] <downloader.http> <http.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\downloader\http.py> <'403 Forbidden' for 'https://wx4.sinaimg.cn/large/71b75667ly1h7e382k63tj21o02804o0.jpg'>
[40][ERROR] <download> <job.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\job.py> <Failed to download 4827438653119981_06.jpg>
[30][WARNING] <downloader.http> <http.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\downloader\http.py> <'403 Forbidden' for 'https://wx2.sinaimg.cn/large/71b75667ly1h7e383ws3fj21ke26h1kz.jpg'>
[40][ERROR] <download> <job.py> <C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\job.py> <Failed to download 4827438653119981_07.jpg>
* M:\gallery-dl\weibo\1907840615 啊板的猫\4827438653119981_08.jpg
* M:\gallery-dl\weibo\1907840615 啊板的猫\4827438653119981_09.jpg

Seems like some anti-hotlinking feature. I've seen it happen for rather old, archived posts, and gotten around it by refering it from the page. Never encountered this on gallery-dl

It can still be downloaded, but not before several attempts running gallery-dl on it. The situation is similar on a browser. Can there be an option to re-attempt several times before skipping over?

Also, I've noticed that if the server domain wx1 fails, wx2, wx3, wx4 might work as the other servers do host some redundancy. Can it also attempt those?

[mikf]

Try sending a Referer header. This usually helps for errors like this.

gallery-dl -o downloader.http.headers.Referer="https://weibo.com/1907840615/MbvNTd5Eh" https://weibo.com/1907840615/MbvNTd5Eh

(Just -o headers.Referer=https://weibo.com/ might be enough)

[pxssy]

Thanks.

-o headers.Referer=https://weibo.com/

worked and its not 403ing anymore but

-o downloader.http.headers.Referer="https://weibo.com/1907840615/MbvNTd5Eh"

doesn't. Doesn't in the sense that I'm getting

gallery-dl -o 'downloader.http.headers.Referer="https://weibo.com/1907840615/MbvNTd5Eh"' https://weibo.com/1907840615/MbvNTd5Eh
Traceback (most recent call last):
  File "C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\pxssy\AppData\Local\Programs\Python\Python39\Scripts\gallery-dl.exe\__main__.py", line 7, in <module>
  File "C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\__init__.py", line 71, in main
    config.set(*opts)
  File "C:\Users\pxssy\AppData\Local\Programs\Python\Python39\lib\site-packages\gallery_dl\config.py", line 171, in set
    conf[key] = value
TypeError: 'NoneType' object does not support item assignment

I just updated gallery-dl and its still like this. Is this a few feature?

[mikf]

No, and it works fine for me on v1.23.5 and master.

[mikf]

Referer headers for weino now get sent automatically: e3abab8

The TypeError: 'NoneType' object does not support item assignment error happens when downloader.http.headers is set to null, like it is in the default config file.

gallery-dl/docs/gallery-dl.conf

Line 384 in e3abab8

"headers": null