Tumblr rate limit problem

[SilkRose]

After the daily rate limit for Tumblr is reached, it doesn't wait and continues on through the list.

2022-09-10 19:47:03 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2050&limit=50&reblog_info=true HTTP/1.1" 200 None
2022-09-10 19:47:04 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2100&limit=50&reblog_info=true HTTP/1.1" 200 None
2022-09-10 19:47:04 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2150&limit=50&reblog_info=true HTTP/1.1" 200 None
2022-09-10 19:47:05 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2200&limit=50&reblog_info=true HTTP/1.1" 200 None
2022-09-10 19:47:05 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2250&limit=50&reblog_info=true HTTP/1.1" 200 None
2022-09-10 19:47:06 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2300&limit=50&reblog_info=true HTTP/1.1" 200 None
2022-09-10 19:47:06 debug: https://api.tumblr.com:443 "GET /v2/blog/4gifs.tumblr.com/posts?offset=2350&limit=50&reblog_info=true HTTP/1.1" 429 144
2022-09-10 19:47:06 ERROR: Daily API rate limit exceeded
2022-09-10 19:47:06 ERROR: Aborting - Rate limit will reset at 08:06:39
2022-09-10 19:47:06 debug: Starting DownloadJob for '4mltp.tumblr.com'
2022-09-10 19:47:06 debug: Using custom OAuth1.0 authentication
2022-09-10 19:47:06 debug: Using TumblrUserExtractor for '4mltp.tumblr.com'
2022-09-10 19:47:06 debug: Starting new HTTPS connection (1): api.tumblr.com:443
2022-09-10 19:47:06 debug: https://api.tumblr.com:443 "GET /v2/blog/4mltp.tumblr.com/posts?offset=0&limit=50&reblog_info=true HTTP/1.1" 429 147
2022-09-10 19:47:06 ERROR: Daily API rate limit exceeded
2022-09-10 19:47:06 ERROR: Aborting - Rate limit will reset at 08:06:39
2022-09-10 19:47:06 debug: Starting DownloadJob for '4testicles.tumblr.com'
2022-09-10 19:47:06 debug: Using custom OAuth1.0 authentication
2022-09-10 19:47:06 debug: Using TumblrUserExtractor for '4testicles.tumblr.com'
2022-09-10 19:47:06 debug: Starting new HTTPS connection (1): api.tumblr.com:443
2022-09-10 19:47:06 debug: https://api.tumblr.com:443 "GET /v2/blog/4testicles.tumblr.com/posts?offset=0&limit=50&reblog_info=true HTTP/1.1" 429 143
2022-09-10 19:47:06 ERROR: Daily API rate limit exceeded
2022-09-10 19:47:06 ERROR: Aborting - Rate limit will reset at 08:06:39
2022-09-10 19:47:06 debug: Starting DownloadJob for '56.media.tumblr.com'
2022-09-10 19:47:06 ERROR: No suitable extractor found for '56.media.tumblr.com'
2022-09-10 19:47:06 debug: Starting DownloadJob for '5bitsabuck.tumblr.com'
2022-09-10 19:47:06 debug: Using custom OAuth1.0 authentication
2022-09-10 19:47:06 debug: Using TumblrUserExtractor for '5bitsabuck.tumblr.com'
2022-09-10 19:47:06 debug: Starting new HTTPS connection (1): api.tumblr.com:443
2022-09-10 19:47:07 debug: https://api.tumblr.com:443 "GET /v2/blog/5bitsabuck.tumblr.com/posts?offset=0&limit=50&reblog_info=true HTTP/1.1" 429 143
2022-09-10 19:47:07 ERROR: Daily API rate limit exceeded
2022-09-10 19:47:07 ERROR: Aborting - Rate limit will reset at 08:06:39
2022-09-10 19:47:07 debug: Starting DownloadJob for '5oussn.tumblr.com'
2022-09-10 19:47:07 debug: Using custom OAuth1.0 authentication
2022-09-10 19:47:07 debug: Using TumblrUserExtractor for '5oussn.tumblr.com'
2022-09-10 19:47:07 debug: Starting new HTTPS connection (1): api.tumblr.com:443
2022-09-10 19:47:07 debug: https://api.tumblr.com:443 "GET /v2/blog/5oussn.tumblr.com/posts?offset=0&limit=50&reblog_info=true HTTP/1.1" 429 148
2022-09-10 19:47:07 ERROR: Daily API rate limit exceeded
2022-09-10 19:47:07 ERROR: Aborting - Rate limit will reset at 08:06:39
2022-09-10 19:47:07 debug: Starting DownloadJob for '5oussnnsfw.tumblr.com'
2022-09-10 19:47:07 debug: Using custom OAuth1.0 authentication

[mikf]

You want gallery-dl to wait for several hours instead of aborting the current job? I guess I can implement an option to control this behavior.

[SilkRose]

Yes, That option would be great.

[mikf]

The new release has a ratelimit option for Tumblr that can be set to "wait" to, well, wait.