Twitter DL stops partway through an attempted DL

[87527z]

I ran this command through a batch file (its worked a million times before so idk why it wouldn't now):

cd C:\Users\iam87\AppData\Local\Programs\Python\Python310\Scripts
gallery-dl https://twitter.com/i/bookmarks

and after downloading 50-60 links it terminates.

I ran the same argument with "-v" and it terminates with this :

[urllib3.connectionpool][debug] https://twitter.com:443 "GET /i/api/graphql/uKP9v_I31k0_VSBmlpq2Xg/Bookmarks?variables=%7B%22count%22%3A100%2C%22includePromotedContent%22%3Afalse%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withBirdwatchPivots%22%3Afalse%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withClientEventToken%22%3Afalse%2C%22withBirdwatchNotes%22%3Afalse%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Afalse%2C%22__fs_interactive_text%22%3Afalse%2C%22__fs_dont_mention_me_view_api_enabled%22%3Afalse%2C%22cursor%22%3A%22HBaAuN%2FNhZ%2FpkAwAAA%3D%3D%22%7D HTTP/1.1" 200 271
[twitter][debug] {'bookmark_timeline': {}}
[twitter][error] Unable to retrieve Tweets from this timeline

[mikf]

This might be because your login session has expired. Depending on how you logged in, use --clear-cache twitter or refresh your cookies.

Attempting to download bookmarked Tweets without being logged in shows a very similar error, although it does not get anything (i.e. no "first page"):

$ gallery-dl -v twitter.com/i/bookmarks
...
[urllib3.connectionpool][debug] https://twitter.com:443 "GET /i/api/graphql/uKP9v_I31k0_VSBmlpq2Xg/Bookmarks?variables=%7B%22count%22%3A100%2C%22includePromotedContent%22%3Afalse%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withBirdwatchPivots%22%3Afalse%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withClientEventToken%22%3Afalse%2C%22withBirdwatchNotes%22%3Afalse%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Afalse%2C%22__fs_interactive_text%22%3Afalse%2C%22__fs_dont_mention_me_view_api_enabled%22%3Afalse%7D HTTP/1.1" 200 58
[twitter][debug] {'bookmark_timeline': {}}
[twitter][error] Unable to retrieve Tweets from this timeline

[87527z]

Yeah that solved it, Thanks!

[87527z]

I have spoken too soon as I often do, its back with a different error

I ran it through a batch file with these parameters

gallery-dl --clear-cache twitter
gallery-dl -v https://twitter.com/i/bookmarks
pause

[urllib3.connectionpool][debug] https://twitter.com:443 "GET /i/api/graphql/uKP9v_I31k0_VSBmlpq2Xg/Bookmarks?variables=%7B%22count%22%3A100%2C%22includePromotedContent%22%3Afalse%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withBirdwatchPivots%22%3Afalse%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withClientEventToken%22%3Afalse%2C%22withBirdwatchNotes%22%3Afalse%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Afalse%2C%22__fs_interactive_text%22%3Afalse%2C%22__fs_dont_mention_me_view_api_enabled%22%3Afalse%2C%22cursor%22%3A%22HBaArPv5%2F66NkQwAAA%3D%3D%22%7D HTTP/1.1" 200 284

[mikf]

its back with a different error

It still stops before it downloaded all bookmarked Tweets, is that it?

I thought Twitter might have changed something and things are broken, but everything still works on my end. It downloads all my bookmarks, even with a new login session.

[87527z]

Yeah it stops on something I bookmarked 3-4 days ago even though I have been bookmarking things since well before that

[87527z]

Okay, new problem. I updated to the latest version just in case that was the issue and now It cant download anything now and returns with a 404 error. Here's the full readout

C:\Users\iam87\AppData\Local\Programs\Python\Python310\Scripts>gallery-dl -v https://twitter.com/i/bookmarks
[gallery-dl][debug] Version 1.21.2
[gallery-dl][debug] Python 3.10.2 - Windows-10-10.0.19044-SP0
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.8
[gallery-dl][debug] Starting DownloadJob for 'https://twitter.com/i/bookmarks'
[twitter][debug] Using TwitterBookmarkExtractor for 'https://twitter.com/i/bookmarks'
[twitter][info] Logging in as [USERNAME CENSORED]
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): twitter.com:443
[urllib3.connectionpool][debug] https://twitter.com:443 "GET /login HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://twitter.com:443 "POST /sessions HTTP/1.1" 404 93
[twitter][error] HttpError: '404 Not Found' for 'https://twitter.com/sessions'

[mikf]

I updated to the latest version
[gallery-dl][debug] Version 1.21.2

The latest version is 1.23.5, but I don't think that makes a difference regarding bookmarks.

I made an attempt to improve bookmark pagination in commit 15cd114, but that hasn't been releases yet. Could you install the -dev version or use an executable from https://github.com/mikf/gallery-dl/actions/runs/3443401960 and see if that fixes your problem?

[twitter][error] HttpError: '404 Not Found' for 'https://twitter.com/sessions'

Login with username & password seems to be broken. Using cookies will still work, though.

[87527z]

Okay, I realized why i wasnt on the latest version. (Forgot to use "py -3")

However the dev version has not solved this issue.
Proof :
C:\Users\iam87\AppData\Local\Programs\Python\Python310\Scripts>gallery-dl -v https://twitter.com/i/bookmarks
[gallery-dl][debug] Version 1.24.0-dev
[gallery-dl][debug] Python 3.10.2 - Windows-10-10.0.19044-SP0
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.8
[gallery-dl][debug] Starting DownloadJob for 'https://twitter.com/i/bookmarks'
[twitter][debug] Using TwitterBookmarkExtractor for 'https://twitter.com/i/bookmarks'
[twitter][info] Logging in as [REDACTED]
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): twitter.com:443
[urllib3.connectionpool][debug] https://twitter.com:443 "GET /login HTTP/1.1" 200 None
[urllib3.connectionpool][debug] https://twitter.com:443 "POST /sessions HTTP/1.1" 404 93
[twitter][error] HttpError: '404 Not Found' for 'https://twitter.com/sessions'

[mikf]

Well, you should have read the entire comment:

[twitter][error] HttpError: '404 Not Found' for 'https://twitter.com/sessions'

Login with username & password seems to be broken. Using cookies will still work, though.