Skip to content

Commit

Permalink
[twitter] allow disabling 'cursor' output (#5990)
Browse files Browse the repository at this point in the history
  • Loading branch information
mikf committed Aug 17, 2024
1 parent 78ae0ba commit c0668f5
Show file tree
Hide file tree
Showing 2 changed files with 33 additions and 2 deletions.
20 changes: 20 additions & 0 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3985,6 +3985,26 @@ Description
* ``"cookies"``: Use token given by the ``ct0`` cookie if present.


extractor.twitter.cursor
------------------------
Type
* ``bool``
* ``string``
Default
``true``
Example
``"1/DAABCgABGVKi5lE___oKAAIYbfYNcxrQLggAAwAAAAIAAA"``
Description
Controls from which position to start the extraction process from.

* ``true``: Start from the beginning.
Log the most recent ``cursor`` value when interrupted before reaching the end.
* ``false``: Start from the beginning.
* any ``string``: Start from the position defined by this value.

Note: A ``cursor`` value from one timeline cannot be used with another.


extractor.twitter.expand
------------------------
Type
Expand Down
15 changes: 13 additions & 2 deletions gallery_dl/extractor/twitter.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,11 @@ def _make_tweet(self, user, url, id_str):
}

def _init_cursor(self):
return self.config("cursor") or None
cursor = self.config("cursor", True)
if not cursor:
self._update_cursor = util.identity
elif isinstance(cursor, str):
return cursor

def _update_cursor(self, cursor):
self.log.debug("Cursor: %s", cursor)
Expand Down Expand Up @@ -590,9 +594,16 @@ def _update_cursor(self, cursor):
return cursor

def tweets(self):
self._cursor = cursor = self.config("cursor") or None
reset = False

cursor = self.config("cursor", True)
if not cursor:
self._update_cursor = util.identity
elif isinstance(cursor, str):
self._cursor = cursor
else:
cursor = None

if cursor:
state = cursor.partition("/")[0]
state, _, tweet_id = state.partition("_")
Expand Down

0 comments on commit c0668f5

Please sign in to comment.