Allow backwards search through cursor-based APIs

[seboslaw]

As a user I'd like to retain my reading position within a feed when the app refreshes the feed in the background.

Consider the following scenario: the user has loaded a feed of 30 posts. The user then scrolls down to post # 20 and subsequently closes the app. When the user opens the app 2 hours later she/he is met with the same view that showed when the app was closed (post # 20). However, in the background the app should refresh the feed and show an indicator at the top that there is new content. The user can then scroll up and read all the new posts that have come in.

To implement the previous scenario I'd need to be able to hand over a searchDirection parameter to the APIs that allow to pass in a cursor. That way I could paginate into the future by handing over the latest cursor I have in my app and load all the content that is newer than that cursor. Right now I can only paginate into the past.

I hope this makes sense :)

[mozzius]

This is by far the most common feature request I get, I'd love to see this

[bnewbold]

This functionality seems reasonable!

The semantics of cursor as I think of it usually to include all the query params, and continue scrolling responses in a consistent order/direction. But maybe we can carefully document the usage here. It is also intentionally ambiguous what the cursor represents: in some cases a timestamp, in other cases the feed has a different ordering.

Rolling this out would require implementation by third party feed generators. Maybe feed generators would need a way to indicate whether they actually support this functionality, otherwise things could get janky and out-of-order when switching scroll order.

cc: @pfrazee @devinivy @whyrusleeping for product and backend thoughts.

[badlogic]

@bnewbold for the time being, it would be great to get some info on the cursor formats used for standard feeds like e.g. app.bsky.feed.getTimeline. For example, this is a cursor returned by this endpoint: 1699997001718::bafyreiagjsnq4mxhmqqk36s4iaznoxyhwivcxq3zsrdogvkrc6sx2kmeeu. That's clearly a UTC timestamp plus the CID of the post of the next page after the current page. Is that correct? If so, how likely is this cursor to change? I assume the UTC timestamp plus CID of the last seen post in my client can be used to look into the future? E.g. "give me all posts newer than CID, starting from UTC timestamp X"

Being able to do this now just for the actor timeline would already be very useful. For other feeds, the UI can tell the user that scroll position memoization is not supported for now.

[devinivy]

That's right that our cursor is a timestamp and then a CID 👍 The timestamp allows us to paginate chronologically, then the CID is a tie-breaker in case multiple items were created in the same millisecond. This allows us to order by creation time then, in the case of a tie, lexically by CID.

That said, the idea is that the cursor is an opaque value, and its format is considered to be an implementation detail of the service. Feed generators also determine their own cursor formats. We don't have plans to change it, but unfortunately I can't promise that it will never change to the same extent that lexicon gives us other API guarantees over time.

[bnewbold]

Yup, second that cursors are opaque, can differ across implementations, and the format could change any time.

[surfdude29]

Some more context in case it's helpful: bluesky-social/social-app#976 requests this feature in the app (mainly for the Following feed, seems to be the consensus) and it currently has 29 👍.

[surfdude29]

Update: it now has 60 👍