Add experimental support for MSC4360: Sliding Sync Threads Extension #19005

devonh · 2025-10-01T16:06:36Z

Implements sliding sync extension from: matrix-org/matrix-spec-proposals#4360
Companion endpoint added in #19041

Disclaimer: I used Claude Code in a consultative manner to assist with parts of this PR.
I used it mainly for considering options for the SQL query and to help generate test cases.

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
Code style is correct (run the linters)

devonh · 2025-10-03T21:58:00Z

synapse/handlers/relations.py

    ) -> JsonDict:
        """Get related events of a event, ordered by topological ordering.

-        TODO Accept a PaginationConfig instead of individual pagination parameters.


This has been done already, the comment just wasn't updated.

MadLittleMods · 2025-10-14T16:38:39Z

synapse/handlers/sliding_sync/extensions.py

+                        # Check if this event is part of a thread
+                        relates_to = event.content.get("m.relates_to")
+                        if not isinstance(relates_to, dict):
+                            continue
+
+                        rel_type = relates_to.get("rel_type")
+
+                        # If this is a thread reply, track the thread
+                        if rel_type == RelationTypes.THREAD:
+                            thread_id = relates_to.get("event_id")


We don't have a helper for this kind of thing? Feels like something we'd be doing elsewhere already and could be wrapped up as a utility

I didn't see it anywhere, but I could take another look and add it as a helper where appropriate.

MadLittleMods · 2025-10-14T16:40:27Z

synapse/handlers/sliding_sync/extensions.py

+                if room_result.timeline_events:
+                    for event in room_result.timeline_events:
+                        # Check if this event is part of a thread
+                        relates_to = event.content.get("m.relates_to")


Probably should add these to EventContentFields

(applies to fields below as well)

Done - take a look & make sure it's as you expected.

MadLittleMods · 2025-10-14T16:43:53Z

synapse/handlers/sliding_sync/extensions.py

+            # Skip this thread if it already has events in the room timeline
+            # (unless include_roots=True, in which case we always include it)
+            if (update.room_id, update.thread_id) in threads_in_timeline:
+                continue


Can we structure things to avoid fetching bundled aggregations for thread_root_events if we end up skipping them anyway?

This should be restructured as you described now.

MadLittleMods · 2025-10-14T16:47:15Z

synapse/rest/client/sync.py

+        time_now = self.clock.time_msec()
+        response_content = await self.encode_response(
+            requester, sliding_sync_results, time_now
+        )


Good call on using the same time_now across all of the serialization that needs it 👍

(consistency)

MadLittleMods · 2025-10-14T16:56:08Z

synapse/storage/databases/main/relations.py

+            original_event_id: (
+                edits.get(edit_id)
+                if (edit_id := edit_ids.get(original_event_id))
+                else None
+            )


This is functionally pretty much the same right? Just making this safer?

The difference being that we won't lookup edits.get(None) where we wouldn't expect there to be a entry with the None key but possible.

Correct. (And to appease the linter)
We do this pattern elsewhere, this just uses the funky python syntax of inline variable assignment inside an if

MadLittleMods · 2025-10-14T17:08:24Z

synapse/storage/databases/main/relations.py

+                  FROM local_current_membership AS lcm
+                  WHERE lcm.room_id = threads.room_id
+                      AND lcm.user_id = ?
+                      AND lcm.membership = ?


We should be using the membership from what we already calculated within the token range. See synapse/handlers/sliding_sync/room_lists.py for where we originally calculate that.

MadLittleMods · 2025-10-14T17:08:43Z

synapse/storage/databases/main/relations.py

+    async def get_thread_updates_for_user(
+        self,
+        *,
+        user_id: str,
+        from_token: Optional[RoomStreamToken] = None,
+        to_token: Optional[RoomStreamToken] = None,
+        limit: int = 5,
+        include_thread_roots: bool = False,
+    ) -> Tuple[Sequence[ThreadUpdateInfo], Optional[StreamToken]]:
+        """Get a list of updated threads, ordered by stream ordering of their
+        latest reply, filtered to only include threads in rooms where the user
+        is currently joined.


Feels like this should be taking in the list of rooms that are being returned in the Sliding Sync response and only returning updates for those.

I thought about that, and even tried implementing it.
The problem becomes what should we return as a token that is usable on the /thread_updates endpoint?
I assume we would need to follow the same pattern as the receipts extension and handle initial, live, and previously rooms separately. The options for what token to return for pagination would be either the oldest from the three sets, or the newest from the three sets.
In the case of the oldest, further pagination could result in missing updates between the oldest token and the tokens from the other two sets.
In the case of the newest, further pagination could result in receiving duplicate entries.

Then there is the issue of how would you associate the set of rooms/lists with the token such that using it for pagination on /thread_updates makes sense. And would using the new endpoint update the sliding sync connection state? What if using different rooms/lists on the new endpoint than in sliding sync?

It's not that we couldn't come up with solutions to these things. I just felt they quickly over complicated things.
The overall intention was to just be able to track new events in threads easily and separately from the entire set of events to ensure clients can manage a threads notification center without having to paginate through all of a user's event history since the last time they logged in.

Maybe it's worth the complication to add room/list filtering, but I'm not convinced after having tried to implement it.

We could implement a bulk /messages endpoint, where the client would specify multiple rooms and prev_batch tokens.

-- matrix-org/matrix-spec-proposals#4186

Per the above, I think we would probably want to use all of the prev_batch tokens from all of the rooms timeline. Perhaps clunky, but maybe good.

And this also gets around the problem where you can't get to the thread updates that you care about without paginating through the bulk. This way, you can specify the rooms you want to get more thread updates in.

MadLittleMods · 2025-10-14T17:16:06Z

synapse/streams/config.py

+        # Helper function to extract StreamToken from either StreamToken or SlidingSyncStreamToken format
+        def extract_stream_token(token_str: str) -> str:


Where is this used? Where are we using a SlidingSyncStreamToken for another endpoint?

Just trying to understand the landscape as I think we eventually need this kind of thing (matrix-org/matrix-spec-proposals#4186 (comment))

It gets used in the /thread_updates companion endpiont #19041

I agree that the compatibility should be specced.
The spec mentions usage of tokens from /sync in the /messages api https://spec.matrix.org/v1.16/client-server-api/#get_matrixclientv3roomsroomidmessages
And the simplified sliding sync MSC says the prev_batch token is compatible with /messages
https://github.com/matrix-org/matrix-spec-proposals/blob/erikj/sss/proposals/4186-simplified-sliding-sync.md
prev_batch -> A token that can be passed as a start parameter to the /rooms/<room_id>/messages API to retrieve earlier messages.

So it seems like the landscape is already leaning in that direction, but maybe hasn't been formalized.

MadLittleMods · 2025-10-14T17:18:03Z

tests/rest/client/sliding_sync/test_extension_threads.py

+EXT_NAME = "io.element.msc4360.threads"
+
+
+class SlidingSyncThreadsExtensionTestCase(SlidingSyncBase):


Needs the standard test_wait_for_new_data, test_wait_for_new_data_timeout tests

MadLittleMods · 2025-10-14T17:18:35Z

changelog.d/19005.feature

@@ -0,0 +1 @@
+Add experimental support for MSC4360: Sliding Sync Threads Extension.


Also comments on the MSC that may change how we approach things here -> matrix-org/matrix-spec-proposals#4360 (review)

devonh added 6 commits October 1, 2025 10:04

Stub in threads sliding sync extension

cd4f422

Stub in threads extension tests

6c460b3

Stub in early db queries to get tests going

4602b56

Don't return empty fields in response

24b3873

Add user room filtering for threads extension

9ef4ca1

Add thread_root events to threads extension response

79ea4be

devonh commented Oct 3, 2025

View reviewed changes

devonh added 7 commits October 3, 2025 16:15

Fix linter error

6e69338

Add newsfile

4dd82e5

Only return rooms where user is currently joined

4c51247

Properly return prev_batch tokens for threads extension

ab7e5a2

Filter events from extension if in timeline

4d7826b

Allow SlidingSyncStreamToken in /relations

4cb0eea

Add indexes to improve threads query performance

c757969

devonh mentioned this pull request Oct 10, 2025

Add companion endpoint for MSC4360: Sliding Sync Threads Extension #19041

Open

3 tasks

devonh marked this pull request as ready for review October 10, 2025 22:49

devonh requested a review from a team as a code owner October 10, 2025 22:49

MadLittleMods reviewed Oct 14, 2025

View reviewed changes

MadLittleMods added the A-Sync label Oct 14, 2025

devonh added 2 commits October 16, 2025 17:37

Move constants to designated file

bf594a2

Don't fetch bundled aggregations if we don't have to

a3c7b3e

MadLittleMods mentioned this pull request Oct 22, 2025

MSC4308: Thread Subscriptions extension to Sliding Sync matrix-org/matrix-spec-proposals#4308

Open

		# Helper function to extract StreamToken from either StreamToken or SlidingSyncStreamToken format
		def extract_stream_token(token_str: str) -> str:

		EXT_NAME = "io.element.msc4360.threads"


		class SlidingSyncThreadsExtensionTestCase(SlidingSyncBase):

		@@ -0,0 +1 @@
		Add experimental support for MSC4360: Sliding Sync Threads Extension.

Add experimental support for MSC4360: Sliding Sync Threads Extension #19005

Are you sure you want to change the base?

Add experimental support for MSC4360: Sliding Sync Threads Extension #19005

Uh oh!

Conversation

devonh commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Checklist

Uh oh!

devonh Oct 3, 2025 • edited by MadLittleMods Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devonh Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devonh Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MadLittleMods Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

devonh commented Oct 1, 2025 •

edited

Loading

devonh Oct 3, 2025 •

edited by MadLittleMods

Loading

devonh Oct 15, 2025 •

edited

Loading

devonh Oct 15, 2025 •

edited

Loading

MadLittleMods Oct 16, 2025 •

edited

Loading