Synapse doesn't backfill missing events when rejoining a room later

[sbug0]

Description

If I'm the only user from my home server in a room (with more users from other severs),
stays there for a while and get's a lot of events,
then leave the room,
then rejoins the room a few days later,
I'm missing the events in the room between the point where I left the room up to the point I rejoined the room.

Synapse does not backfill those events.

Steps to reproduce

• As the only user from your home server, join a room with other servers
• Stay in the room for a while to get events in the room ( like messages, joins etc)
• Leave the room for a while (so long as to there being new events in the room)
• Rejoin the room
• See that the missing events doesn't get backfilled

You will see something like this:

• you joining the room
• messages
• messages
• messages
• you leaving the room

---

missing events you will not see

• you joining the room
• new messages
• new messages
  homeserver_logs.tar.gz

I have reported this before years ago but I shut down that server before Matrix team started to investigate, and when they looked at the bug they closed it because I wasn't active any more. So this is an old bug and not something newly introduced.

Homeserver

sinnesro.se

Synapse Version

{"server_version":"1.84.1","python_version":"3.9.2"}

Installation Method

Debian packages from packages.matrix.org

Database

Single PostgreSQL, never used SQLite, never restored from backup

Workers

Single process

Platform

Debian 11 latest updated
Running in an LXD

Intel Core i5 12th gen, 16 GB RAM, 256 GB SSD

Configuration

No response

Relevant log output

I have included my home server logs earlier in the bug report because I don't know what is relevant to show here.

Anything else that would be useful to know?

No response

[erikjohnston]

Hi @sbug0, can I confirm the room you joined in the logs is !eAoSQVNgWAxwNNJvrX:matrix.org?

Judging from the logs it looks like the client didn't even try to paginate. Am I right in thinking after you join the room a second time on the client you see something like (from old to new messages):

• old message
• old message
• leave
• join
• new message

And there should be messages between the leave and join?

If so, then I think this is a bug where we incorrectly send down the older events in the /sync response after joining the room. This would cause clients to only paginate backwards from the old messages, and so the server would never realise it needed to fetch the events in the gap (because the client never explicitly asked for it).

[sbug0]

Hi.

Yes, that is the room in question.

And yes, that's what I'm seeing in my client. (Doesn't matter which client, or if I start a new fresh client).

And yes, there should be messages between my leave and join. (As seen from a client I have on another server in that room).

[erikjohnston]

Thanks @sbug0

I think we need to add a condition in this loop:

synapse/synapse/handlers/sync.py

Line 579 in dfd77f4

while limited and len(recents) < timeline_limit and max_repeat:

to ensure that we don't send down events from before the gap... somehow

[MadLittleMods]

One alternative is to use MSC3871: Gappy timelines to indicate the gap so clients can retry fetching in the place with missing history.

[rom4nik]

FTR, previous occurence of this issue with a few more reports: #8936