Leave out optional keys from /sync

[deepbluev7]

This leaves out all optional keys from /sync. This should be fine for all clients tested against conduit already, but it may break some clients, as such we should check, that at least most of them don't break horribly and maybe back out some of the individual changes. (We can probably always leave out groups for example, while the others may cause more issues.)

Clients I have tested:

• Nheko
• Element (slightly outdated version)
• FluffyChat (old version)

This also needs an updated sytest, see matrix-org/sytest#1040

In my testing this decreases the sync json response to about half, which saves about 30% of data on every /sync. It also makes debugging a lot easier, since the response json is easier to read. :3

Tell me if the news entry is descriptive enough, since some people may want to be aware of this change, if they are client developers or so. Also, my python skills suck, I'm sure there is a tidier way to do this. Also, no idea how to run against the newer sytest version? Is that automatic, after it is merged?

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
• Pull request includes a sign off
• Code style is correct (run the linters)

Signed-off-by: Nicolas Werner [email protected]

[ShadowJonathan]

Please note this addresses #6579, see if it also fixes it.

---

Also, thank you for taking this on, I made #8336 in the past, but i more-or-less left that one to bitrot because I couldn't figure out how to fix the sytests, thanks :D

[clokep]

I'm going to set the review flag on this even though it has failing tests, since I believe they should be fixed by matrix-org/sytest#1040.

[deepbluev7]

Seems like some integration tests are failing, that I didn't see, when I ran them locally (probably overlooked them). I'll try to fix that soonish.

[deepbluev7]

Should be fine now, although the tests would now fail, if you don't exclude the keys from /sync.

[richvdh]

looks good! a couple of suggestions here.

[richvdh]

lgtm

[richvdh]

thanks very much!

[deepbluev7]

Thank you for the review! <3

[richvdh]

heads-up that this is being reverted in #9940

[bmarty]

Hi,
You said that you've tested Element

Element (slightly outdated version)

It would be nice to be more precise. Element Web, Element Android and Element iOS are three different implementations, so may have different bugs :/

[deepbluev7]

@bmarty My bad, I missed to put Web. I only tested Element Web, since I have no Android or iOS device. That wasn't intentional, I just missed it! Sorry!