prev_content is returned at the top-level by most of the CS API

[ara4n]

The spec is unclear about where prev_content should be returned. Most parts imply it should be at the top level of any events, but there are also parts that show it being returned under unsigned.

In practice, Synapse returns it in both places, except for the following APIs, where it is only returned under unsigned.

• /_matrix/client/r0/sync
• /_matrix/client/r0/notifications
• /_matrix/client/r0/rooms/{room_id}/state/{type}/{key} when called with the unspecified(!) format=event parameter.

[ara4n]

all the examples seem to show prev_content for state events being returned as a top level field on the event. but in practice synapse seems to return it most but not all of the time in unsigned. this is incredibly confusing

[richvdh]

basically a subset of #684

[turt2live]

Related: https://github.com/matrix-org/matrix-doc/issues/1167

[joepie91]

Just to clarify: is prev_content being in unsigned rather than top-level a spec bug or a Synapse implementation error?

[richvdh]

there's a lot of discussion about this on #2648. In short: the spec says it should be in the top-level, so clients and servers have to respect that. It's a shame that it says that, making this a protocol wart. (As opposed to a spec bug, which is where the spec lies about the protocol.)

[richvdh]

Oh man, this is a whole lot worse than I thought it was.

It's correct that Synapse has a v1 event serialisation format and a v2 serialisation format, and that the latter is only used by /sync, /notifications, and the undocumented format=event for /state.

However, my understanding of what it does for v1 was incorrect. Specifically, a bunch of keys (age, redacted_because, replaces_state, prev_content, invite_room_state, knock_room_state), are sent in both unsigned and at the top level of the event.

For most of the above, I can find no mention in the spec of those keys outside an unsigned section. The exception is prev_content, which is specced in various places (https://matrix.org/docs/spec/client_server/r0.6.1#state-event-fields among others) to be present at the top level as well as being mentioned under unsigned in some places.

Sooo... We should consider the two cases separately.

age, redacted_because, replaces_state, invite_room_state, knock_room_state

The spec says nothing about these being present at the top-level. Synapse returns copies there, but I think we can consider that a spec violation. Accordingly, let's track them on matrix-org/synapse#7925.

prev_content

This is absolutely a mess. There's enough in the spec about it being at the top level that we can't just take the same approach as the other key.

The fact that again, synapse returns it in both unsigned and at the top level, but only in unsigned for /sync makes me suspect that any real clients should be using the copy in unsigned, and updating the spec to say that's where it should be is attractive. But that needs an MSC.

[richvdh]

I'm actually going to reopen this, to track it being clarified in the spec.