Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MSC3923: Bringing Matrix into the IETF process #3923

Merged
merged 6 commits into from
Mar 26, 2023
Merged

MSC3923: Bringing Matrix into the IETF process #3923

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
58725e0
First cut of an MSC+IETF spec process
turt2live Nov 2, 2022
749b72a
Rewrite for process considerations
turt2live Mar 10, 2023
d150713
Update links
turt2live Mar 13, 2023
7cab71c
Don't forget affected MSCs
turt2live Mar 13, 2023
20c6900
Apply suggestions from code review
turt2live Mar 15, 2023
c15f99e
Clarify what kind of server-specific behaviour is trapped in the CS API
turt2live Mar 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 120 additions & 0 deletions proposals/3923-ietf-spec-process.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# MSC3923: Bringing Matrix into the IETF process
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just capturing some in-person early feedback from IETF: This appears to be a good way to approach the problem of governance, particularly when trying to integrate with the IETF.


The More Instant Messaging Interoperability ([MIMI](https://datatracker.ietf.org/wg/mimi/about/))
working group aims to specify the minimal set of mechanisms required to support modern messaging
in an interoperable way, and Matrix is an example of how that can be accomplished by acting as a
generic and openly specified communications layer. Matrix currently uses a specification process
that allows for rapid prototyping, which the ecosystem relies on: it is important to Matrix to
maintain this highly dynamic approach to specification while still being able to participate in
venues such as IETF for interoperable messaging.

This proposal covers the process approach on the Matrix side to support a venture into the IETF
process, largely as it relates to being able to rapidly build/test features without going through
a naturally-lengthy IETF review.

## Background

Matrix is currently specified at https://spec.matrix.org/v1.6 and split into several coarse domains:

1. The Client-Server API (or "CS API"), covering the communication between a client and server.
2. The Server-Server API (or "Federation API"), covering communication (transport) between servers.
3. The Room Version specifications, which define how servers (and sometimes clients) are expected to
behave in a given room. This is the core feature of Matrix: how a room actually works.
4. The Application Service API (or "Appservice API"), which defines an interface for high-traffic bots
and bridges to communicate with a homeserver.
5. The Identity Service API, covering how clients and servers interact with an identity server. Identity
servers store mappings of third party identifiers to Matrix IDs, and are not part of the authentication
or user infrastructure in Matrix: you can use Matrix without ever touching an identity server.
6. The Push Gateway API, which enables mobile app developers to use push notifications in a decentralized
environment.
7. Supporting documents such as the introduction page and appendices, largely covering grammar and
general descriptions of Matrix.

The intention of splitting Matrix into these domains is to allow for implementations of Matrix's core
principles without having to be tied unnecessarily to writing APIs that won't be used. For example, it's
entirely reasonable that a purpose-built homeserver only implement some room versions and the federation
API without ever attempting to support appservices or the specified Client-Server API. This is often
the case when a project is adding support for Matrix: they are most interested in interoperating with
other Matrix homeservers and already have their own client-server API to work with.

**Note**: Historically, a lot of the behaviours a server needs to implement have ended up in Matrix's
Client-Server API. It is a goal of the Spec Core Team (SCT) to move the remaining behaviours to a more
correct place, therefore reinforcing the original intent mentioned above more concretely. This MSC
describes a world where the SCT's goal has been achieved, not the current state. A natural consequence
of going through the IETF process is that Matrix's own specification will improve as areas are identified
as needing better segmentation.

## Proposal

With consideration for how Matrix is split into major domains, only a small portion of Matrix's core needs to be specified for MIMI. Specifically, the areas covered by
[I-D.ralston-mimi-matrix-framework](https://datatracker.ietf.org/doc/draft-ralston-mimi-matrix-framework/)
([MSC3977](https://github.com/matrix-org/matrix-spec-proposals/pull/3977)): a single room version,
definitions for what a homeserver, event, room, and user are, loose descriptions for what is needed
of a federation transport API, and noting the importance of end-to-end encryption in the messaging
sphere. The Client-Server API, Application Service API, Identity Service API, and Push Gateway API are all
entirely out of scope because they're simply not needed for MIMI.

Proposing these areas through the IETF process as-is would normally mean that they get transferred
to the IETF, using the IETF process instead of MSCs for any future changes after being accepted.
This could put a damper on Matrix's ability to experiment with features, though. In order to avoid
this damper, we instead use an LTS (Long-Term Stable) approach where the core of Matrix is versioned
as an LTS within the IETF process and regular/non-LTS Matrix continues as-is.

Logistically, this means room versions (as the primary area of concern) will get an "LTS stable"
designation and associated identifier. Room versions in non-LTS Matrix will continue to get created,
though not all of them will end up in the IETF process: when it makes sense to do, such as when a
given room version "feels" stable enough, that room version will get proposed to the IETF through
the normal Internet-Draft process.

For example, if given room versions 10, 11, 12, and 13 from Matrix, room version 13 might be proposed
to the IETF while the others simply aren't. While 13 works through the IETF process (being renamed
as `I.2` in the proposal), changes might happen to it and other non-LTS room versions get created.
Like with Matrix->LTS intent, anything which makes sense to bring to the other process gets raised
as such in the traditional ways (an MSC in the case of non-LTS, and an Internet-Draft in the case
of IETF).

Room versions typically hold the core of Matrix, however anything which needs to enter the IETF
process would do so just the same as room versions: anything which makes sense, and when it makes
sense, would go through the IETF process and carry any useful changes back to Matrix as MSCs.

Similarly, it's entirely possible that an IETF Internet-Draft gets raised without an accompanying
MSC to change how the LTS version of Matrix works: if that change makes sense to bring over, it
would be.

In practice, the Matrix.org Foundation would be the ones ensuring both backwards compatibility
between LTS and non-LTS Matrix as well as ferrying changes back and forth as needed. This would
fall under the Spec Core Team's remit.

### Handling backwards-incompatible changes

Given the IETF and Matrix processes can both modify their copy of the protocol without involving the
other process, it's very possible that one doesn't work with the other anymore. This should be quite
easy to mitigate, however: because we're using room versions to contain the core protocol, servers
intending to support both LTS and non-LTS versions simply implement both room versions and they'll
be covered.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that imply effectively having two incompatible higher-level over-protocols (mainly, room versions) coexisting over the same under-protocol (the federation and a common foundation of the two room versions), segregating the ecosystem into those flying MIMI and those flying Matrix, with a small overlap where not just servers but also the consuming parties agree to use both over-protocols? Would that be different from, for example, Google/Facebook taking XMPP, with only a basic common set supported by both and therefore usable from Pidgin; or UCaaS players taking SIP and growing something on top of it that a modest Asterisk PBX has not much to do with? Not that I have a better solution, just trying to map a potential direction it would take.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not expecting the IETF spec to diverge from regular Matrix too far, but it's certainly possible. The plan to handle this is to ensure the protocol is structured such that it can be contained to purely room version changes (API/transport differences are also possible, though less likely and more easily mitigated). By keeping everything in a room version, it means we can literally copy/paste the IETF spec into regular Matrix as needed (through the MSC process, still1): server implementations honouring the regular Matrix spec will therefore be encouraged to implement that room version, just as they would be today if we introduced a whole new version.

Today's server implementations shouldn't see much impact, if any at all: room versions going to the IETF should effectively be clones of what Matrix already has, just renumbered to handle the risk of change. Worst case is Matrix ends up shipping two room versions in close proximity (say, 11 and I.1) with different semantics, though that's no different than us dropping v3 on the world and a month later releasing v4: it's still [annoying] effort, but contained to room versions. IETF room versions are expected to be every N years as well, making the chances of an annoying conflict even smaller (or less dramatic, at least).

Footnotes

  1. The expected flow for room versions is regular Matrix to IETF, however changes are possible in the IETF process and we'd need to solidify the number anyways. The MSCs which bring the IETF room versions back to regular Matrix would intentionally push all improvements out to other, dedicated, MSCs (like we've done for edits, reactions, etc). The FCP on these MSCs would be asking if there's any good reason to not include that room version in Matrix, such as in an extreme worst case where the IETF room version violates the principles of Matrix (which shouldn't happen anyways: IETF is pretty aligned in the areas that matter). ↩

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the record, out of band we're talking a lot about how an LTS version of Matrix might work, or at least the stepping stones to get there. There's still belief that the process proposed here will function, though there might be some work to get to this point - more to be revealed as we continue to explore this space :)


## Alternatives

Other ideas have been discussed with some members of the Spec Core Team, with the combination of them
appearing in the above proposal. Taking elements of the proposal and creating a new proposal around them
is feasible, though those approaches do not independently solve the concerns the SCT has. Namely, we'd
like to:

* have a compatible version of Matrix specified at the IETF level (specifically for federation)
* be free to experiment without the burden of process, and rapidly respond to shifts in the larger,
external, ecosystem as needed
* keep the layers which aren't needed out of the IETF, for sake of implementation effort for Matrix-compatible
implementations from the IETF level

... and other points along those same sentiments.

Suggestions for alternative approaches are welcome, though unlike other MSCs, solutions which are more
carefully considered than usual are appreciated.

## Dependent MSCs

This MSC ends up affecting the future of the following MSCs, though is not dependent itself:
* https://github.com/matrix-org/matrix-spec-proposals/pull/3918
* https://github.com/matrix-org/matrix-spec-proposals/pull/3919
* https://github.com/matrix-org/matrix-spec-proposals/pull/3977
* Future MSCs/I-Ds for Matrix-as-MIMI