-
Notifications
You must be signed in to change notification settings - Fork 379
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MSC3859: Add well known media domain proposal #3859
Changes from all commits
81ed592
cb259ac
f1ca606
86123b0
7ba95c8
4e7bf60
8aba847
305c9fa
c38579f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
# MSC3859: .well-known Media Domain | ||
|
||
Currently Matrix clients and servers discover homeservers and identify servers via the `.well-known/matrix/client` and | ||
`.well-known/matrix/server` endpoints. This is currently limited to the homeserver and for clients the identity server. | ||
This MSC proposes adding an additional field that can be used to specify a media server that can optionally be used | ||
for `/_matrix/media/*` endpoints. | ||
|
||
The reasoning behind this change is to support hosting media behind a CDN designed and optimised specifically for | ||
serving media. Since media transfers and endpoints are very different to the other JSON based endpoints a separate | ||
domain offers maximum flexibility for server owners. This is particularly important for path optimisation between | ||
client and server over long distances where home internet providers generally offer inferior routing between countries | ||
than CDN providers do. By allowing clients to upload/download from the closest CDN Point of Presence they will benefit from improved | ||
upload and download performance. | ||
|
||
This cannot be achieved currently unless the entire homeserver is served behind a CDN, which may have other cost or | ||
management implications (or simply be impossible, depending on CDN). A separate domain allows server admins to | ||
decide the best setup for their users. | ||
|
||
|
||
## Proposals | ||
|
||
### Extend the client .well-known | ||
|
||
Add a new optional field, `m.media_server` that can specify a separate URL to be used for media requests. Clients can | ||
then optionally use this field for all media endpoints, including both download and upload. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If it's for upload as well, I'm guessing the client is expected to use the same access token as it would use when communicating with the homeserver. Is that correct? |
||
|
||
```json | ||
{ | ||
"m.homeserver": { | ||
"base_url": "https://matrix.example.com" | ||
}, | ||
"m.media_server": { | ||
"base_url": "https://matrix-media.example.com" | ||
} | ||
} | ||
``` | ||
|
||
To ensure backwards compatibility the homeserver base URL must also be able to serve the same media requests (via a | ||
proxy or some other backend system). | ||
|
||
### Extend the server .well-known | ||
|
||
This is the same as above but for the server endpoint: | ||
|
||
```json | ||
{ | ||
"m.server": "matrix.example.com:443", | ||
"m.media_server": "matrix-media.example.com:443" | ||
} | ||
``` | ||
|
||
As above, the non-media endpoint must also serve media requests. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please follow the proposal template. You're missing some sections that I would expect to see for this type of proposal, in particular "Alternatives", "Security considerations", and "Unstable prefix". Some possible alternatives:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Missng sections added in 86123b0. Re: login response I don't think this works because media is accessible without authentication with a known MXID which login would prevent. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
That seems reasonable. I'd suggest still including that in the "Alternatives" section -- part of the purpose of that section is to indicate other possible ways to achieve the goal, and say why the proposed solution is better. |
||
|
||
|
||
## Alternatives | ||
|
||
### Redirects | ||
|
||
For the download path, the homeserver could send a redirect response to a CDN backed domain. This is proposed in | ||
[MSC3860](https://github.com/matrix-org/matrix-spec-proposals/pull/3860). | ||
|
||
For the upload path this is not possible under HTTP. | ||
|
||
|
||
## Security Considerations | ||
|
||
Server admins may have to manage two distinct domains/installs increasing management & attack vector. | ||
|
||
|
||
## Unstable Prefix | ||
|
||
While not released in the Matrix spec implementations should use field `com.beeper.msc3859.m.media_server` in place of | ||
`m.media_server` in the well known responses: | ||
|
||
```json5 | ||
// Client | ||
{ | ||
"com.beeper.msc3859.m.media_server": { | ||
"base_url": "https://matrix-media.example.com" | ||
} | ||
} | ||
// Server | ||
{ | ||
"com.beeper.msc3859.m.media_server": "matrix-media.example.com:443" | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs expanding, or at least covering in the alternatives. As mentioned, for my own server I'm one flag away from having CDN-driven media without spec changes. Is there a particular CDN provider in mind for your use case which prevents working within the existing spec?
For reference, the setup I've been looking at is effectively deploying my own CDN on top of a cloud provider: there's a reverse proxy in several geographical areas using DNS to direct traffic there, with the media repo running in that datacenter to serve media from its CDN-backed backend (so all media is available everywhere). The traffic routing for client-server and server-server traffic is effectively unchanged in this setup, though might use a backbone back to the real server hosted in a single geographical area. The latency would be nearly the same as if a user was talking to the server over an ocean directly.
Granted, this is not a super cost effective solution, but it does work within how Matrix expects a server to operate. In practice, we'd also want to support proper geographically-segmented homeservers for traffic balancing in that respect, however that's much harder to do (and likely an implementation problem rather than spec).
A benefit of this proposal is it allows the media domain to be different from the homeserver domain, which is also something we want to do, though I do wonder if making the media repo be a proper first class citizen in the spec is a good idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will add this to alternatives! The reverse proxy / CDN over everything solution certainly works but forces all traffic to be routed via that solution. Our use case specifically is we'd like to run Synapse inside AWS (as we currently do) but have the media routes served elsewhere due to AWS' prohibitive bandwidth costs. Without a separate domain the only option for that is a solution similar to yours where a matrix-aware reverse proxy needs to sit on the CDN side routing requests accordingly.
While this is certainly possible (cloudflare workers, fly.io, custom build CDN) it's represents a significant amount of work vs. a separate domain which to me feels like a very simple and easy solution. I can't see any negatives to doing this (esp. as an optional field).
Does having a separate domain really make the media repo a first class citizen? Perhaps there should be some kind of reliance on a homeserver to avoid that, or perhaps it's actually a good thing? (Multiple homeserver owners could share a media repo install or something along those lines, way beyond this proposal).