MSC3077: Support for multi-stream VoIP #3077

SimonBrandner · 2021-03-29T18:02:08Z

Fixes https://github.com/matrix-org/matrix-doc/issues/560
Requires #2746
Rendered

js-sdk implementation
react-sdk implementation

My first MSC, please try not to kill me 😅

FCP tickyboxes

Signed-off-by: Šimon Brandner <[email protected]>

proposals/3077-multi-stream-voip.md

Co-authored-by: Tulir Asokan <[email protected]>

Signed-off-by: Šimon Brandner <[email protected]>

…atrix-doc into msc/sdp-metadata

ShadowJonathan

screenshare when 👀

ShadowJonathan

Follow-up: I would suggest adding some more explanation to what m.usermedia and m.screenshare actually means in this (new?) context, to understand what this adds exactly, but otherwise, this looks good.

Signed-off-by: Šimon Brandner <[email protected]>

dbkr

otherwise this looks great - congrats on your first MSC :)

proposals/3077-multi-stream-voip.md

I would be tempted to skip the capability advertisement for this and just say that the absence of stream metadata means clients just take the first or whatever MSC2746 says. I can't think of anything the capability advertisement caters for specifically? - Dave Signed-off-by: Šimon Brandner <[email protected]>

Signed-off-by: Šimon Brandner <[email protected]>

proposals/3077-multi-stream-voip.md

Signed-off-by: Šimon Brandner <[email protected]>

dbkr · 2021-04-07T09:15:30Z

proposals/3077-multi-stream-voip.md

+
+ `purpose` - a string indicating the purpose of the stream. For compatibility
+  between clients values `m.usermedia` and `m.screenshare` are defined.
+  `m.usermedia` is the stream that contains the webcam and microphone tracks.


continuing thread on whether we should identify front/rear facing cameras with this, for purposes like when the user rotates the camera and the same stream changes to a different camera (eg. FaceTime displays an animation on the remote side to show that the camera has changed). I feel like this is probably something for separate MSC, unless we want two separate purposes for front/rear camera.

Yeah, unless we want a separate purpose for that probably a separate MSC. If we wanted a separate purpose, would we have m.usermedia, m.usermedia.front and m.usermedia.rear or something else?

eg. FaceTime displays an animation on the remote side to show that the camera has changed

This bit seems more like a stream replacement problem rather than a purpose problem. But... maybe we could allow sending both and set a main stream which would tell the client on the other side to do the animation

My suggestion would be adding something like camere_type to further distinguish the cameras - this way not all clients need to support this and we avoid having two purposes that are almost the same

proposals/3077-multi-stream-voip.md

Co-authored-by: Richard van der Hoff <[email protected]>

mscbot · 2023-06-20T11:33:55Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

* Draft of multi-stream MSC Signed-off-by: Šimon Brandner <[email protected]> * Remove unnecessary article and don't use CamelCase Signed-off-by: Šimon Brandner <[email protected]> * Fix naming and MSC number Signed-off-by: Šimon Brandner <[email protected]> * Make it prefixy Co-authored-by: Tulir Asokan <[email protected]> * Be more descriptive about keys Signed-off-by: Šimon Brandner <[email protected]> * Add more info about usermedia and screenshare Signed-off-by: Šimon Brandner <[email protected]> * Simplifie backwards compatibility I would be tempted to skip the capability advertisement for this and just say that the absence of stream metadata means clients just take the first or whatever MSC2746 says. I can't think of anything the capability advertisement caters for specifically? - Dave Signed-off-by: Šimon Brandner <[email protected]> * Reword parts of backwards compatibility section Signed-off-by: Šimon Brandner <[email protected]> * Improve explanation of backwards compatibility Signed-off-by: Šimon Brandner <[email protected]> * Add missing spaces to unstable perifix table Signed-off-by: Šimon Brandner <[email protected]> * Remove support for stream-replacement Signed-off-by: Šimon Brandner <[email protected]> * Apply suggestions from code review Co-authored-by: Andrew Morgan <[email protected]> * Fix concerns Signed-off-by: Šimon Brandner <[email protected]> * Link to specific spec version Co-authored-by: Richard van der Hoff <[email protected]> * Be more readable Co-authored-by: Richard van der Hoff <[email protected]> * Clarify Co-authored-by: Richard van der Hoff <[email protected]> * Don't ref non-existing thing Co-authored-by: Richard van der Hoff <[email protected]> * Remove confusing words --------- Signed-off-by: Šimon Brandner <[email protected]> Co-authored-by: Tulir Asokan <[email protected]> Co-authored-by: David Baker <[email protected]> Co-authored-by: Andrew Morgan <[email protected]> Co-authored-by: Richard van der Hoff <[email protected]>

Specs matrix-org/matrix-spec-proposals#3077

turt2live · 2023-08-01T05:09:28Z

Spec PR: matrix-org/matrix-spec#1602

turt2live · 2024-03-13T17:12:18Z

Spec PR: matrix-org/matrix-spec#1735

turt2live · 2024-03-19T18:58:34Z

Merged 🎉

SimonBrandner added 2 commits March 29, 2021 12:09

Draft of multi-stream MSC

6b11473

Signed-off-by: Šimon Brandner <[email protected]>

Remove unnecessary article and don't use CamelCase

f8a38dd

Signed-off-by: Šimon Brandner <[email protected]>

SimonBrandner changed the title ~~MSC: Support for multi-stream VoIP~~ MSC3077: Support for multi-stream VoIP Mar 29, 2021

Fix naming and MSC number

98053bb

Signed-off-by: Šimon Brandner <[email protected]>

SimonBrandner marked this pull request as ready for review March 29, 2021 18:03

turt2live added kind:feature MSC for not-core and not-maintenance stuff proposal A matrix spec change proposal proposal-in-review labels Mar 29, 2021

tulir reviewed Mar 29, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner and others added 3 commits March 29, 2021 20:57

Make it prefixy

e5a95f4

Co-authored-by: Tulir Asokan <[email protected]>

Be more descriptive about keys

5939d19

Signed-off-by: Šimon Brandner <[email protected]>

Merge branch 'msc/sdp-metadata' of https://github.com/SimonBrandner/m…

14b37ba

…atrix-doc into msc/sdp-metadata

ShadowJonathan approved these changes Mar 30, 2021

View reviewed changes

ShadowJonathan reviewed Mar 30, 2021

View reviewed changes

Add more info about usermedia and screenshare

721eec5

Signed-off-by: Šimon Brandner <[email protected]>

dbkr reviewed Apr 1, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner added 2 commits April 1, 2021 20:42

Reword parts of backwards compatibility section

668ef57

Signed-off-by: Šimon Brandner <[email protected]>

kevincox reviewed Apr 2, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

proposals/3077-multi-stream-voip.md Show resolved Hide resolved

SimonBrandner added 2 commits April 2, 2021 16:34

Improve explanation of backwards compatibility

9f372fa

Signed-off-by: Šimon Brandner <[email protected]>

Add missing spaces to unstable perifix table

96a878b

Signed-off-by: Šimon Brandner <[email protected]>

dbkr reviewed Apr 7, 2021

View reviewed changes

SimonBrandner commented Apr 10, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

SimonBrandner commented Apr 10, 2021

View reviewed changes

proposals/3077-multi-stream-voip.md Outdated Show resolved Hide resolved

This was referenced Apr 14, 2021

once supported by m.call.*, add screensharing mid-call element-hq/element-web#239

Closed

Make screen share option more obvious (and toggle from normal call) element-hq/element-web#12652

Closed

SimonBrandner mentioned this pull request May 7, 2021

Support for screen-sharing using multi-stream VoIP (MSC3077) matrix-org/matrix-js-sdk#1685

Merged

turt2live added the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Jun 8, 2021

This was referenced Jul 22, 2021

MSC3291: Muting in VoIP calls #3291

Merged

Readonly support for MSC3077 element-hq/element-android#3734

Open

SimonBrandner and others added 4 commits June 17, 2023 18:28

Be more readable

875edb9

Co-authored-by: Richard van der Hoff <[email protected]>

Clarify

744fa24

Co-authored-by: Richard van der Hoff <[email protected]>

Don't ref non-existing thing

3d22c03

Co-authored-by: Richard van der Hoff <[email protected]>

Remove confusing words

8d43930

mscbot added finished-final-comment-period and removed disposition-merge final-comment-period This MSC has entered a final comment period in interest to approval, postpone, or delete in 5 days. labels Jun 20, 2023

turt2live merged commit ddad150 into matrix-org:old_master Jun 20, 2023

turt2live added spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec and removed finished-final-comment-period labels Jun 20, 2023

SimonBrandner deleted the msc/sdp-metadata branch June 20, 2023 16:22

dbkr added a commit to matrix-org/matrix-spec that referenced this pull request Jul 18, 2023

Spec for MSC3077

47db8f2

Specs matrix-org/matrix-spec-proposals#3077

dbkr mentioned this pull request Jul 18, 2023

Spec for MSC3077 matrix-org/matrix-spec#1602

Closed

turt2live added spec-pr-in-review A proposal which has been PR'd against the spec and is in review and removed spec-pr-missing Proposal has been implemented and is being used in the wild but hasn't yet been added to the spec labels Aug 1, 2023

zecakeh mentioned this pull request Feb 28, 2024

Add support for multi-stream VoIP matrix-org/matrix-spec#1735

Merged

turt2live added merged A proposal whose PR has merged into the spec! and removed spec-pr-in-review A proposal which has been PR'd against the spec and is in review labels Mar 19, 2024

zecakeh mentioned this pull request Mar 22, 2024

Implement Matrix 1.10 changes ruma/ruma#1735

Closed

28 tasks

clokep mentioned this pull request Mar 22, 2024

Declare support for Matrix v1.10 element-hq/synapse#17021

Closed

8 tasks

zecakeh mentioned this pull request Mar 22, 2024

events: Add support for multi-stream VoIP ruma/ruma#1752

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC3077: Support for multi-stream VoIP #3077

MSC3077: Support for multi-stream VoIP #3077

SimonBrandner commented Mar 29, 2021 •

edited by richvdh

Loading

ShadowJonathan left a comment

ShadowJonathan left a comment

dbkr left a comment

dbkr Apr 7, 2021

SimonBrandner Apr 7, 2021

SimonBrandner Apr 7, 2021

SimonBrandner Jul 7, 2022

mscbot commented Jun 20, 2023

turt2live commented Aug 1, 2023

turt2live commented Mar 13, 2024

turt2live commented Mar 19, 2024

MSC3077: Support for multi-stream VoIP #3077

MSC3077: Support for multi-stream VoIP #3077

Conversation

SimonBrandner commented Mar 29, 2021 • edited by richvdh Loading

ShadowJonathan left a comment

Choose a reason for hiding this comment

ShadowJonathan left a comment

Choose a reason for hiding this comment

dbkr left a comment

Choose a reason for hiding this comment

dbkr Apr 7, 2021

Choose a reason for hiding this comment

SimonBrandner Apr 7, 2021

Choose a reason for hiding this comment

SimonBrandner Apr 7, 2021

Choose a reason for hiding this comment

SimonBrandner Jul 7, 2022

Choose a reason for hiding this comment

mscbot commented Jun 20, 2023

turt2live commented Aug 1, 2023

turt2live commented Mar 13, 2024

turt2live commented Mar 19, 2024

SimonBrandner commented Mar 29, 2021 •

edited by richvdh

Loading