Multi Stream in SDK 3.0 Using Aux Streams

What is Multi-stream?

With multi-stream, you'll see the video of the most active participants in your meetings.

• Stream order will be allocated in the order of joining the meeting.
• If the joined participant count goes above 6 (self + remote + aux streams), they will get into the queue and will not be allocated to any of the available streams as our limit is reached.
• If some one from the queue will speak he will get into the allocated stream and will replace the stream who is less active in the meeting.
• If some one from the allocated stream leaves the meeting, the random participant from the queue will replace the stream.
• The Active Speaker API can be used to check if the participant is active speaker.
• The ActiveSpeakerChangedEvent is triggered if active speaker changes.

Typical Scenario

In a meeting with more than two participants, if you want to see the active speaker along with other joined participants, you can use multi-stream to achieve it.

Limitations

• You can only open 4 video streams at most.
• You cannot specify a stream showing a specific participant, as it is only determined by the activity of the participants.

How to Use

To implement multi-stream, clients should first inherit from the MultiStreamObserver interface and then implement it. The following are the main steps you should follow.

interface MultiStreamObserver {
    fun onAuxStreamAvailable(): View?
    fun onAuxStreamUnavailable(): View?
    fun onAuxStreamChanged(event: AuxStreamChangedEvent?)
}

1. Provide a view to open a stream

When there is a newly available stream, such as a new participant join in the meeting, the SDK will trigger callback OnAuxStreamAvailable and the client should give SDK a view for rendering, and the AuxStreamOpenedEvent would be triggered indicating whether the stream is successfully opened.

override fun onAuxStreamAvailable(): View? {
    return mediaRenderView
}

2. Provide a view to close the stream

If a stream is unavailable, such as a participant left the meeting and the number of joined participants is smaller than the number of opened streams, SDK will trigger callback OnAuxStreamUnavailable and the client should give SDK a view handle which will be closed or if the given view is null, SDK will automatically close the last opened stream if needed.

override fun onAuxStreamUnavailable(): View? {
    return null
}

3. Display the view of the stream

When a stream is opened (successfully or not), AuxStreamOpenedEvent will be triggered. On this event, the client can display the view of the stream if the result is successful.

4. Hide the view of the stream

When a stream is closed (successfully or not), AuxStreamClosedEvent will be triggered. On this event, the client can hide the view of the stream if the result is successful.

override fun onAuxStreamChanged(event: MultiStreamObserver.AuxStreamChangedEvent?) {
    val auxStream: AuxStream? = event?.getAuxStream()

    when (event) {
        is MultiStreamObserver.AuxStreamOpenedEvent -> {
            if (event.isSuccessful()) {
                ...
            } else {
                // error occurred
            }
        }
        is MultiStreamObserver.AuxStreamClosedEvent -> {
            if (event.isSuccessful()) {
                ...
            } else {
                // error occurred
            }
        }
        is MultiStreamObserver.AuxStreamSendingVideoEvent -> {
            ...     
        }
        is MultiStreamObserver.AuxStreamPersonChangedEvent -> {
            ...
        }
        is MultiStreamObserver.AuxStreamSizeChangedEvent -> {
            ...
        }
    }
}