Live Annotations on shared screen

Introduction

Live Annotations is a feature provided by the Webex Android SDK that enables users to annotate in real-time on a shared screen stream during a meeting. This means users can add annotations directly onto the shared screen while the actual content is being rendered in real time.

• LiveAnnotations: This component is responsible for initializing the annotation session and receiving annotation data through callbacks. It provides methods to start, stop, and manage the annotation session.

• LiveAnnotationRenderer: This component assists in rendering the annotation data on the screen. It takes the annotation data to render on the shared screen.

Live Annotation is available from 3.11.0 version onwards

Getting Started

To use the Live Annotations feature, you need to obtain a LiveAnnotations object. This can be done using the getAnnotationHandle() method.

val liveAnnotations: LiveAnnotations? = call.getAnnotationHandle()

Note: The getAnnotationHandle() method will return null if, screen share is NOT active, or Live annotation is NOT supported.

Setting Live Annotation Listeners

To handle the events related to live annotations, you need to set a LiveAnnotationListener. This listener will provide callbacks for various events such as when live annotation starts, stops, when an annotation request is received, and when annotation data is received.

liveAnnotations?.setLiveAnnotationListener(object : LiveAnnotationListener {
    override fun onLiveAnnotationStarted() {
        // Use this callback to initialize the rendering part
    }

    override fun onLiveAnnotationStopped() {
        // Use this callback to stop rendering
    }

    override fun onLiveAnnotationRequestReceived(personId: String) {
        // Use this callback to get the user input for annotation permission
    }
    
    override fun onLiveAnnotationRequestExpired(personId: String) {
        // Use this callback to remove any UI related to annotation permission            
    }

    override fun onLiveAnnotationDataReceived(data: String) {
        // Use this callback to feed data to LiveAnnotationRenderer for rendering
    }
})

Annotation Requests

Managing Live Annotation Policies

Available Annotation Policies are:

• AnyoneCanAnnotate: Any participant can annotate freely.
• NeedAskForAnnotate: Participants must request permission to annotate.
• NobodyCanAnnotate: Annotation is prohibited for all participants.

The default policy for live annotations is LiveAnnotationsPolicy.NobodyCanAnnotate. You can check the current policy using the getLiveAnnotationsPolicy() method and set a new policy using the setLiveAnnotationsPolicy() method.

val currentPolicy = liveAnnotations?.getLiveAnnotationsPolicy()
liveAnnotations?.setLiveAnnotationsPolicy(LiveAnnotationsPolicy.NeedAskForAnnotate) { errorCode ->
    if (errorCode == null) {
        // Policy was successfully set
    } else {
        // Handle error
    }
}

Handling Annotation Requests (NeedAskForAnnotate Policy):

When the NeedAskForAnnotate policy is active, the LiveAnnotationListener.onLiveAnnotationRequestReceived() callback is invoked when a participant attempts to annotate. This callback is specific to this policy.

override fun onLiveAnnotationRequestReceived(personId: String) {
    // You can prompt the user to accept or deny the annotation request
}

Responding to Annotation Requests:

Display a UI element to allow the user to grant or decline the annotation request when permission request arived through LiveAnnotationListener.onLiveAnnotationRequestReceived() Send the response to the WebexSDK using the respondToLiveAnnotationRequest() method:

fun handleAnnotationPermission(grant: Boolean, personID: String) {
    liveAnnotationHandle?.respondToLiveAnnotationRequest(personID, grant) {
        if (it.isSuccessful) {
            Log.d(tag, "permission handled")
        } else {
            Log.d(tag, "permission error: ${it.error?.errorMessage}")
        }
    }
}

Managing Annotation Data

LiveAnnotationRenderer

LiveAnnotationRenderer is a class provided by the WebexSDK. It manages the rendering of annotation data on the screen during a live annotation session.

The LiveAnnotationRenderer class offers several key methods:

• startRendering(): This method initiates the rendering process. It prepares the LiveAnnotationSurfaceView and CloseLiveAnnotationButton, and then adds them to the annotation layer.

• stopRendering(): This method halts the rendering process. It ceases the rendering of annotation data and removes the annotation layer from the screen.

• renderData(String data): This method renders the annotation data. It accepts a string of annotation data in a proprietary format, processes it, and then draws the corresponding annotations onto the LiveAnnotationSurfaceView within the annotation layer.

• setAnnotationRendererCallback(LiveAnnotationRendererCallback callback): This method sets a callback for the rendering process. The callback provides methods that are invoked when the rendering process is ready to start and when it has been stopped.

Here's an example of how you might use the LiveAnnotationRenderer in your code:

annotationRenderer = LiveAnnotationRenderer(context).apply {
    setAnnotationRendererCallback(object : LiveAnnotationRenderer.LiveAnnotationRendererCallback {
        override fun onAnnotationRenderingReady() {
            // This callback is triggered when the annotation renderer is ready to draw on the annotation layer
        }
        override fun onAnnotationRenderingStopped() {
            // This callback is triggered when the host stops the annotations by clicking the close button on the annotation layer
            // Use this callback to stop the annotation session
            liveAnnotationHandle()?.stopLiveAnnotations()
        }
    })
    
    // This initializes the Annotation Renderer and draws the annotation layer for rendering annotation data
    startRendering()
}

In this example, context is the context in which the LiveAnnotationRenderer is being used.

LiveAnnotationLayer

LiveAnnotationLayer is a custom view provided by the WebexSDK. It is responsible for rendering the annotation data onto the shared screen. It processes the annotation data and draws the corresponding annotations on the shared screen. It is prepared when startRendering() is called and removed when stopRendering() is called. Here's an example of how you might use the getAnnotationLayer() method in your code:

val annotationLayer = annotationRenderer.getAnnotationLayer()

// Now you can interact with the annotation layer, for example:
annotationLayer?.let {
    // Do something with the annotation layer
}

CloseLiveAnnotationButton

CloseLiveAnnotationButton is a custom button provided by the WebexSDK. It is used in conjunction with SurfaceView to create the annotation layer. This button serves to terminate the annotation drawing and remove the annotation layer from the screen. When this button is clicked, it removes the LiveAnnotationLayer. Additionally, it triggers the onAnnotationRenderingStopped() method of LiveAnnotationRenderer.LiveAnnotationRendererCallback, which can be used to stop the annotation session.

Customizing the Close Button

You can customize the CloseLiveAnnotationButton either by altering the look and feel of the default button or by providing a new custom button.

Altering the Look and Feel of the Default Button To change the appearance of the default button, you can use the getCloseButton() method provided by LiveAnnotationLayer. This method returns the current CloseLiveAnnotationButton instance that you can then customize. For example, you can change the button's image, background, size, text and other properties.

val closeButton = renderer.getAnnotationLayer().getCloseButton()

closeButton?.let {
    // Set the button image
    it.setBackgroundResource(R.drawable.custom_close_button_background)
    // Set the button size
    val params = it.layoutParams
    params.width = 100
    params.height = 100
    // Set Button text
    text = "Close"
    // Set other properties as needed

    it.layoutParams = params
}

Providing a New Custom Button To use a completely custom button, you need to create a new button that inherits from the CloseLiveAnnotationButton class. In this new class, you can override the methods and properties to customize the button's behavior and appearance. Once you've created your custom close button, you can use the setCustomCloseButton() method provided by LiveAnnotationLayer to replace the default close button with your custom button.

class CustomCloseButton(context: Context) : CloseLiveAnnotationButton(context) {
    init {
        // Set the button image
        this.setImageResource(R.drawable.custom_close_image)

        // Set the button size
        this.layoutParams = LayoutParams(100, 100)

        // Set other properties as needed

        this.setOnClickListener(OnClickListener {
            // Stop annotation Rendering and reset the data related to annotation
        })
    }

    // Override other methods as needed
}

// Create an instance of your custom close button
val customCloseButton = CustomCloseButton(context)

// Set the custom close button
liveAnnotationlayer.setCustomCloseButton(customCloseButton, layoutParams)

Rendering Annotation Data

When the onLiveAnnotationStarted callback is triggered, you should initialize the LiveAnnotationRenderer.

override fun onLiveAnnotationsStarted() {
    annotationRenderer = LiveAnnotationRenderer(context).apply {
        setAnnotationRendererCallback(object : LiveAnnotationRenderer.LiveAnnotationRendererCallback {
            override fun onAnnotationRenderingReady() {
                // Do Something
            }

            override fun onAnnotationRenderingStopped() {
                // Do something
            }
        })
        startRendering()
    }
}

LiveAnnotationRendererCallback.onAnnotationRenderingReady() confirms the initialization of the annotation layer. Now, the layer can be used to render the data.

In the onLiveAnnotationDataArrived callback, pass the annotation data to the renderData() method of your LiveAnnotationRenderer instance:

override fun onLiveAnnotationDataArrived(data: String) {
    annotationRenderer?.renderData(data)
}

This ensures that the annotation data is rendered on the screen as it arrives.

Note The WebexSDK uses the <uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" /> permission to draw the annotation layer.

Stopping Annotations

To stop annotations, you need to call the stopLiveAnnotations() method.

liveAnnotations?.stopLiveAnnotations()

Note: It's important to stop the annotation explicitly when screen sharing is stopped and the call is disconnected.