Support touch controls when casting to Smart Displays

[baseballbrad3]

Have you read the FAQ and checked for duplicate open issues?
Yes

Is your feature request related to a problem? Please describe.

I would like the touchscreen of a smart display (Nest Hub or Lenovo Smart Display) to interact with the Shaka Cast receiver.

Describe the solution you'd like

Ideally, the touch controls are the same as the standard Cast framework's controls, but, using the Shaka UI would be better than what's available now. The Shaka UI controls could be styled to be a better touch experience, but it would be inconsistent with other receiver apps.

Describe alternatives you've considered

Our alternative would be to not use Shaka as the receiver and switch to CAF, write our own controls (touch events do seem to be passed to the app), or release without touch support for these devices.

Additional context
Our old cast receiver (v2, not Shaka), had touch controls that were provided by the cast framework. We didn't do anything special to add them, we just got them for free by using the v2 receiver. Part of me just expected that to be the same when starting to work on our new Shaka receiver.

[joeyparrish]

I have never had access to a touchscreen-enabled smart display. I would expect it to just work, as the Shaka receiver UI is the same as the UI we use in the browser, and that works well on touchscreen devices.

Without a device, I'm just guessing, but it could be a configuration issue. Our cast receiver app doesn't have all the same UI elements, but it could be configured the same way as in a browser.

Here's where we configure our own receiver app's UI: https://github.com/google/shaka-player/blob/6d373eb6310095c0f605e0e957d418bb4e924925/demo/cast_receiver/receiver_app.js#L44

I will place an order for a device we can use to test this, but it may take some time to have it shipped. So we will rely on you to try some things for us and provide feedback for now.

Are you testing with our receiver app or your own? If it's your own, how is the UI configured?

[baseballbrad3]

I've tried it on the Shaka demo app and our custom Shaka receiver app (which I basically started from the demo app's receiver code). Ours is configured the same as the link you've provided. Both apps lack any UI controls after touching the screen.

If you need me to test anything, I have both the Nest Hub and the Lenovo Smart Display at my desk, and I'm happy to try whatever needs trying.

[ismena]

@joeyparrish Double-checking - is this the issue that requires an upgrade to CAF on our part?

[joeyparrish]

Yes, this is the one I meant. It seems that an upgrade to the Cast SDK is required to get touch screen support. We are using Cast v2, and touch screen receivers require CAF. It's not clear if we can do that upgrade right now, both because of backward compatibility (we aren't ready to ship Shaka Player v4) and because we have an ongoing conversation with the Cast team about the right way to support our Cast API design on top of CAF.

[baseballbrad3]

@joeyparrish I'm not sure if CAF is required for touch support (unless that information is from a Cast team member, in which case they would absolutely know better than I). Our previous Cast receiver was based on Cast v2 and has Cast's built in touch controls. Also, with our new Shaka 3.0 receiver, I can get touch events when listening to the document.addEventListener('touchstart', ...). I was just surprised that the Shaka UI controls were not responding to those touch events.

Shaka's UI controls are really what I was hoping could get addressed here. Ideally, the Cast framework's touch controls are what's used, but I understand if that's not feasible.

[joeyparrish]

Our UI controls already respond to touch events. There's no conditional logic around touch in our codebase, and you can use our UI on any smart phone.

Our receiver demo is a Custom Receiver app, and the Cast V2 SDK does not give us any touch events for some reason. The Cast team told us that had something to do with using the deprecated V2 SDK. It's possible that using a Styled Media Receiver would not be restricted in this way. (Not sure if your previous receiver was a Custom Receiver app or Styled Media Receiver.)

[avelad]

@joeyparrish I'm not sure we want to support this, since we want to do #4214 in the future

[joeyparrish]

You are right, in a sense. Touch control support will only happen via #4214 as far as I can tell.