HLS AES-128 Live mode not working on Chromecast

[keyaavoja]

Have you read the FAQ and checked for duplicate open issues?
Yes

What version of Shaka Player are you using?
Demo version on the Shaka official Demo page.

Can you reproduce the issue with our latest release version?
Yes

Can you reproduce the issue with the latest code from main?
Yes

Are you using the demo app or your own custom app?
Demo app.

If custom app, can you reproduce the issue using our demo app?
Yes

What browser and OS are you using?
Windows, Chrome

For embedded devices (smart TVs, etc.), what model and firmware version are you using?
Google TV 4K, FW: STTE.230319.008.R1

What are the manifest and license server URIs?
Any Live HLS using AES-128 encryption will fail using Chromecast.

What configuration are you using? What is the output of player.getConfiguration()?
Demo app.

What did you do?
Just try to play HLS AES-128 encrypted Live stream using Chromecast..

What did you expect to happen?
I assumed, it would work.

What actually happened?
Plays only one chunk and then it stops.

[avelad]

@keyaavoja can you share a HLS-AES-128 live stream to test it? Thanks!

[keyaavoja]

Trying to find working stream url.

[Andekas]

@avelad I spoke with @keyaavoja and he told that you couldn't reproduce the problem in main.

I tested it with https://router.euddn.net/95b879fc5407bb2fc4ae3a1c74d69466/smil:live1.smil/playlist.m3u8?c=8000 both in demo app and main branch and in both cases, video stops after first chunk on Chromecast.

I used out-of-the box demo apps, without any custom config. Everything works well if we turn off HLS-AES-128 on the same stream.

[avelad]

What Chromecast device are you using? I used Chromecast 1st gen to test it.

[Andekas]

I used 4th gen, Chromecast with Google TV.

[avelad]

@theodab can you review it?

[Andekas]

Tested with gen 2 also, doesn't work with that either.

[joeyparrish]

I tried to reproduce your issue, but I get a 404 error for the content.

[joeyparrish]

I'm trying now with https://storage.googleapis.com/shaka-demo-assets/sintel-fmp4-aes/master.m3u8, which is public and which we own.

[joeyparrish]

The AES-128 HLS content I have at hand plays fine on both Chrome and on a Chromecast Ultra in our lab.

There was a recent release of the Cast Application Framework (CAF) that updates the default Shaka Player version to v4.3.4. The latest release of Shaka Player is v4.3.7. By default, CAF does not use Shaka Player for HLS, but you can opt-in by following the directions in this document: https://developers.google.com/cast/docs/web_receiver/shaka_migration?hl=en

Does this help?

[Andekas]

@joeyparrish Sorry, the stream ( https://router.euddn.net/95b879fc5407bb2fc4ae3a1c74d69466/smil:live1.smil/playlist.m3u8?c=8000 ) has been restarted and works again.

The problem occurs with live streams; https://storage.googleapis.com/shaka-demo-assets/sintel-fmp4-aes/master.m3u8 is VOD. This works for me also.

To clarify:

• If live stream has AES-128 it plays only first chunk in Chromecast
• If we remove encryption from the same stream, everything works
• VOD content with AES-128 encryption works in Chromecast

[teqm]

@joeyparrish Sorry, the stream ( https://router.euddn.net/95b879fc5407bb2fc4ae3a1c74d69466/smil:live1.smil/playlist.m3u8?c=8000 ) has been restarted and works again.

The problem occurs with live streams; https://storage.googleapis.com/shaka-demo-assets/sintel-fmp4-aes/master.m3u8 is VOD. This works for me also.

To clarify:

• If live stream has AES-128 it plays only first chunk in Chromecast
• If we remove encryption from the same stream, everything works
• VOD content with AES-128 encryption works in Chromecast

☝️ we're experiencing the same issue (with useShakaForHls enabled), @joeyparrish do you need us to provide a stream to reproduce or is the one provided by @Andekas enough?

[avelad]

Can you check if the problem persists with 4.7.8? Thanks!

[Andekas]

@avelad Tested with 4.7.8, problem still persists. Connects and plays first chunk, then stops.

Using debug player, I get following error on console:

log.js:140 Uncaught exception in event handler TypeError: Cannot read properties of undefined (reading 'length')

[joeyparrish]

I can still access the live stream from @Andekas, so I will take a look now.

[joeyparrish]

I tested this in our lab with build/test.py --test-custom-asset 'https://router.euddn.net/95b879fc5407bb2fc4ae3a1c74d69466/smil:live1.smil/playlist.m3u8?c=8000' --browsers ChromecastUltra

My results:

Chrome 90.0.4430.225 (Chromecast 1.56.500000) Player plays Custom / custom : https://router.euddn.net/95b879fc5407bb2fc4ae3a1c74d69466/smil:live1.smil/playlist.m3u8?c=8000 FAILED
	Error: Expected true to be false.
	    at <Jasmine>
	    at _callee4$ (test/player_external.js:131:35 <- test/player_external.js:171:43)
	    at tryCatch (node_modules/@babel/polyfill/dist/polyfill.js:6473:40)
	    at Generator.invoke [as _invoke] (node_modules/@babel/polyfill/dist/polyfill.js:6702:22)
	Failed: Shaka Error 3014
	error properties: shaka.util.Error {
	  "severity": 2,
	  "category": 3,
	  "code": 3014,
	  "data": [
	    0
	  ],
	  "handled": false
	}
	Error: Shaka Error 3014
	    at new P (dist/shaka-player.ui.js:77:311)
	    at SourceBuffer.eval (dist/shaka-player.ui.js:359:100)
	Error: Expected 12.8299 to be greater than 20.
	    at <Jasmine>
	    at _callee4$ (test/player_external.js:148:37 <- test/player_external.js:193:43)
	    at tryCatch (node_modules/@babel/polyfill/dist/polyfill.js:6473:40)
	    at Generator.invoke [as _invoke] (node_modules/@babel/polyfill/dist/polyfill.js:6702:22)

---

Expected true to be false is the following line of the test: expect(player.isLive()).toBe(isLive);

This is bogus. Somewhere along the way, someone changed ShakaDemoAssetInfo to initialize itself with this.features = [shakaAssets.Feature.VOD]. The test only compares player.isLive() to an expectation if asset.features is present. It shouldn't be, since this is a custom asset test. But in any case, we can see that player.isLive() is true, so this is detected as a live stream.

---

Error: Shaka Error 3014 is a the error MEDIA_SOURCE_OPERATION_FAILED. This is a fatal error at the MSE level.

---

Error: Expected 12.8299 to be greater than 20 is an expectation that we can play through the first 20 seconds of the live stream at the point where we joined. It is understandable that this fails, because the fatal MSE error would keep us from reaching an arbitrary milestone like this.

---

Going back to error 3014, running the test again with --uncompiled --enable-logging debug shows ERROR LOG: '(video:11)', 'failed fetch and append: code=3014'

We can assume that the segment was either not decrypted or was decrypted with the wrong key. I'll add some more debug logging to find out which.

Comparing this against --browsers ChromeLinux, we see that there is no MSE error there.

A snippet of the log from Chromecast:

LOG LOG: 'event canplay after 7.15 seconds'
LOG LOG: 'Waiting for movement from 6.149999 to 7.149999'
INFO LOG: 'Jumping forward 5.436000965667724 seconds to catch up with the seek range.'
LOG LOG: '(all): seeked: buffered seek: presentationTime=11.60099983215332'
INFO LOG: 'Updating manifest...'
LOG LOG: 'movement from 6.149999 to 7.149999 after 1.39 seconds'
LOG LOG: 'Waiting for end of media'
ERROR LOG: '(video:11)', 'failed fetch and append: code=3014'

The same section from ChromeLinux:

LOG LOG: 'event canplay after 3.30 seconds'
LOG LOG: 'Waiting for movement from 0.000735 to 1.000735'
LOG LOG: 'movement from 0.000735 to 1.000735 after 1.05 seconds'
LOG LOG: 'Waiting for end of media'

On Chromecast, we see that there is a seek because it takes so long to start playback. This could indicate that the trigger isn't Chromecast at all, but seeking.

[joeyparrish]

I made a quick hack in the test to avoid the seek on Chromecast:

player.configure('streaming.bufferBehind', 180);
player.configure('manifest.availabilityWindowOverride', 600); 

Because the content should stay available, there is no need to catch up to the seek range. But I still get the same failure without the seek:

LOG LOG: 'event canplay after 6.92 seconds'
LOG LOG: 'Waiting for movement from 6.121 to 7.121'
LOG LOG: 'movement from 6.121 to 7.121 after 1.17 seconds'
LOG LOG: 'Waiting for end of media'
INFO LOG: 'Updating manifest...'
INFO LOG: 'Updating manifest...'
WARN LOG: 'No variants met the ABR restrictions. Choosing a variant by lowest bandwidth.'
LOG LOG: 'Calling switch_(), bandwidth=2461 kbps'
LOG LOG: 'switch_'
ERROR LOG: '(video:11)', 'failed fetch and append: code=3014'

[joeyparrish]

With a ton of extra logging, I can see that on both ChromeLinux and ChromecastUltra, we are fetching the same URIs and getting the same key. We're using the same cipher block mode, as well.

I decided I wanted to log the segment data, both before and after decryption, on both devices for comparison. If there was a problem with decryption, I wanted to see it.

What I found was that this was impossible with the live stream, since the segments kept changing. I couldn't get the two test runs to happen on the same segments. So I captured a snapshot of the live stream and hosted it myself: master playlist, the lowest-resolution media playlist, and the key.

Using that content, however, I found that I couldn't reproduce the error.

---

Next I decided to test if the decryption is working correctly. I dumped full segments, their keys, their sequence numbers, and their decrypted versions. I then separately decrypted each segment using OpenSSL from the command-line to compare results. All decryptions matched. The contents are also parseable as TS segments in https://thumb.co.il/

Of note is the fact that the segment that kills playback on Chromecast is not structured like the others. It has a pat with a pmtPid of 2303, no pmt, no video, and no audio.

---

Finally I noticed that there is a big difference between Chromecast and Chrome. Chromecast devices "support" TS content directly, so we are not forced to transmux the content.

Rejecting TS content for Chromecast forces transmuxing and fixes playback.