Changing codecs causes playback to stop

[daveisfera]

Description

Safari and HLS.js will playback video when the codec switches, but Video.js shows an error.

Reduced test case

https://raw.githubusercontent.com/daveisfera/hls_switch/1e04cd07f7fc3077912418584ae3e7b495b224d6/primary_codec_up.m3u8

Steps to reproduce

1. Load https://raw.githubusercontent.com/daveisfera/hls_switch/1e04cd07f7fc3077912418584ae3e7b495b224d6/primary_codec_up.m3u8
2. Press play
3. Observe error

Another example with switching from H.265 to H.264:
https://raw.githubusercontent.com/daveisfera/hls_switch/1e04cd07f7fc3077912418584ae3e7b495b224d6/primary_codec_down.m3u8

NOTE: Safari won't play directly from GitHub because it loads the actual file contents, but if you serve the files in that repo from a web server, then it will show that Safari handles this without issue

Errors

VIDEOJS: ERROR: (CODE:3 MEDIA_ERR_DECODE) Playback cannot continue. No available working or supported playlists.
VIDEOJS: ERROR: (CODE:3 MEDIA_ERR_DECODE) video append of 483435b failed for segment videojs/video.js#5 in playlist 0-video_codec_up.m3u8
VIDEOJS: ERROR: (CODE:3 MEDIA_ERR_DECODE) The media playback was aborted due to a corruption problem or because the media used features your browser did not support.

What version of Video.js are you using?

8.17.4

Video.js plugins used.

N/A

What browser(s) including version(s) does this occur with?

Chrome 129.0.6668.72 and Safari 18.0

What OS(es) and version(s) does this occur with?

macOS 15.0

[adrums86]

Hi @daveisfera it looks like a codec and container switch is happening without specifying all the codecs within the CODECS attribute of the EXT-X-STREAM-INF tag, without this the player doesn't know it needs to call changeType on the source buffer. See the HLS spec:

   any EXT-X-STREAM-INF tag ([Section 4.3.4.2](https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.4.2)) or EXT-X-I-FRAME-STREAM-
   INF tag ([Section 4.3.4.3](https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.4.3)) that references such a Group MUST have a
   CODECS attribute that lists every sample format present in any
   Rendition in the Group, or client playback failures can occur.

I think a next step would be to put both codecs in that attribute and see if that allows for a successful change.

[adrums86]

Moved this to VHS as there seems to be some room for improvement on codec detection, as this media works in Safari and hls.js

[daveisfera]

Hi @daveisfera it looks like a codec and container switch is happening without specifying all the codecs within the CODECS attribute of the EXT-X-STREAM-INF tag, without this the player doesn't know it needs to call changeType on the source buffer. See the HLS spec:

   any EXT-X-STREAM-INF tag ([Section 4.3.4.2](https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.4.2)) or EXT-X-I-FRAME-STREAM-
   INF tag ([Section 4.3.4.3](https://datatracker.ietf.org/doc/html/rfc8216#section-4.3.4.3)) that references such a Group MUST have a
   CODECS attribute that lists every sample format present in any
   Rendition in the Group, or client playback failures can occur.

I think a next step would be to put both codecs in that attribute and see if that allows for a successful change.

Sorry, I made those examples as well and meant to post them here, but forgot to get back to that. For one of them, it doesn't play at all and the other it still has the same problem:

https://raw.githubusercontent.com/daveisfera/hls_switch/784ca7d2a4eff321c2ae3b1c67d50704ef466d42/primary_codec_down_both.m3u8
https://raw.githubusercontent.com/daveisfera/hls_switch/784ca7d2a4eff321c2ae3b1c67d50704ef466d42/primary_codec_up_both.m3u8

[adrums86]

Gotcha, thanks for the follow up. I was hoping this might help allow for a changeType call, but there must be something else going wrong here. I'll do some debugging and share what I find here as soon as I'm able.

[daveisfera]

I traced the use of the CODECS info to this point, where it appears to be discarded and then probed later. This feels like "a problem" but not "the problem" and I wasn't sure where to start the investigation from there. I'm definitely willing to take a look at making a fix, but right now I'm not sure where to look in the code for where the codec handling is incorrectly erroring because of the switch.