Video/audio node plays audio when preloading

[Sacharified]

When constructing an audio or video node and using the preloadTime parameter, the media begins playing in the background in order to load content. If the media has audio, this audio will begin playing out loud immediately (or presumably when the context's currentTime reaches the media's sequenced startTime - preloadTime).

This effectively makes preloading useless if you are preloading assets with audio.

You can reproduce the issue with the following code:

var canvas = document.getElementById("canvas");
var vc = new VideoContext(canvas);

var videoNode1 = vc.video("../../assets/introductions-rant.mp4", 23, 20);
videoNode1.connect(vc.destination);
videoNode1.start(10);
videoNode1.stop(20);

vc.play();

You are be able to hear audio before 10 seconds, even though there is no sequenced audio for that time.

[PTaylour]

Thanks @Sacharified that certainly doesn't look like intended behaviour.

We're almost ready to merge the regression testing. Can jump on this as soon as that's in.

Pete

[Sacharified]

I've tracked this issue down to this function: https://github.com/bbc/VideoContext/blob/master/src/videoelementcache.js#L28

The first time you call play() on the VideoContext, it will call this init() function on the Video element cache. This loops over all video elements in the cache and calls play() on each of them, which is why you hear the audio playing before it's scheduled to play.

This also impacts videos, causing the elements to start playing at 0. When the context's currentTime reaches the start of the video, it will paint whatever the video element is now displaying, which will be out of sync unless that video starts at 0.

I'm not sure what the purpose of this init function is but this seems like a pretty critical flaw.

Also it's quite difficult to reproduce this in the code playground because the whole thing is reconstructed when you click the play button. If you load the example in my first post and call vc.play() in the console, you will see the described behaviour, however you won't see it if you click the play button.

[tinybug]

@Sacharified @PTaylour
it cause by this commit: 5137bee, revert to pre commit it will work fine.

[PTaylour]

If I'm remembering correctly the init function is there to prime all the elements in the cache on user interaction, so that we can play them programatically later (without further user interaction on mobile).

There is a wider question here about how to support mobile without complicating the core videocontext library.

As an interim solution it seems sensible to revert 5137bee, as you suggest @tinybug

Would seem likely that this would reintroduce #54

"On mobile [a] pause sometimes happen straight after playing"

It could be that the init function needs to more careful and only init elements that aren't immediately in use.

[yakca]

Confirmed that this is fixed in 0.53.1 - closing.