Audio stuttering on HTML5 export

[FernandoValcazara]

Godot version:
Godot 3.2.2 Stable & 3.2.3 rc1 & 3.2.3 rc2

OS/device including version:
GLES2 / Chrome Browser

Tested on:
LG k11+ - Android (Outdated Mobile)
LG K10 - Android (Outdated Mobile)
AMD C-50 Processor 1.00 GHZ - 2 GB RAM - Windows 7 (Outdated Computer)

Issue description:
We have start receiving feedback from customers, regarding stuttering audios, it does happens more often in outdated computers, and even more often in outdated mobile devices.
But our build versions in another engines seams to be working properly on the same devices, and only the Godot builds are stuttering.
We thought Godot 3.2.3 rc1 would come with a solution for this stuttering with the update in output latency for web, but it didn't do the trick for us.

Perhaps it is correlated with the drop in FPS, since we stick to use GLES2, we are still using GLES2 because of its compatibility with older mobile devices.
Is it possible the stuttering is happening because of WebGL 1.0? If so, is it a limitation of the technology version?

Steps to reproduce:
N/A

Minimal reproduction project:
AudioTest.zip

[Calinou]

Related to #16259. As a workaround, increase the audio buffer length. (There's no ideal audio buffer length; it depends on the client's CPU power. This is why some games let the player adjust the audio buffer length.)

Is it possible the stuttering is happening because of WebGL 1.0? If so, is it a limitation of the technology version?

Audio stuttering is unrelated to graphics rendering. That said, since WebAssembly threads aren't supported in all browsers yet, Godot doesn't make use of them. This limits Godot's ability to process audio in a separate thread, which can provide lower latency with better reliability.

[FernandoValcazara]

Thanks for the quick response @Calinou,

Roger that, we checked the issue #16259, and considered the idea of ​​releasing the Output Latency as an inside option to the user inside the games, so they can set the best buffer length for their devices.

But, we tested the latency with milliseconds from 0 to 1000 and we didn't get the sound we expected, in other engines we tested, we got better results, perhaps these engines switches automatically to WebAssembly or have WebAssembly as default.
We do understand that WebAssembly is not a feature available in all browsers, but it seems to be present in some major browsers, such as Google Chrome, this support would be an life savior for us.

Is there a possibility of this feature be implemented?
Or perhaps any other suggestions of how we can fix this situation in our builds?

[Calinou]

perhaps these engines switches automatically to WebAssembly or have WebAssembly as default.

Since Godot 3.0, Godot can only use WebAssembly, not asm.js. Therefore, it always uses WebAssembly 🙂

We do understand that WebAssembly is not a feature available in all browsers

WebAssembly is available in all evergreen browsers (latest versions of Firefox, Chrome, Edge, Opera and Safari). What's not available in all evergreen browsers is WebAssembly threads (which depends on SharedArrayBuffer). Browsers are slowly adding support for it, but there are often security in place that can make widespread deployment more difficult. For instance, most browsers will require you to send additional HTTP response headers to be able to use WebAssembly threads.

[Faless]

considered the idea of ​​releasing the Output Latency as an inside option to the user inside the games

That is sadly not currently possible, the latency need to be defined in the project settings. It cannot be changed while the game is running.

[FernandoValcazara]

Understood, and thank you for the replies guys,

So perhaps, the correct approach would be to at least add a flag that enables the WebAssembly thread?

If so, we added the following flags:
scons platform=javascript target=release_debug -j10 threads_enabled=yes
And enabled the WebAssembly options on Chrome (chrome://flags)

But we didn't see any audio improvement, and on some devices we even got a black screen.

Any suggestions of something to send us in the correct path to fix it?

[thomazette]

I've seen at issue #16256 that @Faless said that futher improvement might come with the implementation of "Audio mixing threads", I belive this would help.
Is there any prevision for when something like this would be implemented?

[Faless]

@FernandoValcazara

scons platform=javascript target=release_debug -j10 threads_enabled=yes
And enabled the WebAssembly options on Chrome (chrome://flags)

But we didn't see any audio improvement, and on some devices we even got a black screen.

Using threads for audio mixing is sadly not the same thing as enabling thread support in WASM exports. It has its own API: https://developer.mozilla.org/en-US/docs/Web/API/AudioWorklet and its own limitations in terms of browser compatibility, requirements.

@thomazette

Is there any prevision for when something like this would be implemented?

Hopefully in master in the coming months.

[thomazette]

Thanks for the reply @Faless,
Researching for a solution, I came across the Howler.js project, which has in last past eyers implementing solutions to prevent the instability of web audios by moving the processing of audios for separeted threads similar to webworkers.
Was the possibility of implementing such Web APIs took in consideration?
Howler.js is well known and efficient in this solution.

[Faless]

Was the possibility of implementing such Web APIs took in consideration?

If you mean using an external library no, we plan to use the Web Audio API directly.

[VictorHASilva]

Hey guys i have the same problem with audio stuttering on i3 laptop's and medium phones.
@falles I decided to give a try to the AudioWorklet API mentioned above.
Working with it directly, I thought it would automaticatly create a separated thread after it connected with the Audio.
I think it worked. But when I test it in older devices everytime I move my cursor the Audio starts to stutter again.
Working example: https://jogosdev.afinando.com.br//jogos/Tools/Worklet/index.html
Perhaps I'm missing something?

[Faless]

@VictorHASilva I haven't studied the API enough, but it seems you are doing nothing in process.
You would need to have the AudioDriverJavaScript::mix_to_js function implemented there, and replace the ScriptProcessor in the audio driver with the AudioWorklerProcessor.

[VictorHASilva]

Hello everyone, I decided to give the Audio Worklet a try, and I did manage to make it run without error, and I can see it is running through out the breakpoints.But . . . the only thing is, I can’t seem to find where the Audio Streams connects to the context inside the audio_driver_javascript.

@falles, do you know if this connection perhaps is happening in another script file? Other than the audio_driver_javascript?
I even tried to force ref[“tream”] in the resume function, but with no success.

[RafaRossi]

I had the same problem as you, I identified this issue that had solved the problem in Godot 3.2.1, Issue #35937
I went back to Godot 3.2.2 which has some crucial updates for me, and reapplied the BufferSizer calculation from Issue #35937, but this time with 4000 of OutPut Latency for Mobile and Desktop, and the stutter was minimized for practically all my users. The stutter was resolved for more than 90% of them.
However, the error still exists even if minimized. Usually the stutter happens when the user moves its mouse.
While we wait for the implementation of the new API, is there any suggestion of what else I can do?
Especially in the matter of the mouse still stuttering the audio.

[FernandoValcazara]

We developed a Plugin that may fix this issue. It is not a perfect solution, but it works using any kinda of external Web JS Audio Plugin.

We used Howler.js as external library, but you may use any JS Audio Library you wish. Since it runs using JS, and it does not suffer stuttering, and as far I can tell it does not interfere with the FPS performance.

https://godotengine.org/asset-library/asset/745

We appreciate any feedback regarding the plugin.