Wasmer 4 High CPU Usage Crashing PCs

[gluax]

Describe the bug

We are upgrading to wasmer version 4 from wasmer version 3 as a rust dependency. But when we do so the CPU difference is massive, enough to crash computers.

I'm running on windows 10. However, we've noticed the same problem on MacOS as well.

wasmer -vV; rustc -vV

wasmer 4.2.3 (7cb550d 2023-10-27)
binary: wasmer-cli
commit-hash: 7cb550db60656352cc527eaf869981d39448e38d
commit-date: 2023-10-27
host: x86_64-pc-windows-msvc
compiler: singlepass,cranelift,llvm
rustc 1.74.0 (79e9716c9 2023-11-13)
binary: rustc
commit-hash: 79e9716c980570bfd1f666e3b16ac583f0168962
commit-date: 2023-11-13
host: x86_64-pc-windows-msvc
release: 1.74.0
LLVM version: 17.0.4

Steps to reproduce

We built a minimal reproduction here.
You can build and run the same program, but with different wasmer versions.

Note: our use case is spinning up wasm VMs across threads for parallel execution.

Expected behavior

I would not expect wasmer 4 to use nearly as much CPU usage as it does in the same use case we used wasmer 3.

Actual behavior

Here's the CPU usage of wasmer3.

I'm trying to get a screenshot of wasmer4 CPU usage... but its crashing my PC before I can even get one ☠️.
I will update the issue if I can manage to get one.

Windows doesn't show CPU usage over 100%, but my coworkers on Macos have stated a CPU percentage of 155~%.

Note, this requires us to hard kill the program now, as ctrl-c(I don't remember what signal this sends off the top of my head) is unresponsive. It needs a SIGKILL or for Windows task manager since the terminal app itself becomes unresponsive.

Note This problem doesn't occur until each wasmer after compiling our example wasm. I don't mean building the wasm from rust, but when wasmer compiles the wasm module. Those both use about 2.5% CPU usage on my machine.

Additional context

Does this have to do with the internals of wasmer4 now using Rayon? I didn't go super deep, but it looked like it was spinning up its thread pool now. If that's true... it seems to break the use case of wanting to spin up running multiple wasms on different threads.

[theduke]

@gluax we are currently investigating some related performance issues.

But an important bit of context if you want to run multiple instances is to share the Runtime between the instances, which means they will share a common thread pool.

You will want to manually construct a PluggableRuntime, and then use it with WasiEnvBuilder::runtime(my_rt).

[gluax]

@theduke we have done that approach as well. But the result is the same with the cpu maxing out and crashing.

[maminrayej]

@gluax Can you confirm the CPU usage is reduced on the master branch in your use case?

I can see the improvement on my local machine using your minimal reproduction example.

[gluax]

@maminrayej Can confirm on both windows and macos, the CPU usage is now less than the old wasmer3 version actually when using that branch!

[maminrayej]

Excellent.

@theduke I'm closing this issue.