Cache wasm stacks in thread-local storage to avoid a mutex.

[rlane]

I noticed that when running many concurrent WASM function calls on a 48-core VM the mutex protecting the stack freelist was contended. Replacing it with thread-local storage reduced CPU usage by 75% in a benchmark (20% on my 14-core laptop) and didn't affect single-threaded performance.

[syrusakbary]

This PR looks great to me. Waiting for @ptitSeb or @theduke feedback as well

[ptitSeb]

It looks good to me. Thread local instead of a mutex, why not!

[rlane]

Anything I can help with to get this merged?

[theduke]

@rlane we haven't forgotten about this, we've just been discussing internally if this is the right way to go for all situations, or if we want an additional layered cache tier , with a thread local as the first layer and an additional shared cache.

Or if we need a feature flag to toggle this, because there are scenarios were keeping a stack around in each thread could be suboptimal for memory consumption, like in a scenario with a large thread pool where threads only run occasionally.

We are a bit swamped this week, we'll get back to this next week at the latest.

Would be we be fine with just putting the thread local behind a feature flag to punt on the larger discussion for now, @syrusakbary / @ptitSeb ?

[Arshia001]

@rlane #4196 was just merged with an alternate implementation. Closing this issue.

[rlane]

The crossbeam alternative has some performance issues on large machines (224 cores). Here's what I see in perf top:

  61.24%  tournament         [.] crossbeam_queue::seg_queue::SegQueue<T>::push
   9.50%  tournament         [.] crossbeam_queue::seg_queue::SegQueue<T>::pop
... rest of the profile