Fix large pointers from WASM

[kristoff3r]

Pointers being passed from WASM to JS are interpreted as signed, so large pointers ended up negative. This prevented WASM programs from allocating more than 2GB.

To fix it we make pointer use unsigned (via >>> 0) for all malloc/realloc calls, and for all pointer arguments while generating the shim functions. Ideally there would be an abstraction that guarantees we do it in all cases, but that would require a major refactor.

To test it we use gg-alloc as the global allocator while running tests. This allocator was made specifically for testing this issue, but was never upstreamed. It only allocates memory above 2GiB.

Fixes #2388
Fixes #2957

[kristoff3r]

Hmm, looking at the changes to the reference files it might be simpler to remove pop_ptr and just fix it inside getStringFromWasm instead.

[kristoff3r]

Also it looks like the multithreading tests aren't happy about gg-alloc. I tried writing tests for this without using it, but I couldn't figure out how to force allocations above 2GB in the test environment without being extremely slow. If anyone has ideas for that let me know.

[alexcrichton]

Thanks! And yeah I think it's fine to fix without tests for now, allocating 2G in the browser tests is likely going to cause flakiness as well.

[kristoff3r]

I decided to only change stuff in cli-support/src/js/mod.rs, as it seems cleaner (no pointers being fixed multiple times, avoids a half-baked abstraction).

I also made gg-alloc an optional feature for wasm-bindgen-test, and with it the tests now pass, and they fail if I remove the fix.

[emilk]

Thanks for fixing this - we run into this issue problem frequently in our memory intensive application.

Any plans for a new release with this critical fix? 🙏

EDIT we've worked around it for now with out own fork, so no stress from our side.

For others that want this: just use cargo install wasm-bindgen-cli --git https://github.com/rerun-io/wasm-bindgen.git --rev 13283975ddf48c2d90758095e235b28d381c5762

See https://github.com/rerun-io/wasm-bindgen/commits/0.2.84-patch

[Keavon]

@kristoff3r @alexcrichton quick question: does this fix double the available memory from 2GB to 4GB, or does it increase the cap beyond 4GB (to what?)?

As a secondary (slightly off-topic) question, what's the release process like? Once wasm-bindgen is given a release, do we also have to wait for wasm-pack to update to the latest wasm-bindgen, or is wasm-pack's version unrelated to wasm-bindgen's version?

Thanks!

[daxpedda]

Only to 4GB. Beyond 4GB requires wasm64 support.