std: Spin for a global malloc lock on wasm32 #58855

alexcrichton · 2019-03-01T19:48:34Z

There's lots of comments in the code, but the main gist of this commit
is that the acquisition of the global malloc lock on the
wasm32-unknown-unknown target when threads are enabled will not spin
on contention rather than block.

rust-highfive · 2019-03-01T19:48:37Z

r? @Kimundi

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2019-03-01T19:48:41Z

r? @fitzgen

fitzgen

LGTM.

A potential follow up: it would be semantically correct to only have the main thread spin, and others to wait, right? And the main thread can notify as well, it just can't wait.

Therefore, does it make sense to modify this so that:

only the main thread spins, and non-main-threads wait
all threads notify on drop

?

fitzgen · 2019-03-01T20:33:17Z

src/libstd/sys/wasm/alloc.rs

+            //
+            // 2. Maintain a form of "two level" allocator scheme where the main
+            //    thread has its own allocator. Somehow this allocator would
+            //    also be balanced with a global allocator, no only to have


Suggested change

// also be balanced with a global allocator, no only to have

// also be balanced with a global allocator, not only to have

alexcrichton · 2019-03-04T16:29:52Z

@fitzgen yeah agreed that's a better implementation, although I wasn't sure how to actually implement it today. We currently have no way of knowing if a thread is in a "main thread of a browser" context, or if the "main thread" of the wasm app is actually in a web worker.

Do you think it'd be fine to land this in the meantime while we figure out how to make that distinction later?

fitzgen · 2019-03-04T18:00:46Z

Do you think it'd be fine to land this in the meantime while we figure out how to make that distinction later?

Yeah for sure -- maybe add a comment noting this?

alexcrichton · 2019-03-05T15:19:23Z

@bors: r=fitzgen

bors · 2019-03-05T15:19:24Z

📌 Commit fa6085eb92b160b0103ee481f63af1d2ecc0addb has been approved by fitzgen

rust-highfive · 2019-03-05T15:35:37Z

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

travis_time:end:05287590:start=1551799226617194892,finish=1551799298909673325,duration=72292478433
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
Setting environment variables from .travis.yml
---

[00:03:38] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:03:38] tidy error: /checkout/src/libstd/sys/wasm/alloc.rs:127: TODO is deprecated; use FIXME
[00:03:40] some tidy checks failed
[00:03:40] 
[00:03:40] 
[00:03:40] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:03:40] 
[00:03:40] 
[00:03:40] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:03:40] Build completed unsuccessfully in 0:00:43
[00:03:40] Build completed unsuccessfully in 0:00:43
[00:03:40] Makefile:68: recipe for target 'tidy' failed
[00:03:40] make: *** [tidy] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:071b959d
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Tue Mar  5 15:25:28 UTC 2019
---
travis_time:end:28f4d25e:start=1551799529053024826,finish=1551799529057591606,duration=4566780
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1886eff8
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:05fb263e
travis_time:start:05fb263e
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:01e29b60
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

There's lots of comments in the code, but the main gist of this commit is that the acquisition of the global malloc lock on the `wasm32-unknown-unknown` target when threads are enabled will not spin on contention rather than block.

alexcrichton · 2019-03-05T15:42:34Z

@bors: r=fitzgen

bors · 2019-03-05T15:42:36Z

📌 Commit 72958ac has been approved by fitzgen

…oc, r=fitzgen std: Spin for a global malloc lock on wasm32 There's lots of comments in the code, but the main gist of this commit is that the acquisition of the global malloc lock on the `wasm32-unknown-unknown` target when threads are enabled will not spin on contention rather than block.

Rollup of 37 pull requests Successful merges: - #58854 (appveyor: Use VS2017 for all our images) - #58855 (std: Spin for a global malloc lock on wasm32) - #58873 (Fix "Auto-hide item methods documentation" setting) - #58901 (Change `std::fs::copy` to use `copyfile` on MacOS and iOS) - #58933 (Move alloc::prelude::* to alloc::prelude::v1, make alloc a subset of std) - #58938 (core: ensure VaList passes improper_ctypes lint) - #58941 (MIPS: add r6 support) - #58949 (SGX target: Expose thread id function in os module) - #58959 (Add release notes for PR #56243) - #58976 (Default to integrated `rust-lld` linker for UEFI targets) - #59009 (Fix SGX implementations of read/write_vectored.) - #59025 (Fix generic argument lookup for Self) - #59036 (Fix ICE in MIR pretty printing) - #59037 (Avoid some common false positives in intra doc link checking) - #59072 (we can now skip should_panic tests with the libtest harness) - #59079 (add suggestions to invalid macro item error) - #59082 (A few improvements to comments in user-facing crates) - #59102 (Consistent naming for duration_float methods and additional f32 methods) - #59118 (rustc: fix ICE when trait alias has bare Self) - #59139 (Unregress using scalar unions in constants.) - #59146 (Suggest return lifetime when there's only one named lifetime) - #59147 (Make std time tests more robust for platform differences) - #59152 (Stabilize Range*::contains.) - #59156 ([wg-async-await] Add regression test for #55809.) - #59158 (Revert "Don't generate minification variable if minification disabled") - #59169 (Add `-Z allow_features=...` flag) - #59173 (bootstrap: Default to a sensible llvm-suffix.) - #59175 (Don't run test launching `echo` since that doesn't exist on Windows) - #59180 (Use try blocks in rustc_codegen_ssa) - #59185 (No old chestnuts in iter::repeat docs) - #59201 (Remove restriction on isize/usize in repr(simd)) - #59204 (Output diagnostic information for rustdoc) - #59206 (Improved test output) - #59208 (Reduce a Code Repetition Related to Bit Operation) - #59212 (Add x86_64 musl host to the manifest) - #59221 (Option and Result: Add references to documentation of as_ref and as_mut) - #59231 (Stabilize Option::copied)

rust-highfive assigned Kimundi Mar 1, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 1, 2019

fitzgen approved these changes Mar 1, 2019

View reviewed changes

alexcrichton force-pushed the wasm-multithreaded-alloc branch from 30d9eac to fa6085e Compare March 5, 2019 15:19

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 5, 2019

std: Spin for a global malloc lock on wasm32

72958ac

There's lots of comments in the code, but the main gist of this commit is that the acquisition of the global malloc lock on the `wasm32-unknown-unknown` target when threads are enabled will not spin on contention rather than block.

alexcrichton force-pushed the wasm-multithreaded-alloc branch from fa6085e to 72958ac Compare March 5, 2019 15:42

Centril mentioned this pull request Mar 10, 2019

Rollup of 16 pull requests #59066

Closed

Centril mentioned this pull request Mar 10, 2019

Rollup of 17 pull requests #59067

Closed

Centril mentioned this pull request Mar 10, 2019

Rollup of 16 pull requests #59069

Closed

Centril mentioned this pull request Mar 10, 2019

Rollup of 17 pull requests #59075

Closed

Centril mentioned this pull request Mar 11, 2019

Rollup of 15 pull requests #59081

Closed

kennytm mentioned this pull request Mar 11, 2019

Rollup of 23 pull requests #59108

Closed

alexcrichton mentioned this pull request Mar 14, 2019

Use rayon for the raytrace-parallel example rustwasm/wasm-bindgen#1349

Closed

kennytm mentioned this pull request Mar 15, 2019

Rollup of 31 pull requests #59207

Closed

kennytm mentioned this pull request Mar 16, 2019

Rollup of 37 pull requests #59226

Merged

bors merged commit 72958ac into rust-lang:master Mar 16, 2019

alexcrichton deleted the wasm-multithreaded-alloc branch March 19, 2019 19:54

alexcrichton mentioned this pull request Mar 21, 2019

Add the ability to customize thread spawning rayon-rs/rayon#636

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

std: Spin for a global malloc lock on wasm32 #58855

std: Spin for a global malloc lock on wasm32 #58855

alexcrichton commented Mar 1, 2019

rust-highfive commented Mar 1, 2019

alexcrichton commented Mar 1, 2019

fitzgen left a comment

fitzgen Mar 1, 2019

alexcrichton commented Mar 4, 2019

fitzgen commented Mar 4, 2019

alexcrichton commented Mar 5, 2019

bors commented Mar 5, 2019

rust-highfive commented Mar 5, 2019

alexcrichton commented Mar 5, 2019

bors commented Mar 5, 2019

	// also be balanced with a global allocator, no only to have
	// also be balanced with a global allocator, not only to have

std: Spin for a global malloc lock on wasm32 #58855

std: Spin for a global malloc lock on wasm32 #58855

Conversation

alexcrichton commented Mar 1, 2019

rust-highfive commented Mar 1, 2019

alexcrichton commented Mar 1, 2019

fitzgen left a comment

Choose a reason for hiding this comment

fitzgen Mar 1, 2019

Choose a reason for hiding this comment

alexcrichton commented Mar 4, 2019

fitzgen commented Mar 4, 2019

alexcrichton commented Mar 5, 2019

bors commented Mar 5, 2019

rust-highfive commented Mar 5, 2019

alexcrichton commented Mar 5, 2019

bors commented Mar 5, 2019