Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious segfault in tests for musl #38618

Closed
alexcrichton opened this issue Dec 26, 2016 · 12 comments
Closed

Spurious segfault in tests for musl #38618

alexcrichton opened this issue Dec 26, 2016 · 12 comments
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-bug Category: This is a bug. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics.

Comments

@alexcrichton
Copy link
Member

First seen on Travis, the error looks like:

     Running build/x86_64-unknown-linux-gnu/stage2-std/x86_64-unknown-linux-musl/release/deps/coretest-a84303f8bbcf2658
running 607 tests
..........................................................................................................................................................................................................................................................................................................................ii....................................................................................................................................................................................................................................................error: process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/x86_64-unknown-linux-musl/release/deps/coretest-a84303f8bbcf2658 --quiet` (signal: 11, SIGSEGV: invalid memory reference)
To learn more, run the command again with --verbose.
command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "-j" "4" "--target" "x86_64-unknown-linux-musl" "--release" "--frozen" "--manifest-path" "/checkout/src/rustc/std_shim/Cargo.toml" "--features" "panic-unwind jemalloc backtrace" "-p" "std_shim" "-p" "core" "-p" "std" "-p" "libc" "-p" "compiler_builtins" "-p" "unwind" "-p" "build_helper" "-p" "std_unicode" "-p" "rand" "-p" "alloc" "-p" "alloc_system" "-p" "collections" "-p" "panic_abort" "--" "--quiet"

cleaned up as:

error: process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/x86_64-unknown-linux-musl/release/deps/coretest-a84303f8bbcf2658 --quiet` (signal: 11, SIGSEGV: invalid memory reference)

This means that the test harness itself segfaulted. Sounds quite suspicious!

@alexcrichton alexcrichton added the A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) label Dec 26, 2016
@brson brson added the I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. label Dec 28, 2016
@est31
Copy link
Member

est31 commented Dec 28, 2016

maybe the actual failing test has nothing to do with the actual issue, but counting the dots, the test that's failing is number 560, which is test ptr::test_as_mut.

@est31
Copy link
Member

est31 commented Dec 28, 2016

err, that was the last one that succeeded, the one which would come next is ptr::test_as_ref

@alexcrichton
Copy link
Member Author

@est31 sounds plausible! Note though that tests are run in parallel so I don't think we can know for certain which test failed, unfortunately :(

I'll also note that I tried running this locally thousands of times, but none of them segfaulted :(

@alexcrichton
Copy link
Member Author

Looks like this just happened in collectionstest as well. Still musl specific.

@alexcrichton alexcrichton changed the title Spurious segfault in coretest Spurious segfault in tests for musl Dec 29, 2016
@alexcrichton
Copy link
Member Author

After running all of these test suites locally thousands of times I'm still unable to reproduce. I'm not entirely sure what's going on here..

@alexcrichton
Copy link
Member Author

Debugging this a bit, I took the two most recent failures and diffed the set of tests which completed successfully from the previous successful run. The two diffs I got were:

I was hoping that perhaps one test was spuriously segfaulting, but that does not appear to be the case.

alexcrichton added a commit to alexcrichton/rust that referenced this issue Apr 5, 2017
This is a random stab towards rust-lang#38618, no idea if it'll work. But hey more
up-to-date software is better, right?
frewsxcv added a commit to frewsxcv/rust that referenced this issue Apr 6, 2017
travis: Update musl for i686/x86_64

This is a random stab towards rust-lang#38618, no idea if it'll work. But hey more
up-to-date software is better, right?
frewsxcv added a commit to frewsxcv/rust that referenced this issue Apr 6, 2017
travis: Update musl for i686/x86_64

This is a random stab towards rust-lang#38618, no idea if it'll work. But hey more
up-to-date software is better, right?
frewsxcv added a commit to frewsxcv/rust that referenced this issue Apr 6, 2017
travis: Update musl for i686/x86_64

This is a random stab towards rust-lang#38618, no idea if it'll work. But hey more
up-to-date software is better, right?
@alexcrichton
Copy link
Member Author

Musl hasn't been updated yet, but I've heard that it's going to be fixed before the next release, although it's unclear when the next release will be.

@Mark-Simulacrum Mark-Simulacrum added the C-bug Category: This is a bug. label Jul 26, 2017
This was referenced Oct 2, 2017
alexcrichton added a commit to alexcrichton/rust that referenced this issue Oct 19, 2017
Apparently there's at least one CVE fixed in the new version of musl, and
because we're distributing it seems like a good opportunity to update!

Unfortunately it looks like rust-lang#38618 still hasn't been fixed.
bors added a commit that referenced this issue Oct 21, 2017
ci: Update musl with new release

Apparently there's at least one CVE fixed in the new version of musl, and
because we're distributing it seems like a good opportunity to update!

Unfortunately it looks like #38618 still hasn't been fixed.
@alexcrichton
Copy link
Member Author

This hasn't shown up in quite some time now so perhaps it was indeed fixed with the last musl update!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-bug Category: This is a bug. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics.
Projects
None yet
Development

No branches or pull requests

4 participants