Fuzzing: Cannot compile code using compiler plugin with -C panic=abort

[PaulGrandperrin]

This bug is probably technically equivalent to those two closed issues #2738 and #3166 but I want to present the issue from another perspective.

At rust-fuzz, we use this feature to transform panics into something that the fuzzer can interpret as an abnormal process termination.
Indeed by default, fuzzers do not interpret writing on stderr + returning a non-zero exit code as an abnormal process termination.
So it means that we really rely on this behavior for our fuzzers to work nominally.
If this behavior is ignored when linking to compiler crates (as suggested in #3166 (comment)) then, our fuzzers will silently miss all bugs triggering panics.

That being said, for instance, right now it's impossible to build with panic=abort a project linking to proc_macro2:

cargo fuzz run fuzz_target_1                                                                                                                                                         17:34:37
       Fresh cc v1.0.12
       Fresh unicode-xid v0.1.0
       Fresh arbitrary v0.1.1
       Fresh proc-macro2 v0.3.7 (https://github.com/alexcrichton/proc-macro2.git#6b46debb)
       Fresh libfuzzer-sys v0.1.0 (https://github.com/rust-fuzz/libfuzzer-sys.git#4594b1f3)
   Compiling foo-fuzz v0.0.1 (file:///tmp/foo/fuzz)
     Running `rustc --crate-name fuzz_target_1 fuzz_targets/fuzz_target_1.rs --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=af20ade196174b98 -C extra-filename=-af20ade196174b98 --out-dir /tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps --target x86_64-unknown-linux-gnu -C incremental=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/incremental -L dependency=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps -L dependency=/tmp/foo/fuzz/target/debug/deps --extern libfuzzer_sys=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps/liblibfuzzer_sys-135211f33f4e7f2c.rlib --extern proc_macro2=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps/libproc_macro2-9eae6d0c58594618.rlib --cfg fuzzing -Cpasses=sancov -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-trace-pc-guard -Cllvm-args=-sanitizer-coverage-trace-compares -Cllvm-args=-sanitizer-coverage-trace-divs -Cllvm-args=-sanitizer-coverage-trace-geps -Cllvm-args=-sanitizer-coverage-prune-blocks=0 -Zsanitizer=address -Cpanic=abort -C target-cpu=native -L native=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/build/libfuzzer-sys-1478a0911e919e3e/out`
error: the linked panic runtime `panic_unwind` is not compiled with this crate's panic strategy `abort`

error: aborting due to previous error

error: Could not compile `foo-fuzz`.

Caused by:
  process didn't exit successfully: `rustc --crate-name fuzz_target_1 fuzz_targets/fuzz_target_1.rs --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=af20ade196174b98 -C extra-filename=-af20ade196174b98 --out-dir /tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps --target x86_64-unknown-linux-gnu -C incremental=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/incremental -L dependency=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps -L dependency=/tmp/foo/fuzz/target/debug/deps --extern libfuzzer_sys=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps/liblibfuzzer_sys-135211f33f4e7f2c.rlib --extern proc_macro2=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/deps/libproc_macro2-9eae6d0c58594618.rlib --cfg fuzzing -Cpasses=sancov -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-trace-pc-guard -Cllvm-args=-sanitizer-coverage-trace-compares -Cllvm-args=-sanitizer-coverage-trace-divs -Cllvm-args=-sanitizer-coverage-trace-geps -Cllvm-args=-sanitizer-coverage-prune-blocks=0 -Zsanitizer=address -Cpanic=abort -C target-cpu=native -L native=/tmp/foo/fuzz/target/x86_64-unknown-linux-gnu/debug/build/libfuzzer-sys-1478a0911e919e3e/out` (exit code: 101)
error: could not build fuzz script: "cargo" "build" "--manifest-path" "/tmp/foo/fuzz/Cargo.toml" "--verbose" "--bin" "fuzz_target_1" "--target" "x86_64-unknown-linux-gnu"

We do have stop-gap solutions but they are not perfect:

• The first one is to use panic::catch_unwind() on our target closure and calling abort on unwinding.
  This method has a major drawback: to distinguish between already reported bugs and new ones, fuzzers inspects the state of the stack frames of the process being signaled (here SIGABRT). However, with this method we call abort() after all of the closure stack frames have been unrolled and so the information of where the panic was thrown is lost from the point of view of the fuzzer. The fuzzer then sees all bugs coming from the same place and so will only report the first one to the user and ignore all the others as duplicates.

• the second one is to use panic::set_hook() to register a panic handler that directly calls abort(). Functionally, it works fine as the process aborts before unwinding and the fuzzer can then read the stack frames and tell apart different bugs.
  This is however not perfect because it means we need to setup this hook at runtime before starting fuzzing but this is not always obvious in a fuzzing context.
  Also, @nagisa feels strongly about this being a hack that should only exist temporarily until the real fix is implemented.

references:
https://github.com/rust-fuzz/libfuzzer-sys/issues/31
https://github.com/rust-fuzz/libfuzzer-sys/pull/32
rust-fuzz/afl.rs@201d80b
rust-fuzz/afl.rs#134
rust-fuzz/cargo-fuzz#152
rust-fuzz/cargo-fuzz#153
rust-fuzz/honggfuzz-rs@abe2b4c
rust-fuzz/honggfuzz-rs@1d7df87

[alexcrichton]

Thanks for the report! Those are pretty old issues at this point and they're not entirely in cache for me, though. Do you have a sample project I could check out and explore what's going on here?

[PaulGrandperrin]

I set up a concrete example for you.
please clone the cargo-issue-5423 branch of honggfuzz-rs : https://github.com/rust-fuzz/honggfuzz-rs/tree/cargo-issue-5423

In this project there is an example directory that I use to showcase and test the fuzzer.
In this branch I removed the 2 calls to abort from the code (the 2 stop-gap solutions I was speaking about) and added proc_macro2 to the dependencies of the example. commit

now, from the example directory:

• if you launch cargo hfuzz run example, the fuzzer will never be able to find the (obvious) panic and will run forever because abort is never called.
• if you launch RUSTFLAGS="-C panic=abort" cargo hfuzz run example, the compilation of the crate will fail
• if you try the first command on master, the fuzzer will find the panic in less than a second.

[alexcrichton]

Ah ok, thanks @PaulGrandperrin! As I suspected though this isn't really related to Cargo at all, rather it's a rustc/proc-macro2 issue:

$ echo 'fn main() {}' > foo.rs
$ rustc -C panic=abort foo.rs -C prefer-dynamic
error: the linked panic runtime `panic_unwind` is not compiled with this crate's panic strategy `abort`

error: aborting due to previous error

The problem is that the proc-macro2 crate has extern crate proc_macro; which is compiled as a dynamic library. This dynamic library depends on the standard library as a dynamic library. Above I've emulated that with -C prefer-dynamic. You unfortunately simply cannot compile a program that dynamically links against libstd with -C panic=abort, as you're seeing here.

There's two fixes here:

• Disable the proc-macro feature of proc-macro2. This will disable linking to the proc_macro crate which will no longer force libstd to be linked dynamically. This is probably what you want for fuzzed program.
• Wait for Decouple proc_macro from the rest of the compiler. rust#49219 which means that the proc_macro crate in the standard sysroot will no longer force libstd to be linked dynamically.