SIGABRT: abort crashes starting with Alpine 3.19

[webmaster128]

Hey there!

We got multiple reports from users that bring up "SIGABRT: abort" crashes in systems using Alpine 3.19. If Alpine 3.17 or 3.18 is used, everything is fine. I cannot easily reproduce it yet but would like to provide as much info as I have at this point to avoid extra work.

Describe the bug

We are seeing crashes in this stack when Alpine 3.19 is used

(gdb) bt
#0  a_crash () at ./arch/x86_64/atomic_arch.h:108
#1  abort () at src/exit/abort.c:27
#2  0x00007fbe28db00e1 in ?? () from /usr/lib/libgcc_s.so.1
#3  0x00007fbe28dc867a in __deregister_frame () from /usr/lib/libgcc_s.so.1
#4  0x0000000002e95f78 in <wasmer_compiler::engine::unwind::systemv::UnwindRegistry as core::ops::drop::Drop>::drop ()
#5  0x0000000002c4d515 in core::ptr::drop_in_place<wasmer_compiler::engine::code_memory::CodeMemory> ()
#6  0x0000000002c4e4d2 in core::ptr::drop_in_place<std::sync::mutex::Mutex<wasmer_compiler::engine::inner::EngineInner>> ()
#7  0x0000000002c43e0d in alloc::sync::Arc<T,A>::drop_slow ()
#8  0x0000000002c4b6f8 in core::ptr::drop_in_place<wasmer::engine::Engine> ()
#9  0x000000000320ce98 in cosmwasm_vm::cache::Cache<A,S,Q>::save_wasm_unchecked ()
#10 0x0000000002c45da8 in wasmvm::cache::do_save_wasm ()

where save_wasm_unchecked just takes a Wasm code, compiles it and stores it to disk (no instance or execution).

The affected systems show that the build system Alpine 3.19 makes the difference here:

Looking at the stack trace you see that libgcc is used on the Alpine system which is not what unsafe fn register_frames( seems to expect.

It turns our that the exact same problem is discussed and fixed in Wasmtime: bytecodealliance/wasmtime#7997

Previously this decision was static. FreeBSD and Linux glibc would assume libgcc and everything else was assumed to be libunwind. It's possible to use libgcc on other platforms, however, such as with musl.

Steps to reproduce

Unfortunately I don't have a minimal reproducer yet

Expected behavior

No crashes

Actual behavior

Crashes as above

Additional context

Wasmer 4.2.2 and 4.2.6 behave the same way

[linear]

RUN-113 SIGABRT: abort crashes starting with Alpine 3.19

[xdoardo]

Hello! I was able to reproduce the issue pointed out in bytecodealliance/wasmtime#7997, but unfortunately I was not able to reproduce this one. I tried to recreate the state outlined in save_wasm_unchecked, that is

fn test() -> anyhow::Result<()> {
    for i in 0.. {
        println!("{i}");
        let mut config = wasmer::Cranelift::new();
        config.canonicalize_nans(true);
        wasmer::CompilerConfig::push_middleware(
            &mut config,
            std::sync::Arc::new(wasmer_middlewares::Metering::new(0, |_| 0)),
        );
        let engine = wasmer::Engine::from(config);
        let module = wasmer::Module::new(&engine, include_bytes!("test.wasm"))?;
        drop(engine);
        drop(module);
    }

    Ok(())
}

fn main() -> anyhow::Result<()> {
    test()?;
    Ok(())
}

The test function is a result of what's outlined in the above-mentioned issue, the save_wasm_unchecked function, and #1568. I tested with this docker image:

FROM rust:1.75-alpine3.19
RUN apk add musl-dev gcc
COPY drop /root/drop
ENV RUSTFLAGS='-Ctarget-feature=-crt-static'
WORKDIR /root/drop
RUN cargo build --verbose

FROM rust:1.75-alpine3.19
COPY --from=0 /root/drop/target/debug/drop /bin/drop
CMD ["/bin/drop"]

After various unsuccessful iterations of the test on both aarch64 and x86_64, I simply ported the patch from wasmtime to our SysV unwind mod. If there's any more information that you can share to reproduce the crash it'd be greatly appreciated - alternatively, if you can confirm that this patch solves the issue on your side, we can proceed with our tests and get this merged.

Edit: selecting Singlepass as engine allowed me to reproduce the error and check that the patch mentioned above solves it. Nonetheless, before merging the patch and closing this issue, it'd be great if you can confirm that it solves the issue on your side as well!

[syrusakbary]

At the end were able to reproduce the issue (with a loop running and compiling modules many times), and then test that with the fix the issue no longer occurs.

So this should be resolved