Replace libunwind + libosxunwind with LLVM's libunwind

[ararslan]

Currently for unwinding we're using nongnu libunwind and our own libosxunwind, the latter of which is based on Apple's open source implementation. Apple's implementation was later contributed upstream into the LLVM project as LLVM's libunwind. A brief discussion about the history and relationship between nongnu and LLVM's libraries can be found in this Clang developer's thread.

One of the primary benefits of migrating to LLVM's libunwind is the ability to ditch libosxunwind, which is not kept in sync with any upstream code and is not actively maintained, giving us the opportunity to consolidate the dependencies on nongnu libunwind and libosxunwind into a single dependency on LLVM libunwind.

[jamie-arcc]

The more we can move away from custom external deps, the better

[ViralBShah]

We will need @Keno and @vtjnash to chime in. It is certainly the right direction if we can do this.

[Keno]

It certainly makes sense to replace libosxunwind since the LLVM code base is a fork of the same Apple source code. libunwind is an independently maintained code base, so there's less need to get rid of it.

[nalimilan]

I'd say libunwind is independently unmaintained, given that it's been in RC for one year now, which causes problems for distributions due to bugs in the last release (libunwind/libunwind#30). So it would be nice to get rid of that dependency if possible.

[ararslan]

Something to note is LLVM libunwind's platform support (modified from the latest docs):

OS	Arch	Compilers	Unwind info
Any	i686, x86-64, ARMv7	Clang	SjLj
Bare Metal	ARMv7	Clang, GCC	EHABI
FreeBSD	i686, x86-64, AArch64	Clang	DWARF CFI
Linux	ARMv7	Clang, GCC	EHABI
Linux	i686, x86-64, AArch64	Clang, GCC	DWARF CFI
macOS	i686, x86-64	Clang, GCC	DWARF CFI
Windows	i686, x86-64, ARMv7, AArch64	Clang	DWARF CFI

This covers our supported platforms quite well, but note that GCC is not listed for Windows and there's no mention of PowerPC, so if we ever support that again we may be in trouble.

[nalimilan]

Ah, sounds like a problem indeed.

[ararslan]

We could at least start by using LLVM libunwind for macOS builds only. In terms of implementation, we could have a Make flag USE_LLVM_LIBUNWIND that's set for macOS and can be optionally enabled for other platforms at the user's own risk.

[musm]

We can use clang with a mingw backend using the llvm-mingw project https://github.com/mstorsjo/llvm-mingw

[imciner2]

One thing to note is that the LLVM version of libunwind supports the 64-bit RISC-V architecture (https://reviews.llvm.org/D68362?id=222937) while the nongnu libunwind does not (and the upstream author of the nongnu version doesn't have time to implement it themselves, so it is unclear when it will be available there: libunwind/libunwind#99). While I realize it isn't technically supported by Julia yet, it has to start somewhere.

Where exactly is libunwind used in Julia? Is it just used as a dependency in the LLVM build, or is it actually used on its own in other places? (my code search doesn't seem to find any includes of unwind.h in the Julia source and really only turns up

julia/deps/llvm.mk

Line 292 in 21e7486

BUILT_UNWIND := $(build_prefix)/manifest/osxunwind

as its usage point).

[c42f]

Where exactly is libunwind used in Julia?

See src/stackwalk.c

Also look for the appearances of #ifdef LIBOSXUNWIND in the source if you want to see where the differences lie.

[IanButterworth]

Has there been any work on this, or is anyone planning to take it on?
Just given that:

• It's is a blocker for Darwin/ARM64 Darwin/ARM64 tracking issue #36617
• Darwin/x86 is suffering from many detached profiling samples, which might benefit from consolidating unwind sources across platforms? Profiling on MacOS suffering a lot of detached samples #38350

[imciner2]

The first step is probably to build the LLVM libunwind in the Yggdrasil LLVM script and ensure it builds properly there.

After that, I was waiting on the JLL work in the main repo to calm down a little before trying anything, because I think it is easier to make this switch after the JLL work is done.

[omus]

I'm attempting to get the LLVM libunwind JLL built. From my initial attempts it appears that PowerPC builds fine but I have to work through some issues with Windows.

[imciner2]

I'm attempting to get the LLVM libunwind JLL built. From my initial attempts it appears that PowerPC builds fine but I have to work through some issues with Windows.

I would say don't focus on all architectures at once. The first platform that can benefit the most is just OSX (since it allows the removal of libosxunwind), but it is also useful for Linux (because of the support for more diverse and newer architectures). Windows can always be a last-supported OS if need be. (provided the Yggdrasil script can allow you to do architecture-specific things like this).

[omus]

I'm starting with replacing libosxunwind with LLVM's libunwind. I've managed to get Julia build with the alternate libunwind on macOS and am working through the failed tests.

[IanButterworth]

Great! Once you have a PR I'd happily test for this profiling bug #38350 (comment)

[omus]

With #39127 merged the current state of things is that macOS has switched to LLVM libunwind. All other platforms are using nongnu libunwind. We can, and should, switch over to using LLVM libunwind for other platforms.

[kasperk81]

afaict, debian/ubuntu does not have a standalone package for llvm-libunwind. also, it requires libcxx from llvm. so maybe the missing package is just an oversight or more involved like distros which primarily use gnu libc[xx] stack avoid mixing up things.

it would be nice if we push nognu libunwind to add support for new instruction set instead, as overall it has more millage than llvm's unwinder (since 1990's vs. since 2013 when they implemented the initial version) and the latter is not as mature in terms of features it provides either, like saving location of cntx ptrs. demand could lead to a good change in the end.

[kasperk81]

riscv64 support was added in libunwind/libunwind#267. Thanks to @zhaofengli!