Figure out how to make symbolisation code smaller

[joboet]

Rust binaries are notoriously large. For instance, a simple, release-mode "Hello, world!" binary has a size of over 400 kB on my machine (ARM macOS). Much of this size (probably more than half) comes from the symbolisation that std performs when printing backtraces, as that pulls in a DWARF parser, an ELF/Mach-O parser, four instantiations of sort and some other things in order to support printing line numbers and file names. Annoyingly, this code is almost completely redundant when a binary is compiled with the release profile, as that profile disables the generation of the very DWARF debug info that all that code aims to use. Thus, when compiling with -C debuginfo=none, all this code could be replaced by a simple dladdr call without any loss of functionality.

Unfortunately, std being pre-compiled makes fixing this is rather difficult since we can't just add a feature flag to std and turn that on and off depending on compiler flags. My idea here would be to copy the strategy used to implement -C panic and make -C debuginfo select between two separate crates: one that calls back into std to employ the backtrace-rs logic and one that uses dladdr or a similar platform function. I'm not sure that's feasible, however.

This issue exists to coordinate work on reducing the size of backtrace-rs (there are some obvious some fixes like using dynamic dispatch to reduce the number of sort instantiations) and to discuss strategies for reducing the binary size further.

[hanna-kruppe]

For the sorting, if it’s not performance critical then you can shrink the code to a few hundred bytes per monomorphization by using very simple textbook algorithms instead of the smart modern hybrid methods (see https://github.com/Voultapher/tiny-sort-rs for example).

[bjorn3]

-Cdebuginfo=none can still mean that other crates have debuginfo. Also dladdr seems to be relatively slow on glibc as it does a linear scan for each symbol (and so does musl). dladdr is also entirely non-functional when statically linking musl.

[joboet]

-Cdebuginfo=none can still mean that other crates have debuginfo.

As far as I can tell, -C debuginfo is applied DSO-wide (at least the std paths don't show up in release mode), so this would only affect backtraces that unwind through foreign modules – which I don't think is very important to support.

Also dladdr seems to be relatively slow on glibc as it does a linear scan for each symbol (and so does musl).

I'm genuinely curious: Do you know if anyone uses symbolisation in the hot path? It just seems unlikely to me that anyone would care about performance, especially in cases where the program has encountered a bug anyway.

dladdr is also entirely non-functional when statically linking musl.

That's good to know, we'd need to keep that in mind.

[bjorn3]

As far as I can tell, -C debuginfo is applied DSO-wide (at least the std paths don't show up in release mode), so this would only affect backtraces that unwind through foreign modules – which I don't think is very important to support.

No, std debuginfo is absent in release mode because cargo passes -Cstrip=debuginfo by default in release mode when debuginfo is not enabled. Before cargo did started passing -Cstrip=debuginfo hello world was like 1MB bigger because of all libstd debuginfo being present.

I'm genuinely curious: Do you know if anyone uses symbolisation in the hot path? It just seems unlikely to me that anyone would care about performance, especially in cases where the program has encountered a bug anyway.

anyhow shows backtraces on regular errors when RUST_LIB_BACKTRACE=1 or RUST_BACKTRACE=1 is used.

[joboet]

No, std debuginfo is absent in release mode because cargo passes -Cstrip=debuginfo by default in release mode when debuginfo is not enabled. Before cargo did started passing -Cstrip=debuginfo hello world was like 1MB bigger because of all libstd debuginfo being present.

Ah, fair enough...

anyhow shows backtraces on regular errors when RUST_LIB_BACKTRACE=1 or RUST_BACKTRACE=1 is used.

Oh yeah, regressing that wouldn't be great...

[hanna-kruppe]

(Edit: oops, typed too slow and didn’t see the earlier comment by @bjorn3 saying the same more succinctly)

Capturing and formatting backtraces in-process can be performance critical when error values include a backtrace and errors occur and are (for example) logged with non-trivial frequency. See dtolnay/anyhow#380 for a complaint about this in the wild.

As for -C debuginfo=none, this can be set separately for each crate, including via profile overrides in Cargo, and the linked will dutifully preserve whatever subset of debug info was present in the object files. The effects you describe usually happen either because Cargo strips debuginfo that was originally included e.g. in the pre-compiled std objects (see rust-lang/cargo#13257) or because some of the debuginfo was load bearing for being able to walk the stack fully. But these aren’t fundamental problems: Cargo’s defaults can be overridden and on many platforms stack traces can be captured without full debug info by either preserving frame pointers or by keeping enough DWARF tables for unwinding (-Cpanic=abort or -Cforce-unwind-tables=on).

[hanna-kruppe]

Tangential point: even if we can’t rip out the symbolication code entirely in favor of dladdr or something, there’s some tension w.r.t. what data sources are supported. If I ship a binary with -Cstrip=debuginfo and care about binary size a lot (but still want some backtrace on panic), I might be fine with a build of backtrace that only walks the stack and uses ELF symbol tables and Rust symbol demangling. If I want to ship a binary with -Cdebuginfo=line-tables-only for more useful backtraces, I’ll want all the functionality included today and possibly even more (#130417 is blocked on the size increase it brings for all Rust binaries, but for a sufficiently large binary it’s a net win because the compression gains from using zstd instead of zlib more than compensate it). All roads lead to wishing for stable -Zbuild-std…

[Enselic]

For instance, a simple, release-mode "Hello, world!" binary has a size of over 400 kB on my machine (ARM macOS). Much of this size (probably more than half) comes from the symbolisation that std performs when printing backtraces

If you build std without its default backtrace feature, you will get a binary without that symbolisation code. On x86_64-unknown-linux-gnu that results in an hello-world binary size of 48kB:

$ cargo +nightly run --release -Zbuild-std -Zbuild-std-features= --config profile.release.codegen-units=1 --config profile.release.lto=\"fat\" --config profile.release.opt-level=\"s\" --config profile.release.strip=true
Hello, world!
$ ls -lh target/release/rust-hello-world
-rwxr-xr-x 2 martin martin 48K May 29 20:53 target/release/rust-hello-world

If you also remove the panic printing code with panic_immediate_abort the binary becomes 13kB:

$ cargo +nightly run --release -Zbuild-std=std,panic_abort -Zbuild-std-features=panic_immediate_abort --config profile.release.codegen-units=1 --config profile.release.lto=\"fat\" --config profile.release.opt-level=\"s\" --config profile.release.strip=true --config profile.release.panic=\"abort\"
Hello, world!
$ ls -lh target/release/rust-hello-world
-rwxr-xr-x 2 martin martin 13K May 29 20:58 target/release/rust-hello-world

Traige: Does this solve your use case? If not, I'm unsure when this issue can be closed as fixed. All code can always (more or less) be made smaller. Removing it completely by opting out from the backtrace feature seems like a quite good option if you need really small binaries?

[bjorn3]

Making the symbolication code will make all binaries smaller. Removing it entirely however requires using nightly (unacceptable as solution in certain cases, like distro builds) and manually doing something, which most people complaining about binary size likely won't even know about. (Also -Zbuild-std doesn't work for distro builds even on nightly as it is currently not offline compatible)