Skip to content

Comments

elfutils: enable debuginfod support#223240

Merged
symphorien merged 1 commit intoNixOS:stagingfrom
symphorien:elfutils_full
Apr 10, 2023
Merged

elfutils: enable debuginfod support#223240
symphorien merged 1 commit intoNixOS:stagingfrom
symphorien:elfutils_full

Conversation

@symphorien
Copy link
Member

elfutils contains several libraries, and when debuginfod support is enabled, only libdebuginfod.so depends on libarchive and curl, among others, leading to a large closure size increase.

To mitigate this effect, libdebuginfod.so is moved to a separate output. libarchive and libcurl remain in the build closure, but derivations which depend on, say, libelf, but not libdebuginfod only retain a reference to the out output and not the libdebuginfod output and their runtime closure size does not increase.

Enabling libdebuginfod by default allows to use nixseparatedebuginfod without recompiling gdb and perf, mostly.
This admittedly complex contraption ensures we don't get two copies of libelf (one for systemd withouth debuginfod support, and one for debuginfod enabled gdb) in this case.

The bin output contains build tools, and since curl and libarchive are always in the build closure we dont move debuginfod and debuginfod-find to a separate output.

tested by rebuilding
/nix/store/axqgmkgmhy2rzlbffglhjv1npy5ni0ra-gdb-13.1
/nix/store/73cbnzc96xir60xff5qblhvw6fb7n309-perf-linux-6.1.21
/nix/store/nk7lla63w1pbzyavgz7hcvdhmrd9qgqy-systemtap-6.1.21-4.8
/nix/store/bhss1whsh67b0wa1mj5h3z1p4nwndxd4-valgrind-3.20.0
/nix/store/v2hlzj0gbnycj7ssznvjvhw5bxwxsbmm-hotspot-1.4.1
/nix/store/7cnyh97201zk780zwbh1sh913wflsgjp-heaptrack-1.4.0
/nix/store/1893dw0r67hh6iqbjdhwbpb8dsc1i8iy-networkmanager-1.40.12
/nix/store/xc5yfb7v042ypmbwwswpsq91jwapls4k-dwz-0.14
/nix/store/vfyvlmrnns4k0dppd6fnddkxc1s3dkck-linux-6.1.21
and checking which of these keep the debuginfod reference.

Description of changes
Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot requested review from edolstra, globin, lsix, nbp and r-burns March 26, 2023 14:22
@ofborg ofborg bot added 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Mar 26, 2023
@symphorien
Copy link
Member Author

@ofborg eval

Copy link
Contributor

@trofi trofi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. Added one suggestion not to manually split out the dependency.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest it to move to existing $bin, not the new $libdebuginfo. bin/debuginfod depends on libdebuginfo.so anyways. There seems to be no reason to keep a new snowflake derivation output if nothing uses it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most debug tools (gdb, perf, ...) depend on libdebuginfod.so but not on debuginfod itself (the lib is a client while debuginfod is a server implem), so moving debuginfod to $bin seems wasteful from a closure size point of view.

Comment on lines 90 to 94
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The need for patchelf to keep the package running always irks me: it always has a potential missing some reference to patch. Why not keep it in $lib instead? elfutils is a toolchain-style package I would say. It's OK to have a few extra dependencies. Looking at the individual deps you mention:

  • libarchive seems to pull in extra 2MB: 800K of libarchive itself and 1.2MB of libxml2 (the rest looks like baseline packages)
  • libcurl seems to pull in extra 5.2MB: curl itself (800KB), libktb5 (2.2MB), brotli (1.7MB) (the rest looks like baseline packages)

I would say these are benign and worth pulling in unconditionally. The curl one is especially weak I would say as curl is already present in a typical system.

WDYT of just installing debuginfod by default?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT of just installing debuginfod by default?

tbh I expected that people would oppose that, given that elfutils is in the closure of approximately everything through systemd (udev), hence the extra work I did here. If you don't mind I'll wait for a second opinion before going in this direction.

The need for patchelf to keep the package running always irks me: it always has a potential missing some reference to patch.

there are install tests, and they catch the regression if one remove the patchelf invocation, so that seems unlikely.

To give actual numbers: elfutils.bin grows from 47MB to 73MB in closure size by enabling debuginfod support, and the exact list of added dependencies is sqlite, curl, libmicrohttpd, libarchive. It is true that systemd depends on curl and libmicrohttpd, and that nix depends the remaining two, so we will likely have them in most nixos closures.

most reverse dependencies are debuggers which benefit from this support,
plus systemd which already depends on half the additional
depenendencies.
@symphorien
Copy link
Member Author

I changed the pull request to just setting enableDebuginfod to true without patchelf hacks.

@symphorien symphorien merged commit 0fabf9f into NixOS:staging Apr 10, 2023
@symphorien symphorien deleted the elfutils_full branch April 10, 2023 11:17
@IvarWithoutBones
Copy link
Member

This PR breaks the gdb build on x86_64-darwin (and aarch64-darwin assuming that worked before), as elfutils only supports platforms.linux. I have confirmed that manually disabling gdb.enableDebuginfod with an override fixes this problem, unfortunately people on darwin now have to manually do as i did and compile gdb from source.

It would be nice if we could only enable enableDebuginfod by default on platforms that actually support it.

@trofi
Copy link
Contributor

trofi commented Apr 28, 2023

Yeah, conditionally enabling in gdb sounds very reasonable. Let me try to add a trivial solution: #228697

Can you give it a try?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants