bintools-wrapper: Set ZERO_AR_DATE and re-enable LC_UUID on Darwin

[zhaofengli]

Background

(See #178366)

macOS uses UUIDs to associate binaries with the corresponding debug symbol bundles (.dSYM). #77632 added -no_uuid to the linker flags in the wrapper so UUIDs aren't added in order to improve reproducibility. This breaks normal symbolication for debugging and profiling.

However, the UUIDs generated by ld64 should be deterministic by default, based on the contents of the output file. See related discussion in the Chromium list: https://groups.google.com/a/chromium.org/g/security-dev/c/xAL44GDnaVI/m/qJJm5mNkAAAJ

Why is the UUID not deterministic?

Since LC_UUID is supposed to be deterministic, there must be some differences in the input. I built libuv twice, and here is the diffoscope output against the two build directories.

Immediately we see the values of the SymDebugTable symbols (N_STAB) are different:

   Symbol {
     Name: /private/tmp/nix-build-libuv-1.44.2.drv-1/source/src/.libs/libuv_la-fs-poll.o (12596)
     Type: SymDebugTable (0x66)
     Section:  (0x0)
     RefType: ReferenceFlagUndefinedLazy (0x1)
     Flags [ (0x0)
     ]
-    Value: 0x6307EE91
+    Value: 0x6307F1B8
   }

This 0x6307F1B8 value is a timestamp generated here. To disable it, we can simply set ZERO_AR_DATE.

But a --check build still shows differences!

Sadly, without sandboxing, --check builds expose a different $out to the builder, leading to a different UUID in the resulting binaries (logs). If --keep-failed is used, the different build directories for subsequent builds also cause the UUID to change (e.g., /private/tmp/nix-build-libuv-1.44.2.drv-0 and /private/tmp/nix-build-libuv-1.44.2.drv-1).

The UUIDs seem to be stable across initial builds and --check builds with the same build directory.

How do I test that this works without rebuilding stdenv?

I have a branch with an overridden stdenv (stdenvWip) that uses a modified bintools-wrapper: https://github.com/zhaofengli/nixpkgs/commits/darwin-uuid-wip. To test:

$ nix-build -A libuvWip --no-out-link
[...]
 ***      UUID of libuv.1.dylib in $out:     uuid D7737082-E47B-3BC7-851B-44D4540BAC19
/nix/store/nasrlj1wz0lhm0aj1k90fs8cf5w65wni-libuv-1.44.2
$ sudo nix-store --delete /nix/store/nasrlj1wz0lhm0aj1k90fs8cf5w65wni-libuv-1.44.2
[...]
$ nix-build -A libuvWip --no-out-link
[...]
 ***      UUID of libuv.1.dylib in $out:     uuid D7737082-E47B-3BC7-851B-44D4540BAC19
/nix/store/nasrlj1wz0lhm0aj1k90fs8cf5w65wni-libuv-1.44.2

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.11 Release Notes (or backporting 22.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[tjni]

@zhaofengli

Last year I ran across this PR when looking for a way to enable debug symbols on Darwin. Today, I encountered the same need and found this again. Do you remember if there is any additional work or testing left to do?

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/announcing-nixseparatedebuginfod-a-tool-to-fetch-sources-debug-symbols-of-nix-packages-in-gdb/25171/3

[reckenrode]

While Darwin will be using llvm-ar by default once the stdenv rework lands in master (which zeroes out dates, uids, and gids by default), it would still be great if this could be merged to improve the debugging experience on Darwin. Is there anything else this needs to be made ready for review?

[tjni]

I can spend some time testing if this still works. Do I understand correctly:

1. Even though llvm-ar zeroes out dates, not every package invokes it?
2. While we still use Apple's linker, we need to set ZERO_AR_DATE.

[zhaofengli]

I think this is ready for review and testing already. To reduce the blast radius, we could gate the removal of -no_uuid behind a environment variable so users can opt into it themselves.

[reckenrode]

Even though llvm-ar zeroes out dates, not every package invokes it?

I don’t think anything is reverting back to ar from cctools (yet?), so everything should be invoking it. It doesn’t hurt to include the environment variable just in case though.

Edit: Also, I didn’t notice that ld64 also respects this variable. It’s worth setting for that as well.

[joshchngs]

I've been using a bintools-wrapper with this patch applied for 7 months now – it hasn't caused any problems and works for getting debuggable outputs. I haven't done any formal verification of their reproducibility though.

[lovesegfault]

Let's see how it cooks in staging :)