minimal-bootstrap: make sources a non-tarballs.nixos.org FOD

[ghost]

Alternative to #232576, does not require manual uploads to tarballs.nixos.org.

Description of changes

This commit adjusts #232576 to break the fetchurl<->minimal-bootstrap-sources dependency cycle without needing an upload to tarballs.nixos.org. It does this by appending the low-level FOD attributes onto the runCommand derivation.

https://nixos.org/manual/nix/unstable/language/advanced-attributes.html#adv-attr-outputHash

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[ghost]

The gigantic comment should be moved into the manual once we switch over to using minimal-bootstrap as our source of bootstrapFiles.

[emilytrau]

This is awesome! Super creative way of solving the bootstrap sources issue! 😍

Have you got a method to validate that this won't have infinite recursion once we integrate into the bootstrap-tools dependency chain? I'm not sure how to test that

[emilytrau]

@amjoseph-nixpkgs would you be interested in adding yourself the minimal-bootstrap maintainers team as well?

[ghost]

Squashed.

Have you got a method to validate that this won't have infinite recursion once we integrate into the bootstrap-tools dependency chain? I'm not sure how to test that

Yes, we can have a test which initializes a completely empty store and does a nix-instantiate, like this

@amjoseph-nixpkgs would you be interested in adding yourself the minimal-bootstrap maintainers team as well?

Sure; I probably need to add myself to maintainer-list.nix first though. Procrastination, procrastination...

[emilytrau]

One last thing and very happy to go! 🚀🚀

[Ericson2314]

This will eventually not work because the drv files are cyclic (cyclic hashing problem) even if the input-addressing isn't cyclic. But that's fine. We can use a stub failing FOD with the same hash at that point, just like the fetchers for proprietary things.

[ghost]

You're absolutely right. We should just do it now and get it over with. @Ericson2314 @emilytrau, please take a look at c8080b0 which implements this. Here's what the failure looks like on the console:

[ghost]

I tested this by making bootstrapTools depend on minimal-bootstrap-tools as shown below, and eval succeeded. The build began, but obviously it won't complete.

diff --git a/pkgs/stdenv/linux/bootstrap-tools/default.nix b/pkgs/stdenv/linux/bootstrap-tools/default.nix
index 569f0c6f31e2..6aa8e6fbc6cf 100644
--- a/pkgs/stdenv/linux/bootstrap-tools/default.nix
+++ b/pkgs/stdenv/linux/bootstrap-tools/default.nix
@@ -3,7 +3,7 @@
 derivation ({
   name = "bootstrap-tools";

-  builder = bootstrapFiles.busybox;
+  builder = "/bin/sh";

   args = [ "ash" "-e" ./scripts/unpack-bootstrap-tools.sh ];

diff --git a/pkgs/stdenv/linux/default.nix b/pkgs/stdenv/linux/default.nix
index 9cfe21e3640d..e475787287d6 100644
--- a/pkgs/stdenv/linux/default.nix
+++ b/pkgs/stdenv/linux/default.nix
@@ -55,6 +55,8 @@
 #     $ nix-tree --derivation $(nix-instantiate -A stdenv)
 { lib
 , localSystem, crossSystem, config, overlays, crossOverlays ? []
+, minimal-bootstrap-tools ?
+  (import ../../os-specific/linux/minimal-bootstrap/stage0-posix/bootstrap-sources.nix {}).minimal-bootstrap-sources

 , bootstrapFiles ?
   let table = {
@@ -92,7 +94,10 @@
     or (abort "unsupported libc for the pure Linux stdenv");
   files = archLookupTable.${localSystem.system} or (if getCompatibleTools != null then getCompatibleTools
     else (abort "unsupported platform for the pure Linux stdenv"));
-  in files
+  in {
+    bootstrapTools = minimal-bootstrap-tools;
+    busybox = null;
+  }
 }:

 assert crossSystem == localSystem;

[emilytrau]

Linking back to the thread #227914

[ghost]

I squashed all of the suggested-changes.

c8080b0 has new content; please take a look. See also #238357 (comment)

[Ericson2314]

Looks good to me!

[Ericson2314]

Uh oh, did I merge to soon? the hex1 build seems to be spinning in a loop for me.