feat: nix builder

[juliamertz]

I made a little POC nix builder.

It spawns nix-eval-jobs which concurrently evaluates the nix expressions and outputs .drv files to ./gc_roots (this should probably just be a temp directory)

While it's still evaluating we parse incoming messages containing ready to be built derivations. and queue these to be built by the worker tasks. This should probably be rewritten to use a JoinSet so we can forego the worker pool.

Only thing that is still a little ambiguous is the output of nix-built containers. Ideally we use something like pkgs.dockerTools.buildImage But there are some problems with the current implementation:

• Does not adhere to the OCI spec, but rather the docker spec.
• Only supports outputting tar files
• Quite slow

As an alternative we could use nix2container but this also brings many problems with it. I think our best bet is to modify pkgs.dockerTools.buildImage so we skip tarring and potentially fix some docker related quirks

[dmeijboom]

Very nice, looking forward to the finished version!

I think our best bet is to modify pkgs.dockerTools.buildImage so we skip tarring and potentially fix some docker related quirks

Personally, I think nix2container is the way to go. Either directly or by rewriting it to fit our use-case. I don't think it's as simple as 'build without tar' when using pkgs.dockerTools.buildImage.

[juliamertz]

Personally, I think nix2container is the way to go. Either directly or by rewriting it to fit our use-case. I don't think it's as simple as 'build without tar' when using pkgs.dockerTools.buildImage.

I'm sure it's probably way more complicated than I'm hoping, but it does look like they just tar the image directory right at the end of the build script which does give me some hope.

I'll try this tomorrow to see if / how well this works

[juliamertz]

@dmeijboom It really was that simple

diff --git a/pkgs/build-support/docker/default.nix b/pkgs/build-support/docker/default.nix
index 09096dfc78a2..d57462b2b745 100644
--- a/pkgs/build-support/docker/default.nix
+++ b/pkgs/build-support/docker/default.nix
@@ -876,7 +876,7 @@ rec {
             chmod -R a-w image
 
             echo "Cooking the image..."
-            tar -C image --hard-dereference --sort=name --mtime="@$SOURCE_DATE_EPOCH" --owner=0 --group=0 --xform s:'^./':: -c . | ${compress.compress} > $out
+            cp -r image $out
 
             echo "Finished."
           '';

; ls result   
17c0c49089dc760c7f05d45ea10b775199b6e6d46ddd762aacda40f1462fccc7.json  manifest.json
3102b4a7ba8890727b3431a1f41b8e4e8e798250536f2ec3a04720b0606c6f34       repositories

It's still not as fast as nix2container, and it looks like it does output the docker format instead of OCI, but it takes about ~5 seconds now to build a container image containing gnu hello

I think the ideal solution is waiting for #390624 to get merged into nixpkgs, that would solve all these problems, we could use that branch and provide an overlay in the meantime.

[dmeijboom]

and it looks like it does output the docker format instead of OCI

^ This is what I was referring to. Not doing TAR is easy but the Docker Image spec and OCI image spec aren't compatible

I think the ideal solution is waiting for NixOS/nixpkgs#390624 to get merged into nixpkgs, that would solve all these problems, we could use that branch and provide an overlay in the meantime.

I agree!

[dmeijboom]

It looks very clean, very nice! Some minor stuff left. Let's also add it in the README

[dmeijboom]

Very nice, design looks sound. Some minor things and it should be ready to merge!

[dmeijboom]

LGTM