nixUnstable: pre20200721_ff314f1 -> pre20200829_f156513

[kalekseev]

Motivation for this change

Flakes are very slow in current version on mac os NixOS/nix#3874

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[Ma27]

The tests seem broken btw:

nix> ran test tests/recursive.sh... [FAIL]
nix> common.sh: line 94: unshare: command not found
nix> ++ uname
nix> + [[ Linux != Linux ]]
nix> + clearStore
nix> + echo 'clearing store...'
nix> clearing store...
nix> + chmod -R +w /build/nix-test/recursive/store
nix> + rm -rf /build/nix-test/recursive/store
nix> + mkdir /build/nix-test/recursive/store
nix> + rm -rf /build/nix-test/recursive/var/nix
nix> + mkdir /build/nix-test/recursive/var/nix
nix> + nix-store --init
nix> + clearProfiles
nix> + profiles=/build/nix-test/recursive/var/nix/profiles
nix> + rm -rf /build/nix-test/recursive/var/nix/profiles
nix> + rm -f /build/nix-test/recursive/result
nix> ++ nix add-to-store ./recursive.sh
nix> warning: you don't have Internet access; disabling some network-dependent features
nix> + export unreachable=/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh
nix> + unreachable=/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh
nix> + nix --experimental-features 'nix-command recursive-nix' build -o /build/nix-test/recursive/result -L --impure --expr '
nix> with import ./config.nix;
nix> with import <nix/config.nix>;
nix> mkDerivation {
nix> name = "recursive";
nix> dummy = builtins.toFile "dummy" "bla bla";
nix> SHELL = shell;
nix> # Note: this is a string without context.
nix> unreachable = builtins.getEnv "unreachable";
nix> requiredSystemFeatures = [ "recursive-nix" ];
nix> buildCommand = '\'''\''
nix> mkdir $out
nix> PATH=${nixBinDir}:$PATH
nix> opts="--experimental-features nix-command"
nix> # Check that we can query/build paths in our input closure.
nix> nix $opts path-info $dummy
nix> nix $opts build $dummy
nix> # Make sure we cannot query/build paths not in out input closure.
nix> [[ -e $unreachable ]]
nix> (! nix $opts path-info $unreachable)
nix> (! nix $opts build $unreachable)
nix> # Add something to the store.
nix> echo foobar > foobar
nix> foobar=$(nix $opts add-to-store ./foobar)
nix> nix $opts path-info $foobar
nix> nix $opts build $foobar
nix> # Add it to our closure.
nix> ln -s $foobar $out/foobar
nix> [[ $(nix $opts path-info --all | wc -l) -eq 3 ]]
nix> # Build a derivation.
nix> nix $opts build -L --impure --expr '\''
nix> derivation {
nix> name = "inner1";
nix> builder = builtins.getEnv "SHELL";
nix> system = builtins.getEnv "system";
nix> fnord = builtins.toFile "fnord" "fnord";
nix> args = [ "-c" "echo $fnord blaat > $out" ];
nix> }
nix> '\''
nix> [[ $(nix $opts path-info --json ./result) =~ fnord ]]
nix> ln -s $(nix $opts path-info ./result) $out/inner1
nix> '\'''\'';
nix> }
nix> '
nix> warning: you don't have Internet access; disabling some network-dependent features
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> recursive> /build/nix-test/recursive/store/bxgxg5zcfy4l291y9x943qg0p6ylrs65-dummy
nix> 2 operations
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 4 operations
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 2 operations
nix> recursive> error: --- InvalidPath --- nix
nix> recursive> path '/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh' is not valid
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 2 operations
nix> recursive> error: --- Error --- nix
nix> recursive> error: --- InvalidPath --- nix
nix> recursive> cannot build unknown path '/build/nix-test/recursive/store/g930n38jva165vwd1n36aqjs6im47f31-recursive.sh' in recursive Nix
nix> recursive> warning: you don't have Internet access; disabling some network-dependent features
nix> 2 operations
nix> recursive> error: --- Error --- nix
nix> recursive> error: --- Error --- nix
nix> recursive> invalid operation 120
nix> error: --- Error --- nix
nix> builder for '/build/nix-test/recursive/store/a3kzhq1bri96pviigkqfa4rw9n4v8sni-recursive.drv' failed with exit code 1

[kalekseev]

I tested only on mac os, and that test is skipped on it, I thought that if nix passed tests on github actions it should pass on linux, will look at it when I get access to my linux box.

[Ma27]

Hmm, when I build it locally (with nix develop in my git checkout and with nix-build) it's not reproducible anymore. I assume this is because the nix-daemon on my remote builder is too old (i.e. Nix 2.3).

@edolstra not sure what we want to do about this tbh.

[kalekseev]

Hmm, when I build it locally (with nix develop in my git checkout and with nix-build) it's not reproducible anymore. I assume this is because the nix-daemon on my remote builder is too old (i.e. Nix 2.3).

@edolstra not sure what we want to do about this tbh.

Tested on Ubuntu: tests pass, binaries work in all my use cases.

[Ma27]

Tested on Ubuntu: tests pass, binaries work in all my use cases.

Well I assume you already have a Nix 2.4/3.0 running as nix-daemon? The remote builder I used for my first (and only unsuccessful) attempt still runs Nix 2.3 because I prefer a stable release on my servers.

This is just a guess, but seems reasonable looking at the "invalid operation 120" which seems related to the worker-protocol which is (IIRC) also used in the nix-daemon.

I think that this can become a problem as a lot of users still use Nix 2.3 for good reasons and will run into the same issues if my theory is right. That's why I'd like to get feedback from @edolstra on this first before merging this anywhere.

[edolstra]

We're looking into this here: NixOS/nix#3943

[kalekseev]

Hi @Ma27, NixOS/nix#3943 was fixed, so I updated pr with latest commit, could you test it please, I tested on macOS and ubuntu with nix2.4/nix2.3.

[Ma27]

@edolstra should we backport this as well?