Low default ulimit

[tobiasBora]

Describe the bug

I just reinstalled a NixOs from scratch (23.11), with a very minimalist configuration, and then I tried to load my old, bigger, configuration. But it failed with an error Too many open files. As pointed by @bennofs in #6007 (comment) , the limit has been raised in the nix-daemon process used when running nix-build as a single user, but not when building a new derivation as root (no idea why).

Steps To Reproduce

Create a test.py python file with:

l = []
for i in range(10000):
    print(i)
    l += [ open(f"myfile-{i}.txt", "w") ]

print("All files were successfully opened.")

for d in l:
    d.close()

Create a derivation.nix containing:

{ lib
, stdenv
, python3
}:

stdenv.mkDerivation rec {
  pname = "foo";
  version = "0.0";

  src = ./.;
  
  nativeBuildInputs = [ python3 ];
  buildInputs = [ ];

  installPhase = ''
    python3 $src/test.py
  '';
}

and a default.nix containing:

{ pkgs ? import <nixpkgs> {} }:
pkgs.callPackage ./derivation.nix {}

Then, you can run nix-build as a normal user, this phase will succeed (of course not everything since the install phase does not create the $out directory) and you will see the text print("All files were successfully opened."). On the other hand, do:

$ sudo nix-build

and you will get error:

…
1019
1020
1021
Traceback (most recent call last):
  File "/nix/store/2bnkvm5bsymxxkqq4373j1rrqifb50sn-foo/test.py", line 4, in <module>
OSError: [Errno 24] Too many open files: 'myfile-1021.txt'

For now, my workaround is to build my configuration first as a normal user, and then once it fails due to root right missing, I run again the command as root… but highly unpractical.

Priorities

Add 👍 to issues you find important.

[thufschmitt]

@tobiasBora quick fix: use NIX_REMOTE=daemon before running nixos-rebuild. This will force Nix to run with the daemon and get the right ulimit.

[kevincox]

The correct solution here is that Nix should raise its soft limit to its hard limit upon startup and set it back down in any forked processes.

See https://0pointer.net/blog/file-descriptor-limits.html for more explanation.

This would also remove the need for the explicit ulimit override that the nix-daemon has in NixOS.