Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

installer fails on macOS with error: unexpected EOF reading a line #7242

Open
jessestricker opened this issue Oct 31, 2022 · 4 comments
Open

Comments

@jessestricker
Copy link

Describe the bug

In my GitHub workflow I use cachix/install-nix-action to install Nix. It usually works fine, but this time it crashed and said I should post an issue here.

The version of the action: daddc62a2e67d1decb56e028c9fa68344b9b7c2a
The Nix installer is invoked with: --no-channel-add --darwin-use-unencrypted-nix-store-volume --nix-extra-conf-file /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.bZo25Clt/nix.conf --daemon --daemon-user-count 6

Is this problem specific to the GitHub action?

Log output

  Switching to the Multi-user Installer
  Welcome to the Multi-User Nix Installation
  
  This installation tool will set up your computer with the Nix package
  manager. This will happen in a few stages:
  
  1. Make sure your computer doesn't already have Nix. If it does, I
     will show you instructions on how to clean up your old install.
  
  2. Show you what I am going to install and where. Then I will ask
     if you are ready to continue.
  
  3. Create the system users and groups that the Nix daemon uses to run
     builds.
  
  4. Perform the basic installation of the Nix files daemon.
  
  5. Configure your shell to import special Nix Profile files, so you
     can use Nix.
  
  6. Start the Nix daemon.
  
  Would you like to see a more detailed list of what I will do?
  No TTY, assuming you would say yes :)
  
  I will:
  
   - make sure your computer doesn't already have Nix files
     (if it does, I will tell you how to clean them up.)
   - create local users (see the list above for the users I'll make)
   - create a local group (nixbld)
   - install Nix in to /nix
   - create a configuration file in /etc/nix
   - set up the "default profile" by creating some Nix-related files in
     /var/root
   - back up /etc/bashrc to /etc/bashrc.backup-before-nix
   - update /etc/bashrc to include some Nix configuration
   - back up /etc/zshrc to /etc/zshrc.backup-before-nix
   - update /etc/zshrc to include some Nix configuration
   - create a Nix volume and a LaunchDaemon to mount it
   - create a LaunchDaemon (at /Library/LaunchDaemons/org.nixos.nix-daemon.plist) for nix-daemon
  
  Ready to continue?
  No TTY, assuming you would say yes :)
  
  ---- let's talk about sudo -----------------------------------------------------
  This script is going to call sudo a lot. Normally, it would show you
  exactly what commands it is running and why. However, the script is
  run in a headless fashion, like this:
  
    $ curl -L https://nixos.org/nix/install | sh
  
  or maybe in a CI pipeline. Because of that, I'm going to skip the
  verbose output in the interest of brevity.
  
  If you would like to
  see the output, try like this:
  
    $ curl -L -o install-nix https://nixos.org/nix/install
    $ sh ./install-nix
  
  
  ~~> Fixing any leftover Nix volume state
  Before I try to install, I'll check for any existing Nix volume config
  and ask for your permission to remove it (so that the installer can
  start fresh). I'll also ask for permission to fix any issues I spot.
  
  ~~> Checking for artifacts of previous installs
  Before I try to install, I'll check for signs Nix already is or has
  been installed on this system.
  
  ---- Nix config report ---------------------------------------------------------
          Temp Dir:	/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.KKyzeC03
          Nix Root:	/nix
       Build Users:	6
    Build Group ID:	30000
  Build Group Name:	nixbld
  
  build users:
      Username:	UID
       _nixbld1:	301
       _nixbld2:	302
       _nixbld3:	303
       _nixbld4:	304
       _nixbld5:	305
       _nixbld6:	306
  
  Ready to continue?
  No TTY, assuming you would say yes :)
  
  ---- Preparing a Nix volume ----------------------------------------------------
      Nix traditionally stores its data in the root directory /nix, but
      macOS now (starting in 10.15 Catalina) has a read-only root directory.
  
      To support Nix, I will create a volume and configure macOS to mount it
  ~~> Configuring /etc/synthetic.conf to make a mount-point at /nix
      at /nix.
  
  ~~> Creating a Nix volume
  disk1s7 was already unmounted
  
  ~~> Configuring /etc/fstab to specify volume mount options
  
  ~~> Configuring LaunchDaemon to mount 'Nix Store'
  
  ~~> Setting up the build group nixbld
              Created:	Yes
  
  ~~> Setting up the build user _nixbld1
             Created:	Yes
              Hidden:	Yes
      Home Directory:	/var/empty
                Note:	Nix build user 1
     Logins Disabled:	Yes
    Member of nixbld:	Yes
      PrimaryGroupID:	30000
  
  ~~> Setting up the build user _nixbld2
             Created:	Yes
              Hidden:	Yes
      Home Directory:	/var/empty
                Note:	Nix build user 2
     Logins Disabled:	Yes
    Member of nixbld:	Yes
      PrimaryGroupID:	30000
  
  ~~> Setting up the build user _nixbld3
             Created:	Yes
              Hidden:	Yes
      Home Directory:	/var/empty
                Note:	Nix build user 3
     Logins Disabled:	Yes
    Member of nixbld:	Yes
      PrimaryGroupID:	30000
  
  ~~> Setting up the build user _nixbld4
             Created:	Yes
              Hidden:	Yes
      Home Directory:	/var/empty
                Note:	Nix build user 4
     Logins Disabled:	Yes
    Member of nixbld:	Yes
      PrimaryGroupID:	30000
  
  ~~> Setting up the build user _nixbld5
             Created:	Yes
              Hidden:	Yes
      Home Directory:	/var/empty
                Note:	Nix build user 5
     Logins Disabled:	Yes
    Member of nixbld:	Yes
      PrimaryGroupID:	30000
  
  ~~> Setting up the build user _nixbld6
             Created:	Yes
              Hidden:	Yes
      Home Directory:	/var/empty
                Note:	Nix build user 6
     Logins Disabled:	Yes
    Member of nixbld:	Yes
      PrimaryGroupID:	30000
  
  ~~> Setting up the basic directory structure
  install: mkdir /nix/var
  install: mkdir /nix/var/log
  install: mkdir /nix/var/log/nix
  install: mkdir /nix/var/log/nix/drvs
  install: mkdir /nix/var/nix
  install: mkdir /nix/var/nix/db
  install: mkdir /nix/var/nix/gcroots
  install: mkdir /nix/var/nix/profiles
  install: mkdir /nix/var/nix/temproots
  install: mkdir /nix/var/nix/userpool
  install: mkdir /nix/var/nix/daemon-socket
  install: mkdir /nix/var/nix/gcroots/per-user
  install: mkdir /nix/var/nix/profiles/per-user
  install: mkdir /nix/store
  install: mkdir /etc/nix
  
  ~~> Installing Nix
        Alright! We have our first nix at /nix/store/60sx4c6xflgqk11gvijwzlsczbxgxgwh-nix-2.11.1
  warning: $HOME ('/Users/runner') is not owned by you, falling back to the one defined in the 'passwd' file ('/var/root')
        Just finished getting the nix database ready.
  
  ~~> Setting up shell profiles: /etc/bashrc /etc/profile.d/nix.sh /etc/zshrc /etc/bash.bashrc /etc/zsh/zshrc
  
  # Nix
  if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
    . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
  fi
  # End Nix
  
  
  # Nix
  if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
    . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
  fi
  # End Nix
  
  
  # Nix
  if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
    . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
  fi
  # End Nix
  
  
  ~~> Setting up shell profiles for Fish with with conf.d/nix.fish inside /etc/fish /usr/local/etc/fish /opt/homebrew/etc/fish /opt/local/etc/fish
  
  ~~> Setting up the default profile
  installing 'nix-2.11.1'
  building '/nix/store/w8wrglxx13q20r9yqxskna16vn4xgxym-user-environment.drv'...
  installing 'nss-cacert-3.80'
  error: unexpected EOF reading a line
  (use '--show-trace' to show detailed location information)
  
  ---- oh no! --------------------------------------------------------------------
  Jeeze, something went wrong. If you can take all the output and open
  an issue, we'd love to fix the problem so nobody else has this issue.
  
  :(
@jessestricker
Copy link
Author

relevant: #3605

@jessestricker
Copy link
Author

maybe this has something to do with the installer being run without a TTY?

@abathur
Copy link
Member

abathur commented Oct 31, 2022

#3605 speculates as much, though I think it meant underneath Nix or the test framework. There are still human-run cases like:

@maddiemort
Copy link

We're seeing this quite frequently on macOS 12 in GitHub Actions CI when using the cachix/install-nix-action action. Has there been any progress on the issue?

edolstra added a commit to edolstra/nix that referenced this issue Mar 15, 2023
Hopefully this fixes "unexpected EOF" failures on macOS
(NixOS#3137, NixOS#3605, NixOS#7242, NixOS#7702).

The problem appears to be that under some circumstances, macOS
discards the output written to the slave side of the
pseudoterminal. Hence the parent never sees the "sandbox initialized"
message from the child, even though it succeeded. The conditions are:

* The child finishes very quickly. That's why this bug is likely to
  trigger in nix-env tests, since that uses a builtin builder. Adding
  a short sleep before the child exits makes the problem go away.

* The parent has closed its duplicate of the slave file
  descriptor. This shouldn't matter, since the child has a duplicate
  as well, but it does. E.g. moving the close to the bottom of
  startBuilder() makes the problem go away. However, that's not a
  solution because it would make Nix hang if the child dies before
  sending the "sandbox initialized" message.

* The system is under high load. E.g. "make installcheck -j16" makes
  the issue pretty reproducible, while it's very rare under "make
  installcheck -j1".

As a fix/workaround, we now open the pseudoterminal slave in the
child, rather than the parent. This removes the second condition
(i.e. the parent no longer needs to close the slave fd) and I haven't
been able to reproduce the "unexpected EOF" with this.
Ericson2314 pushed a commit to Ericson2314/nix that referenced this issue Oct 31, 2023
Hopefully this fixes "unexpected EOF" failures on macOS
(NixOS#3137, NixOS#3605, NixOS#7242, NixOS#7702).

The problem appears to be that under some circumstances, macOS
discards the output written to the slave side of the
pseudoterminal. Hence the parent never sees the "sandbox initialized"
message from the child, even though it succeeded. The conditions are:

* The child finishes very quickly. That's why this bug is likely to
  trigger in nix-env tests, since that uses a builtin builder. Adding
  a short sleep before the child exits makes the problem go away.

* The parent has closed its duplicate of the slave file
  descriptor. This shouldn't matter, since the child has a duplicate
  as well, but it does. E.g. moving the close to the bottom of
  startBuilder() makes the problem go away. However, that's not a
  solution because it would make Nix hang if the child dies before
  sending the "sandbox initialized" message.

* The system is under high load. E.g. "make installcheck -j16" makes
  the issue pretty reproducible, while it's very rare under "make
  installcheck -j1".

As a fix/workaround, we now open the pseudoterminal slave in the
child, rather than the parent. This removes the second condition
(i.e. the parent no longer needs to close the slave fd) and I haven't
been able to reproduce the "unexpected EOF" with this.

(cherry picked from commit c536e00)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants