stage-1.nix / mktemp: failed to create directory via template ‘initrd-secrets.XXXXXXXXXX’: Read-only file system #73404
Comments
|
Thank you for your contributions.
|
stale
bot
added
the
2.status: stale
|
I'm getting bitten by this as well, trying to install initrd secrets via a remote-deploy method that doesn't really let me control CWD. |
stale
bot
removed
the
2.status: stale
Due to some nixos bugs (e.g. NixOS/nixpkgs#73404), the activation requires CWD to be writeable; the canonical writeable directory is /tmp, so let's use that.
|
Bit me today. Great tips that this only happens when executing e.g. |
|
I'm confused. I'm in |
|
Strange, I'm getting this behavior: nixos-install -I nixpkgs=/mnt/root/nixpkgs # git checkout 8d8a28b With reverted #73780 the installation works (i.e. changing following in /mnt/root/nixpkgs/nixos/modules/system/boot/stage-1.nix and clearing /mnt/nix). nixos-install -I nixpkgs=/mnt/root/nixpkgs |
|
I'm on a fresh install and once again, |
|
Perhaps re-use this: nixpkgs/nixos/modules/system/boot/stage-1.nix Line 378 in 4df4429 Or rather we need some more info about what the exact failure is. Does /tmp exist? How is it mounted? TMPDIR? etc. This error ( |
|
I'm hitting this same problem using initrd.secrets, trying to install 21.11. Same config works fine on 21.05. Happens right after "updating GRUB 2 menu ..." Kills the GRUB 2 install ... BTW, @colemickens method of |
It's attempting to create a file in a |
|
Ah, I found it, it's set by EDIT: I feel like this is arguably a bug in |
|
FYI, today I tried to install NixOS following ladinu's guide and got a similar error, but instead of I'm going to try this with a 21.05 image I have available to see if that makes any kind of difference. UPDATE: I didn't find the 21.05 ISO, so I followed @colemickens advice. It worked, although I'm not sure about the uninitialized /etc warnings. |
|
It just bit me as well, solved with a hacky |
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: |
|
Just ran into this. I successfully worked around it by combing everyone's workarounds. Can I do anything to help get a fix ready? I see this has been open for quite a while. |
|
I can confirm this is still an issue too. I can also confirm Here's a permalink to my NixOS configuration repo at the current time of writing for context if needed:
|
|
Can we have a proper fix for 22.05? This is an absolutely terrible experience for new users. |
|
Using 22.05, following the install guide for Kimsufi and the remote unlocking guide from the wiki.
I get the following error message: Both proposed hacks did not work:
|
|
I think I've went further by |
|
I've linked another similar error to this issue: #156121 (comment). It was reported that the same workaround (#156121 (comment) ) works for that too. So, whatever the problem is, it's getting worse: it's not limited to initrd secrets but likely anything using the /tmp during the installation. |
|
A link to the currently working workaround should be prepended to the issue body; I wanted to do it, but it seems like it's unclear which workaround is to be used. |
|
Is this a release blocker for 22.11 ? cc @mweinelt ? |
|
Probably not, given that it has been an open issue for years and nobody provided a fix until now. |
|
please make it a release blocker, I'm sick of googling this issue every half year. |
|
I'm not involved in the release process so my thoughts don't count for much, but I think there's a real case to make this a release blocker, given that it makes the experience worse for first-time users. Regardless of whether it's designated a blocker, I'd strongly encourage anyone who has the background on install internals to take a look at this, both because it's one of the ugliest things I've encountered when running NixOS systems, and because it appears to violate build hygiene, which is a hole in the theoretical underpinnings of nix. |
|
It would be nice if anyone still having this issue could try my fix in the pull request #199425 to confirm if it solves this issue as well. It might accelerate a merge if it could solve multiple issues at once. |
There are probably several distinct issues being reported here. |
Do you maybe know which script causes this error? |
|
@rnhmjoj Since it doesn't fail with |
The problem is almost surely that the activation script is running inside the chroot environment and the paths, relative to the real root, don't exist there. The solution, assuming the paths actually exist under the new root (say, /mnt) is to strip prefix (/mnt) before doing anything with them.
It should be very easy to reproduce: just put the a file in the configuration directory, say |
|
I should probably debug this and check for the pwd at this point in the script. Unfortunately I'm very busy with a different project and don't see me doing this in the foreseeable future. |
|
|
…activation
|
Let's try to wrap up this mess. As far as I can tell there are at least five different issues being reported in this thread:
As the bulk of the issue seems to be resolved, I'm going to close this issue (but leave it pinned because it might still contain helpful workarounds). If you encounter a problem that's not listed here, please open a new issue. I will try to add |
When installing NixOS in the target filesystem /mnt, paths relative to configuration.nix in `initrd.secrets` are turned by Nix into absolute paths that reference /mnt. While building the system derivation works, installing the bootloader fails because the latter process takes place inside the chroot environment where /mnt does not exist. Ideally, we would also build the system within chroot, but this greatly complicates the matter as it requiers manually copying over Nix, its runtime dependencies and all channels. Possibly, this would also break several assumptions users have about how nixos-install works. A simpler and safer (but less neat) solution is to temporarily bind mount all mount points in /mnt under /mnt/mnt to keep the paths functional while the bootloader is being installed. This is essentially the workaround described in issue NixOS#73404.
When installing NixOS in the target filesystem /mnt, paths relative to configuration.nix in `initrd.secrets` are turned by Nix into absolute paths that reference /mnt. While building the system derivation works, installing the bootloader fails because the latter process takes place inside the chroot environment where /mnt does not exist. Ideally, we would also build the system within chroot, but this greatly complicates the matter as it requires manually copying over Nix, its runtime dependencies and all channels. Possibly, this would also break several assumptions users have about how nixos-install works. A simpler and safer (but less neat) solution is to temporarily bind mount all mount points in /mnt under /mnt/mnt to keep the paths functional while the bootloader is being installed. This is essentially the workaround described in issue NixOS#73404.
When installing NixOS in the target filesystem /mnt, paths relative to configuration.nix in `initrd.secrets` are turned by Nix into absolute paths that reference /mnt. While building the system derivation works, installing the bootloader fails because the latter process takes place inside the chroot environment where /mnt does not exist. Ideally, we would also build the system within chroot, but this greatly complicates the matter as it requires manually copying over Nix, its runtime dependencies and all channels. Possibly, this would also break several assumptions users have about how nixos-install works. A simpler and safer (but less neat) solution is to temporarily bind mount all mount points in /mnt under /mnt/mnt to keep the paths functional while the bootloader is being installed. This is essentially the workaround described in issue NixOS#73404.
|
On unstable the remaining issue has been fixed: so, you can finally install NixOS with relative paths in |
|
well done all! super glad to see this wrapped up! (I know I'm late with my congratulations but it was such a long-running issue that I figure that's fine) |
and others
When "nixos-rebuild boot" is executed from /nix/store (cwd is readonly) it fails with an error message.
It works when you call it from a writeable directory.
In my case the problem only occurs, when the option "boot.initrd.network.ssh" is enabled. Without it works.
What also happend is a broken bootloader. The system did not boot anymore after this failure. A reinstallation of the bootloader was needed.
To Reproduce
boot.initrd.network.enable = true;
boot.initrd.network.ssh.enable = true;
boot.initrd.network.ssh.port = 5022;
boot.initrd.network.ssh.hostECDSAKey = /etc/nixos/secrets/ecdsa-hostkey;
open a shell and change the working directory to /nix/store
execute "nixos-rebuild boot"
Expected behavior
nixos-rebuild should work from every working directory.
Additional context
I think it is related to function "initialRamdiskSecretAppender" in "nixos/modules/system/boot/stage-1.nix"
The mktemp-call creates a directory in the working directory.
It only happens when config.boot.initrd.secrets is not empty.
Metadata
nix run nixpkgs.nix-info -c nix-info -m
"x86_64-linux"Linux 4.19.81, NixOS, 20.03pre200231.7827d3f4497 (Markhor)yesyesnix-env (Nix) 2.3.1"""nixos-20.03pre200231.7827d3f4497"/nix/var/nix/profiles/per-user/root/channels/nixosconsole.txt
The text was updated successfully, but these errors were encountered: