Skip to content

nixos-rebuild: switch to tmpDir during rebuilds#150065

Merged
thiagokokada merged 1 commit intoNixOS:masterfrom
thiagokokada:workaround-issue-144811
Dec 15, 2021
Merged

nixos-rebuild: switch to tmpDir during rebuilds#150065
thiagokokada merged 1 commit intoNixOS:masterfrom
thiagokokada:workaround-issue-144811

Conversation

@thiagokokada
Copy link
Copy Markdown
Contributor

@thiagokokada thiagokokada commented Dec 10, 2021 •
edited
Loading

Motivation for this change

Workaround issue: #144811

The idea of this fix is very simple: since running nixos-rebuild inside a symbolic link directory pointing to /nix/store broke the nixos-rebuild, what we are going to do instead is simply switch to another directory. We already create a temporary directory tmpDir for some things, so we can switch to that directory without much loss.

Keep in mind that this is a workaround. The actual issue seems to be how nix handles references with path:: NixOS/nix#5510

I did a nixos-rebuild switch inside /run/opengl-driver/lib and this workaround works fine. Also tested `nixos-rebuild {dry-build,dry-activate,build-vm,build-vm-with-bootloader,test,edit} commands and they seem to work fine, however I can't test remote builds and other functionality.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@thiagokokada
nixos-rebuild: switch to tmpDir during rebuilds
f88bd76 
This is a workaround for issue #144811 until this issue is either fixed
on nix itself.
@github-actions github-actions bot added the 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS label Dec 10, 2021
@thiagokokada thiagokokada mentioned this pull request Dec 10, 2021
Closed
@thiagokokada
Copy link
Copy Markdown
Contributor Author

thiagokokada commented Dec 10, 2021

This issue affects both release-21.05 and release-21.11 so marking them as backports.

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Dec 10, 2021
@figsoda
Copy link
Copy Markdown
Member

figsoda commented Dec 10, 2021

would pushd and popd be better here?

@thiagokokada
Copy link
Copy Markdown
Contributor Author

thiagokokada commented Dec 10, 2021

would pushd and popd be better here?

The issue of pushd and popd is that since this script is long another directory change could break the expectations (and ends up in an unrelated directory). This is why I decided to use cd instead.

@thiagokokada
Copy link
Copy Markdown
Contributor Author

thiagokokada commented Dec 10, 2021

CC @colemickens @figsoda @tejing1.

@thiagokokada
Copy link
Copy Markdown
Contributor Author

thiagokokada commented Dec 12, 2021

CC @SuperSandro2000 @jonringer.

figsoda
figsoda approved these changes Dec 12, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@figsoda figsoda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I forgot to come back to this, fixes the issue for me

@thiagokokada
Copy link
Copy Markdown
Contributor Author

thiagokokada commented Dec 15, 2021

I really would like one more review here to make sure I am not breaking anything.

CC @rnhmjoj @nrdxp @regnat @domenkozar

@thiagokokada thiagokokada merged commit d95c811 into NixOS:master Dec 15, 2021
@thiagokokada thiagokokada deleted the workaround-issue-144811 branch December 15, 2021 15:01
@github-actions github-actions bot mentioned this pull request Dec 15, 2021
Merged
1 task
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 15, 2021

Successfully created backport PR #150856 for release-21.05.

@github-actions github-actions bot mentioned this pull request Dec 15, 2021
Merged
1 task
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Dec 15, 2021

Successfully created backport PR #150857 for release-21.11.

@TredwellGit
Copy link
Copy Markdown
Member

TredwellGit commented Dec 16, 2021

This seems to break boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages causing it to use linuxPackages.

@TredwellGit TredwellGit mentioned this pull request Dec 16, 2021
Merged
13 tasks
@thiagokokada
Copy link
Copy Markdown
Contributor Author

thiagokokada commented Dec 16, 2021

This seems to break boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages causing it to use linuxPackages.

Let's revert it them.

@thiagokokada thiagokokada mentioned this pull request Dec 16, 2021
Merged
@thiagokokada thiagokokada mentioned this pull request Jan 4, 2022
Merged
13 tasks
github-actions bot pushed a commit that referenced this pull request Jan 8, 2022
@thiagokokada @github-actions
nixos-rebuild: do not resolve flake path
49d0ae2 
The removed lines converted the flake path passed by the command line
from `/some/path` to `git+file:///some/path`.

This technically shouldn't cause any issues, however running
`nixos-rebuild switch` inside a directory `/nix/store` will cause the
switch to fail and leave a partially construct generation (see issue #144811
for details).

By itself this shouldn't be too much of an issue, however thanks to
another issue in `systemd-boot-builder.py` this can leave the system
in a broken state for those using `boot.loader.systemd-boot` (AFAIK the
default), where future `nixos-rebuild switch` will fail
(see issue #93694 for details).

The issue can be fixed by running
`nix-env -p /nix/var/nix/profiles/system --delete-generations old`,
however this makes newbies very confused and it is showing in our
support threads in Matrix and Discourse (see
https://discourse.nixos.org/t/need-help-on-failure-of-building-my-configuration/16842).

Keep in mind this is a workaround. The actual issue seems to be in nix
itself (see: NixOS/nix#5510).

See also #150065 for an alternative fix that caused other issues.

Kudos for @figsoda for figuring out this fix.

(cherry picked from commit c274d04)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@colemickens colemickens colemickens approved these changes

@figsoda figsoda figsoda approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@thiagokokada @figsoda @TredwellGit @colemickens