nixos-rebuild-ng: init

[thiagokokada]

The current state of nixos-rebuild is dare: it is one of the most critical piece of code we have in NixOS, but it has tons of issues:

• The code is written in Bash, and while this by itself is not necessary bad, it means that it is difficult to do refactorings due to the lack of tooling for the language
• The code itself is a hacky mess. Changing even one line of code can cause issues that affects dozens of people
• Lack of proper testing (we do have some integration tests, but no unit tests and coverage is probably pitiful)
• The code predates some of the improvements nix had over the years, e.g.: it builds Flakes inside a temporary directory and read the resulting symlink since the code seems to predate --print-out-paths flag

Given all of those above, improvements in the nixos-rebuild are difficult to do. A full rewrite is probably the easier way to improve the situation since this can be done in a separate package that will not break anyone. So this is an attempt of the rewrite.

The language of choice here is Python. I am open to other options here, and I mostly choose Python since it is the language I am most comfortable here, but I am open for other options. Still, I think Python is a good choice because:

• It is the language of choice for many critical things inside nixpkgs, like the NixOSTestVM and systemd-boot activation scripts
• It is a language with great tooling, e.g.: mypy for type checking, ruff for linting, pytest for unit testing
• It is a scripting language that fits well with the scope of this project
• Python's standard library is great and it means we will probably need zero external dependencies for this project. For example, nixos-rebuild currently depends in jq for JSON parsing, while Python has json in standard library

I am aware about the current switch-to-configuration-ng rewrite, however I am not sure what is the scope of that project vs nixos-rebuild. If the idea is just a drop-in replacement than both of those rewrites are ortogonal, since nixos-rebuild also includes some extra logic for e.g.: profile management. If the idea is to migrate more and more logic to switch-to-configuration-ng and eventually drop nixos-rebuild, I am happy to close this PR.

Objectives and Non-objectives

• Be as much of a drop-in replacement as possible
• Fix obvious bugs
• Obvious improvements that are non-breaking
• Do not change logic even if this would be an improvement

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-rebuild-ng-a-nixos-rebuild-rewrite/55606/1

[Scrumplex]

I think this looks good for an early adopter release

[thiagokokada]

CC @Sigmanificient since you made some good reviews about my Python code.

[thiagokokada]

@ofborg test nixos-rebuild-ng

[thiagokokada]

@ofborg build nixos-rebuild-ng

[lucasew]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 354029

---

x86_64-linux

✅ 2 packages built:

• nixos-rebuild-ng
• nixos-rebuild-ng.dist

[thiagokokada]

For fun:

$ hyperfine -i "nixos-rebuild list-generations" "./result/bin/nixos-rebuild-ng list-generations"
Benchmark 1: nixos-rebuild list-generations
  Time (mean ± σ):     510.0 ms ±  13.2 ms    [User: 182.7 ms, System: 361.7 ms]
  Range (min … max):   492.9 ms … 532.5 ms    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 2: ./result/bin/nixos-rebuild-ng list-generations
  Time (mean ± σ):     108.4 ms ±   5.3 ms    [User: 81.6 ms, System: 26.0 ms]
  Range (min … max):    98.5 ms … 119.0 ms    27 runs

Summary
  ./result/bin/nixos-rebuild-ng list-generations ran
    4.71 ± 0.26 times faster than nixos-rebuild list-generations

BTW, I either need to run nixos-rebuild with sudo or ignore the fact that it exists with error (even if it outputs the result), because it fails with find: ‘/nix/var/nix/profiles/per-container’: Permission denied. This is fixed in nixos-rebuild-ng.

[lucasew]

Nice!

It's using the manpage of nixos-rebuild now. Maybe we should gather more feedback around this --help behaviour but I tested here and it works as expected.

LGTM

[thiagokokada]

We had the branch-off for release-24.11 already so I am going to merge this.

There is a failure for x86_64-darwin right now but this doesn't matter too much since one of the first changes I will do in the next PR is to reduce the build closure, that should fix this issue.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-rebuild-ng-a-nixos-rebuild-rewrite/55606/17