initrd: Opt-in bare bones systemd-based initrd by ElvishJerricco · Pull Request #164943 · NixOS/nixpkgs

ElvishJerricco · 2022-03-20T08:04:00Z

Third time's the charm.

Motivation

This PR adds an option, boot.initrd.systemd.enable, that causes NixOS to generate an initrd that uses systemd for PID 1. This brings several benefits:

Upstream units can be used to implement almost all of stage 1 logic, eliminating the need for the many lines of shell scripting NixOS has in its traditional stage 1.
Early boot becomes parallelized.
systemd-ask-password can be used to request credentials, providing a more streamlined way for early boot tasks to communicate with users.
Configuration of stage 1 becomes declarative in exactly the same way that systemd in stage 2 is declarative, instead of adding imperative shell scripts in a variety of places.

How this compares to my previous attempts

This version of a systemd-based initrd has three major improvements over the previous attempts.

It is an opt-in alternative to NixOS's traditional initrd, instead of a replacement.
It uses the same systemd option types that NixOS uses for stage 2 systemd configuration.
It will be implemented piecemeal, instead of in one large PR.
The produced initrd appears to be quite a bit smaller than the previous attempts for whatever reason, at just 9.1M (which is quite good compared to NixOS's tradtional initrd).

The third improvement is especially important. This PR only implements the core functionality needed to have a systemd-based initrd. There are many things that NixOS's traditional initrd supports that this does not yet. But with this framework in place, we can begin to add those features to the systemd-initrd one by one in separate, parallel PRs.

Notably, this PR does keep the make-initrd-ng concept from the previous one. You can check the README included with it in this PR for details, but the short version is that we don't copy full closures into initrd, and we don't patch things. Instead, we try to automate copying only the exactly correct things to their original paths to the best of our ability, and manually add what the automation misses in the boot.initrd.systemd.objects option. The benefits are outlined in the README.

This can be tested with nix-build ./nixos -A vm.

What this approach will not do

is include a compatibility layer for options like postDeviceCommands and the like. Although this is possible in principle with special units that run the concatenated script with Before= and After= clauses on the appropriate targets, I think this is a bad idea for several reason.

It is antithetical to the purpose of using systemd. We want stage 1 to be parallelized, and declarative. These scripts are the opposite of both. It would slow down boot times and make the code for systemd-initrd less clean.
These scripts sometimes expect to be run in the user interactive console. Trying to accommodate this would be troublesome.
Maddness this way lies. I can only imagine the plethora of complexity and edge cases involved in trying to keep extraUtils and the *Commands options alive in this initrd.

Next steps

Here are a number of things to do in future PRs, in very loose, decreasing order of how I view their importance.

Testing. Currently this will only boot extremely simple setups. It'd be good to know how simple that is, and it will be important to test the other things we work on as we go to make sure they work as well.
Support for the script style options in boot.initrd.systemd.services. Currently they do not copy the necessary scripts in.
Proper udev rules configuration.
Basic LUKS support.
ZFS / btrfs / LVM / mdadm support.
Plymouth support
Networking / SSH support
Advanced LUKS features (e.g. yubikeys).
Move activation into stage 1 so that we can immediately switch-root to systemd. This is nice because systemd can serialize and deserialize some useful information if it knows it's switch-root'ing to systemd.

Most of these should be doable independently of each other, and likely need little more API than the core functionality provided by this PR.

Things done

SuperSandro2000 · 2022-03-20T17:53:17Z

nixos/lib/utils.nix

    lib = import ./systemd-lib.nix { inherit lib config pkgs; };
    unitOptions = import ./systemd-unit-options.nix { inherit lib systemdUtils; };
+    types = import ./systemd-types.nix { inherit lib systemdUtils; };


Suggested change

lib = import ./systemd-lib.nix { inherit lib config pkgs; };

unitOptions = import ./systemd-unit-options.nix { inherit lib systemdUtils; };

types = import ./systemd-types.nix { inherit lib systemdUtils; };

lib = import ./systemd/lib.nix { inherit lib config pkgs; };

unitOptions = import ./systemd/unit-options.nix { inherit lib systemdUtils; };

types = import ./systemd/types.nix { inherit lib systemdUtils; };

Maybe? I am not sure. Just an idea.

nixos/tests/systemd-initrd.nix

pkgs/build-support/kernel/make-initrd-ng.nix

ckiee

I started this review yesterday, there's a chance you already touched some of the things I reviewed.

Mostly code quality nitpicks, but more test{s,ing} for [this] core system component would be nice to see.

pkgs/build-support/kernel/make-initrd-ng.nix

pkgs/build-support/kernel/make-initrd-ng/src/main.rs

ckiee · 2022-03-21T02:32:58Z

nixos/modules/system/boot/systemd/initrd.nix

+           mapAttrs' (n: v: nameValuePair "${n}.path"    (pathToUnit          n v)) cfg.paths
+        // mapAttrs' (n: v: nameValuePair "${n}.service" (initrdServiceToUnit n v)) cfg.services
+        // mapAttrs' (n: v: nameValuePair "${n}.slice"   (sliceToUnit         n v)) cfg.slices
+        // mapAttrs' (n: v: nameValuePair "${n}.socket"  (socketToUnit        n v)) cfg.sockets
+        // mapAttrs' (n: v: nameValuePair "${n}.target"  (targetToUnit        n v)) cfg.targets
+        // mapAttrs' (n: v: nameValuePair "${n}.timer"   (timerToUnit         n v)) cfg.timers
+        // listToAttrs (map
+                     (v: let n = escapeSystemdPath v.where;
+                         in nameValuePair "${n}.mount" (mountToUnit n v)) cfg.mounts)
+        // listToAttrs (map
+                     (v: let n = escapeSystemdPath v.where;
+                         in nameValuePair "${n}.automount" (automountToUnit n v)) cfg.automounts);


This feels pretty repetitive but it does keep the code simpler vs. a higher level approach. Not sure what to make of that.

Yea it's not the first time this is repeated in nixos. I suppose I could factor it out into a function. Would that be good?

Yeah, that'd be nice! The generic-unit-ification being in what is effectively the consumer of the systemd lib seems weird.

nixos/tests/systemd-initrd.nix

nixos/lib/systemd-lib.nix

ElvishJerricco · 2022-03-21T14:27:45Z

#164016 has been merged, so I've rebased onto its last commit and removed the WIP label from this PR

dasJ

I haven't really looked at the Rust code apart from my one suggestion which helped me in debugging yesterday.

The approach is very promising and apart from the first commit not really splittable anymore. Most things I comment here are so that we nail some details right at the beginning instead of people starting to work on different aspects of the initrd while the core of it is still in flux.

I'm currently discussing stuff with @tfc to make sure we can get better test coverage. While I don't think this test needs more, I'd love future PRs with more features having the ability to add more stage 1 tests.

About the switch to the real system - I don't know if this PR should already contain code to switch to the new systemd directly without the shell script in between. While this approach in this PR already works of course, it's another thing people may make assumptions about ("this worked in systemd stage-1 before!!"). As you know, I'm experimenting with this detail and will provide my findings when they come up.

nixos/modules/system/boot/stage-1.nix

nixos/modules/system/boot/systemd/initrd.nix

nixos/tests/systemd-initrd.nix

pkgs/build-support/kernel/make-initrd-ng-tool.nix

pkgs/build-support/kernel/make-initrd-ng/src/main.rs

nixos/lib/systemd-types.nix

TODO: Auto formatting / partitioning / resizing

Make this reachable from pkgs.fakeNss. This is useful outside docker contexts, too. NixOS#164943 (comment)

We can just use serviceToUnit here.

NixOS#164943 (comment)

nixos/modules/system/boot/systemd/initrd.nix

nixos/lib/systemd-types.nix

nixos/modules/system/boot/systemd/initrd.nix

roberth

Let's use module composition instead of generated modules.
It keeps the types simple.

nixos/lib/systemd-lib.nix

nixos/modules/system/boot/systemd/initrd.nix

dasJ · 2022-04-01T08:45:11Z

@roberth can you take another look? I added two commits, one that does the type composition as requested and another one that adds type composition to the unit types to get rid of options that are not used

@roberth

As requested by @roberth, we now have an option similar to environment.etc. There's also extra store paths to copy and a way to suppress store paths to make customizations possible. We also link mount and umount to /bin to make recovery easier when something fails

This is more in line with what dracut does (it appends "Initramfs") and makes it clear where the boot is currently at when it hangs.

ElvishJerricco · 2022-04-01T17:30:46Z

@dasJ Those changes look great!

dasJ · 2022-04-02T01:59:17Z

If there are no more objections and pushes, I'll merge this in the next 24-48h

bobvanderlinden · 2022-04-03T19:44:11Z

Awesome! This unlocks so much potential improvements. Thanks a lot for all the work of everyone! 🥳🥳🥳

ncfavier · 2022-06-20T16:33:00Z

Tried this out on my laptop, it's almost usable except for two issues:

nixos-discourse · 2025-10-04T08:15:32Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/is-gpt-partition-automounting-possible-such-that-dev-disk-by-id-is-not-necessary/34790/4

ElvishJerricco requested a review from dasJ as a code owner March 20, 2022 08:04

ElvishJerricco mentioned this pull request Mar 20, 2022

[WIP]: initrd: Use systemd #120015

Closed

20 tasks

ElvishJerricco force-pushed the systemd-initrd-reuse-systemd-module branch 2 times, most recently from 57296ec to 5253f1c Compare March 20, 2022 09:03

SuperSandro2000 reviewed Mar 20, 2022

View reviewed changes

ElvishJerricco force-pushed the systemd-initrd-reuse-systemd-module branch from d6f3bd9 to 4b4e589 Compare March 20, 2022 20:05

mweinelt reviewed Mar 20, 2022

View reviewed changes

nixos/tests/systemd-initrd.nix Outdated Show resolved Hide resolved

mweinelt reviewed Mar 20, 2022

View reviewed changes

pkgs/build-support/kernel/make-initrd-ng.nix Outdated Show resolved Hide resolved

ElvishJerricco force-pushed the systemd-initrd-reuse-systemd-module branch from f97afcd to 954b88f Compare March 21, 2022 00:50

ckiee suggested changes Mar 21, 2022

View reviewed changes

ElvishJerricco force-pushed the systemd-initrd-reuse-systemd-module branch from 55a6600 to 9d86e36 Compare March 21, 2022 03:25

nixos: systemd: Split unit types into separate module

52c98fc

ElvishJerricco force-pushed the systemd-initrd-reuse-systemd-module branch from 5d8aeef to 89ccc7d Compare March 21, 2022 14:25

ElvishJerricco changed the title ~~[WIP]: initrd: Opt-in bare bones systemd-based initrd~~ initrd: Opt-in bare bones systemd-based initrd Mar 21, 2022

dasJ reviewed Mar 21, 2022

View reviewed changes

ElvishJerricco added 2 commits March 22, 2022 07:02

make-initrd-ng: init

d193ef8

nixos: systemd-lib: Make generateUnits general with default args

2511374

ElvishJerricco force-pushed the systemd-initrd-reuse-systemd-module branch from 89ccc7d to c730ab0 Compare March 22, 2022 11:02

ElvishJerricco added 3 commits March 22, 2022 21:28

initrd: Optional systemd-based initrd

2d4ebf1

systemd-initrd: Partially fix qemu-vm

be10e86

TODO: Auto formatting / partitioning / resizing

systemd-initrd: Add PATH to everything

1abf154

flokli added 2 commits March 24, 2022 18:47

fakeNss: move to toplevel

76d05df

Make this reachable from pkgs.fakeNss. This is useful outside docker contexts, too. NixOS#164943 (comment)

systemd-initrd: use pkgs.fakeNss, document why we need libnss_files.so

74bae06

ElvishJerricco requested a review from roberth as a code owner March 24, 2022 17:53

flokli added 2 commits March 24, 2022 18:57

systemd-initrd, systemd-lib: drop initrdServiceToUnit

e3083de

We can just use serviceToUnit here.

nixos/lib/systemd-lib.nix: move comment back down to packages

fc91cdb

NixOS#164943 (comment)

dasJ reviewed Mar 24, 2022

View reviewed changes

nixos/modules/system/boot/systemd/initrd.nix Outdated Show resolved Hide resolved

dasJ requested changes Mar 28, 2022

View reviewed changes

nixos/lib/systemd-types.nix Outdated Show resolved Hide resolved

nixos/modules/system/boot/systemd/initrd.nix Outdated Show resolved Hide resolved

roberth reviewed Mar 29, 2022

View reviewed changes

nixos/lib/systemd-lib.nix Outdated Show resolved Hide resolved

nixos/lib/systemd-lib.nix Outdated Show resolved Hide resolved

nixos/lib/systemd-lib.nix Outdated Show resolved Hide resolved

roberth reviewed Mar 31, 2022

View reviewed changes

nixos/modules/system/boot/systemd/initrd.nix Outdated Show resolved Hide resolved

roberth reviewed Mar 31, 2022

View reviewed changes

nixos/modules/system/boot/systemd/initrd.nix Outdated Show resolved Hide resolved

nixos/systemd-lib: Use module composition

1e5261f

dasJ force-pushed the systemd-initrd-reuse-systemd-module branch from 1ea78c4 to 1e5261f Compare April 1, 2022 07:57

dasJ force-pushed the systemd-initrd-reuse-systemd-module branch from 22ac161 to 0d69a99 Compare April 1, 2022 09:51

dasJ added 2 commits April 1, 2022 11:58

nixos/systemd-initrd: Remove unit options that don't work

b7c62b8

nixos/systemd-initrd: Make emergency access more flexible

c465c8d

dasJ force-pushed the systemd-initrd-reuse-systemd-module branch from 0d69a99 to c465c8d Compare April 1, 2022 09:58

dasJ added 2 commits April 1, 2022 13:26

nixos/systemd-stage-1: Append (Initrd) to /etc/initrd-release

7ebb4eb

This is more in line with what dracut does (it appends "Initramfs") and makes it clear where the boot is currently at when it hangs.

dasJ merged commit 7cdc4dd into NixOS:master Apr 3, 2022

arianvp mentioned this pull request Apr 3, 2022

PSA: systemd in initramfs #72401

Closed

yorickvP mentioned this pull request Apr 16, 2022

Request ZFS/LUKS passwords via Plymouth #88789

Closed

13 tasks

alyssais mentioned this pull request May 28, 2022

Expressions using builtins.fromTOML + builtins.readFile can't be evaluated on the installer image. #175125

Closed

infinisil mentioned this pull request Jan 31, 2023

nixos: add service to propagate status of activationScripts #208942

Open

13 tasks

infinisil added the 1.severity: significant Novel ideas, large API changes, notable refactorings, issues with RFC potential, etc. label Apr 19, 2023

Uh oh!

Comments

Conversation

ElvishJerricco commented Mar 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

How this compares to my previous attempts

What this approach will not do

Next steps

Things done

Uh oh!

SuperSandro2000 Mar 20, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ckiee left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ckiee Mar 21, 2022

Choose a reason for hiding this comment

Uh oh!

ElvishJerricco Mar 21, 2022

Choose a reason for hiding this comment

Uh oh!

ckiee Mar 21, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ElvishJerricco commented Mar 21, 2022

Uh oh!

dasJ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

roberth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dasJ commented Apr 1, 2022

Uh oh!

ElvishJerricco commented Apr 1, 2022

Uh oh!

dasJ commented Apr 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bobvanderlinden commented Apr 3, 2022

Uh oh!

ncfavier commented Jun 20, 2022

Uh oh!

nixos-discourse commented Oct 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

ElvishJerricco commented Mar 20, 2022 •

edited

Loading

dasJ commented Apr 2, 2022 •

edited

Loading