Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions nixos/doc/manual/release-notes/rl-2505.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,10 @@

- `services.gitea` now supports CAPTCHA usage through the `services.gitea.captcha` variable.

- The GRUB bootloader (`boot.loader.grub`) now generates [boot loader entries](https://uapi-group.org/specifications/specs/boot_loader_specification/).
These files are used by userspace tools (for example, `bootctl`) to inspect the bootloader status, getting the default boot entry, the path of the kernel binary, etc.
As a consequence, `systemctl kexec` now works automatically: specifying the kernel and its arguments with `kexec --load` is no longer required.

- `bind.cacheNetworks` now only controls access for recursive queries, where it previously controlled access for all queries.

- [`services.mongodb.enableAuth`](#opt-services.mongodb.enableAuth) now uses the newer [mongosh](https://github.com/mongodb-js/mongosh) shell instead of the legacy shell to configure the initial superuser. You can configure the mongosh package to use through the [`services.mongodb.mongoshPackage`](#opt-services.mongodb.mongoshPackage) option.
Expand Down
14 changes: 14 additions & 0 deletions nixos/modules/system/boot/loader/grub/grub.nix
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ let
then realGrub.override { efiSupport = cfg.efiSupport; }
else null;

bootPath = if cfg.mirroredBoots != [ ]
then (builtins.head cfg.mirroredBoots).path
else "/boot";
Comment on lines +53 to +55
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part confused me for several reasons. I thought there was an error here, but after looking closer, I see there isn't. But I do still have some nitpicks about what confused me.

  • There is an assertion that mirroredBoots cannot be empty, so I don't think we need to account for that case with a bogus value.
  • I think there should be a comment explaining why just any of the mirroredBoots is acceptable.
  • We shouldn't reuse the same bootPath variable name that is already used nearby in this module


f = x: optionalString (x != null) ("" + x);

grubConfig = args:
Expand Down Expand Up @@ -756,6 +760,16 @@ in

environment.systemPackages = mkIf (grub != null) [ grub ];

# Link /boot under /run/boot-loder-entries to make
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: loder -> laoder

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: laoder -> loader

# systemd happy even on non-EFI system
systemd.mounts = lib.optional (!cfg.efiSupport) {
what = bootPath;
where = "/run/boot-loader-entries";
type = "none";
options = "bind";
requiredBy = [ "local-fs.target" ];
};

boot.loader.grub.extraPrepareConfig =
concatStrings (mapAttrsToList (n: v: ''
${pkgs.coreutils}/bin/install -Dp "${v}" "${efi.efiSysMountPoint}/"${escapeShellArg n}
Expand Down
77 changes: 75 additions & 2 deletions nixos/modules/system/boot/loader/grub/install-grub.pl
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,15 @@ sub runCommand {

print STDERR "updating GRUB 2 menu...\n";

make_path("$bootPath/grub", { mode => 0700 });
# Make GRUB directory
make_path("$bootPath/grub", { mode => 0700 });

# Make BLS entries directory, see addBLSEntry
make_path("$bootPath/loader/entries", { mode => 0700 });
writeFile("$bootPath/loader/entries.srel", "type1");

# and a temporary one for new entries
make_path("$bootPath/loader/entries.tmp", { mode => 0700 });

# Discover whether the bootPath is on the same filesystem as / and
# /nix/store. If not, then all kernels and initrds must be copied to
Expand Down Expand Up @@ -460,6 +468,7 @@ sub copyToKernelsDir {
}

sub addEntry {
# Creates a Grub menu entry for a given system
my ($name, $path, $options, $current) = @_;
return unless -e "$path/kernel" && -e "$path/initrd";

Expand Down Expand Up @@ -521,6 +530,58 @@ sub addEntry {
$conf .= "}\n\n";
}

sub addBLSEntry {
# Creates a Boot Loader Specification[1] entry for a given system.
# The information contained in the entry mirrors a boot entry in GRUB menu.
#
# [1]: https://uapi-group.org/specifications/specs/boot_loader_specification
my ($prof, $spec, $gen, $link) = @_;

# collect data from system
my %bootspec = %{decode_json(readFile("$link/boot.json"))->{"org.nixos.bootspec.v1"}};
my $date = strftime("%F", localtime(lstat($link)->mtime));
my $kernel = $bootspec{kernel} =~ s@$storePath/@@r =~ s@/@-@r;
my $initrd = $bootspec{initrd} =~ s@$storePath/@@r =~ s@/@-@r;
my $kernelParams = readFile("$link/kernel-params");
my $machineId = readFile("/etc/machine-id");

if ($grubEfi eq "" && !$copyKernels) {
# workaround for https://github.com/systemd/systemd/issues/35729
make_path("$bootPath/kernels", { mode => 0755 });
symlink($bootspec{kernel}, "$bootPath/kernels/$kernel");
symlink($bootspec{initrd}, "$bootPath/kernels/$initrd");
$copied{"$bootPath/kernels/$kernel"} = 1;
$copied{"$bootPath/kernels/$initrd"} = 1;
}
Comment on lines +548 to +555
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The correctness of this is highly suspect. BLS doesn't specify how to interpret files in the tree that are symlinks to an absolute path like this. What root should be the root for resolving those symlinks? Obviously, in the case of running bootctl list or systemctl kexec on the booted NixOS system, this happens to use the system's root FS as the root. But that isn't a given in the BLS and doesn't really make sense for an arbitrary BLS boot loader.

Additionally, why must this not be an EFI setup to take this code path? You can still do copyKernels = false; on an EFI system, if $bootPath is on the root fs and efiSysMountPoint is something else.

Copy link
Copy Markdown
Contributor Author

@rnhmjoj rnhmjoj Mar 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is working around a bug in systemd, of course it's not "correct" according to the BLS spec.
See what I wrote in the issue I linked:

systemctl kexec expects the paths of the kernel and initrd to be relative to /run/boot-loader-entries/ and fails with File not found. Without an ESP I would expect the path to be absolute (relative to /), given the updated specification even says:

linux specifies the Linux kernel image to execute. The value is a path relative to the root of the file system containing the boot entry snippet itself.

Additionally, why must this not be an EFI setup to take this code path? You can still do copyKernels = false; on an EFI system, if $bootPath is on the root fs and efiSysMountPoint is something else.

It's unrelated to copyKernels: it has to do with linking the entries to /run/boot-loader-entries on non-EFI systems to expose the entries to systemd. In this (edge) case the BLS taken literally says to interpret paths relative to /, while systemd interprets them relative to /run/boot-loader-entries.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the reasoning for this. It's still highly suspect, and should probably not be considered proper BLS.

And yes, it does have to do with copyKernels. Try it out. I did. If you make an EFI system with /boot on the root fs and efiSysMountPoint = "/efi"; or something similar, this will produce a broken result that does not work. The BLS files will point to files that should be in the boot tree, but are not, because you disabled creating these symlinks in the boot tree for EFI systems.

IMO the feature simply shouldn't work without copyKernels enabled, because like I said, these symlinks don't make sense in the context of BLS. The fact that it doesn't work with copyKernels disabled on EFI systems is just something else that's wrong.


# fill in the entry
my $extras = join(' ', $prof = $prof ne "system" ? " [$prof] " : "",
$spec = $spec ne "" ? " ($spec) " : "");
my $entry = <<~END;
title @distroName@$extras
sort-key nixos
version Generation $gen $bootspec{label}, built on $date
linux kernels/$kernel
initrd kernels/$initrd
options init=$bootspec{init} $kernelParams
END
$entry .= "machine-id $machineId" if defined $machineId;

# entry file basename
my $name = join("-", grep { length $_ > 0 }
"nixos", $prof ne "system" ? $prof : "",
"generation", $gen,
$spec ? "specialisation-$spec" : "");

# write entry to the temp directory
writeFile("$bootPath/loader/entries.tmp/$name.conf", $entry);

# mark the default entry
if (readlink($link) eq $defaultConfig) {
writeFile("$bootPath/loader/loader.conf", "default $name.conf");
}
Comment on lines +579 to +582
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, loader.conf is not actually part of BLS. That's specifically a systemd-boot thing.

But much more importantly, if another OS owns the loader.conf file on this machine, overwriting it is not acceptable. We cannot do that. We take NixOS's ownership of loader.conf as a given in systemd-boot-builder.py, but that is not a given with grub, because it doesn't use it. It is also a significant change in behavior for NixOS's existing grub support.

}

sub addGeneration {
my ($name, $nameSuffix, $path, $options, $current) = @_;

Expand Down Expand Up @@ -592,12 +653,18 @@ sub addProfile {
warn "skipping corrupt system profile entry ‘$link’\n";
next;
}
my $gen = nrFromGen($link);
my $date = strftime("%F", localtime(lstat($link)->mtime));
my $version =
-e "$link/nixos-version"
? readFile("$link/nixos-version")
: basename((glob(dirname(Cwd::abs_path("$link/kernel")) . "/lib/modules/*"))[0]);
addGeneration("@distroName@ - Configuration " . nrFromGen($link), " ($date - $version)", $link, $subEntryOptions, 0);
addGeneration("@distroName@ - Configuration " . $gen, " ($date - $version)", $link, $subEntryOptions, 0);

addBLSEntry(basename($profile), "", $gen, $link);
foreach my $spec (glob "$link/specialisation/*") {
addBLSEntry(basename($profile), $spec, $gen, $spec);
}
}

$conf .= "}\n";
Expand All @@ -611,6 +678,12 @@ sub addProfile {
addProfile $profile, "@distroName@ - Profile '$name'";
}

# Atomically replace the BLS entries directory
my $entriesDir = "$bootPath/loader/entries";
rename $entriesDir, "$entriesDir.bak" or die "cannot rename $entriesDir to $entriesDir.bak: $!\n";
rename "$entriesDir.tmp", $entriesDir or die "cannot rename $entriesDir.tmp to $entriesDir: $!\n";
rmtree "$entriesDir.bak" or die "cannot remove $entriesDir.bak: $!\n";
Comment on lines +681 to +685
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is similar to the loader.conf issue, but more severe. We do not necessarily own all the entries on the system. We can't just delete the whole entries directory. The systemd-boot-builder.py takes great care to only remove entries and files known to be owned by NixOS.


# extraPrepareConfig could refer to @bootPath@, which we have to substitute
$extraPrepareConfig =~ s/\@bootPath\@/$bootPath/g;

Expand Down
2 changes: 1 addition & 1 deletion nixos/tests/all-tests.nix
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ in {
greetd-no-shadow = handleTest ./greetd-no-shadow.nix {};
grocy = handleTest ./grocy.nix {};
grow-partition = runTest ./grow-partition.nix;
grub = handleTest ./grub.nix {};
grub = import ./grub.nix { inherit pkgs runTest; };
guacamole-server = handleTest ./guacamole-server.nix {};
guix = handleTest ./guix {};
gvisor = handleTest ./gvisor.nix {};
Expand Down
169 changes: 116 additions & 53 deletions nixos/tests/grub.nix
Original file line number Diff line number Diff line change
@@ -1,60 +1,123 @@
import ./make-test-python.nix ({ lib, ... }: {
name = "grub";
{ pkgs, runTest }:

meta = with lib.maintainers; {
maintainers = [ rnhmjoj ];
};
{
# Basic GRUB setup with BIOS and a password
basic = runTest {
name = "grub-basic";
meta.maintainers = with pkgs.lib.maintainers; [ rnhmjoj ];

nodes.machine = { ... }: {
virtualisation.useBootLoader = true;
boot.loader.timeout = null;
boot.loader.grub = {
enable = true;
users.alice.password = "supersecret";
# OCR is not accurate enough
extraConfig = "serial; terminal_output serial";
};
};

testScript = ''
def grub_login_as(user, password):
"""
Enters user and password to log into GRUB
"""
machine.wait_for_console_text("Enter username:")
machine.send_chars(user + "\n")
machine.wait_for_console_text("Enter password:")
machine.send_chars(password + "\n")


def grub_select_all_configurations():
"""
Selects "All configurations" from the GRUB menu
to trigger a login request.
"""
machine.send_monitor_command("sendkey down")
machine.send_monitor_command("sendkey ret")


machine.start()

# wait for grub screen
machine.wait_for_console_text("GNU GRUB")

grub_select_all_configurations()
with subtest("Invalid credentials are rejected"):
grub_login_as("wronguser", "wrongsecret")
machine.wait_for_console_text("error: access denied.")

grub_select_all_configurations()
with subtest("Valid credentials are accepted"):
grub_login_as("alice", "supersecret")
machine.send_chars("\n") # press enter to boot
machine.wait_for_console_text("Linux version")

with subtest("Machine boots correctly"):
machine.wait_for_unit("multi-user.target")
'';
};

# Test boot loader entries on EFI
bls-efi = runTest {
name = "grub-bls-efi";
meta.maintainers = with pkgs.lib.maintainers; [ rnhmjoj ];

nodes.machine = { pkgs, ... }: {
virtualisation.useBootLoader = true;
virtualisation.useEFIBoot = true;
boot.loader.efi.canTouchEfiVariables = true;
boot.loader.grub.enable = true;
boot.loader.grub.efiSupport = true;
};

testScript = ''
with subtest("Machine boots correctly"):
machine.wait_for_unit("multi-user.target")

with subtest("Boot entries are installed"):
entries = machine.succeed("bootctl list")
print(entries)
error = "NixOS boot entry not found in bootctl list."
assert "version: Generation 1" in entries, error

nodes.machine = { ... }: {
virtualisation.useBootLoader = true;
with subtest("systemctl kexec can detect the kernel"):
machine.succeed("systemctl kexec --dry-run")

boot.loader.timeout = null;
boot.loader.grub = {
enable = true;
users.alice.password = "supersecret";
with subtest("systemctl kexec really works"):
machine.execute("systemctl kexec", check_return=False)
machine.connected = False
machine.connect()
machine.wait_for_unit("multi-user.target")
'';
};

# Test boot loader entries on BIOS
bls-bios = runTest {
name = "grub-bls-bios";
meta.maintainers = with pkgs.lib.maintainers; [ rnhmjoj ];

# OCR is not accurate enough
extraConfig = "serial; terminal_output serial";
nodes.machine = { pkgs, ... }: {
virtualisation.useBootLoader = true;
boot.loader.grub.enable = true;
};

testScript = ''
with subtest("Machine boots correctly"):
machine.wait_for_unit("multi-user.target")

with subtest("Boot entries are installed"):
machine.succeed("test -f /boot/loader/entries/nixos-generation-1.conf")

with subtest("systemctl kexec can detect the kernel"):
machine.succeed("systemctl kexec --dry-run")

with subtest("systemctl kexec really works"):
machine.execute("systemctl kexec", check_return=False)
machine.connected = False
machine.connect()
machine.wait_for_unit("multi-user.target")
'';
};

testScript = ''
def grub_login_as(user, password):
"""
Enters user and password to log into GRUB
"""
machine.wait_for_console_text("Enter username:")
machine.send_chars(user + "\n")
machine.wait_for_console_text("Enter password:")
machine.send_chars(password + "\n")


def grub_select_all_configurations():
"""
Selects "All configurations" from the GRUB menu
to trigger a login request.
"""
machine.send_monitor_command("sendkey down")
machine.send_monitor_command("sendkey ret")


machine.start()

# wait for grub screen
machine.wait_for_console_text("GNU GRUB")

grub_select_all_configurations()
with subtest("Invalid credentials are rejected"):
grub_login_as("wronguser", "wrongsecret")
machine.wait_for_console_text("error: access denied.")

grub_select_all_configurations()
with subtest("Valid credentials are accepted"):
grub_login_as("alice", "supersecret")
machine.send_chars("\n") # press enter to boot
machine.wait_for_console_text("Linux version")

with subtest("Machine boots correctly"):
machine.wait_for_unit("multi-user.target")
'';
})
}
3 changes: 3 additions & 0 deletions nixos/tests/nixos-rebuild-install-bootloader.nix
Original file line number Diff line number Diff line change
Expand Up @@ -71,5 +71,8 @@ import ./make-test-python.nix ({ pkgs, lib, withNg ? false, ... }: {
# at this point we've tested regression #262724, but haven't tested the bootloader itself
# TODO: figure out how to how to tell the test driver to start the bootloader instead of
# booting into the kernel directly.

with subtest("New boot entry has been added"):
machine.succeed("test -f /boot/loader/entries/nixos-generation-2.conf")
'';
})