Dedup bcachefs filesystems before unlocking

[VlaDexa]

I have /, /nix, and /home on the same device, each mounted from a separate subvolume. Previously, I had to enter my password three times during boot, once for each mountpoint. This patch deduplicates bcachefs unlocks by grouping devices, so the password only needs to be entered once per device.

Things done

• Built on platform:
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• Tested, as applicable:
  • NixOS tests in nixos/tests.
  • Package tests at passthru.tests.
  • Tests in lib/tests or pkgs/test for functions and "core" functionality.
• Ran nixpkgs-review on this PR. See nixpkgs-review usage.
• Tested basic functionality of all binary files, usually in ./result/bin/.
• Nixpkgs Release Notes
  • Package update: when the change is major or breaking.
• NixOS Release Notes
  • Module addition: when adding a new NixOS module.
  • Module update: when the change is significant.
• Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

---

Add a 👍 reaction to pull requests you find important.

[Frontear]

A change like this should come with an accompanying release note, it's a pretty useful change and I'd hate for it to go unnoticed.

[VlaDexa]

Not sure if this wording is the clearest, because there is a clevis way to unlock a filesystem, so this note can be mistaken by some to mean that their clevis setups will also ask for a password, but I don't have any better ideas how to describe this without overloading a reader with all the caveats

[ElvishJerricco]

I kind of think this isn't the right level to do this at. I'm going to ignore scripted stage 1 for the moment, because the options we have for systemd stage 1 and for stage 2 are much broader and more interesting.

For one thing, the thing we actually want to dedup on is the FS UUID, not the devices and not the mountpoints. Technically, robustly speaking, this information is only available at runtime.

Separately, systemd-cryptsetup avoids password reentry among multiple LUKS devices by simply sharing a kernel keyring key among them, with a common name: cryptsetup.

Finally, there are PAM modules that read the cryptsetup keyring key to auto-unlock things like GNOME keyring.

So I think what I'm getting at is... For systemd stage 1 and stage 2, we should simply change the systemd-ask-password call to use --keyname=cryptsetup and --accept-cached. That way we can share an encryption passphrase among multiple file systems, LUKS, and userspace keyrings. That way it isn't necessary to attempt to deduplicate the services at eval time when we don't have enough information. This will require adding KeyringMode=shared to the services.

As for scripted stage 1... Well to be frank we're hoping to put that on life support in general here soon so I'd rather just advise people to switch to systemd stage 1.

[VlaDexa]

For one thing, the thing we actually want to dedup on is the FS UUID, not the devices and not the mountpoints.

A user can specify their fs by /dev/disk/by-uuid/* and this will work just fine. It will also work for all the users of disko, because it does exactly that.

Technically, robustly speaking, this information is only available at runtime.

Well, we do have access to runtime scripting, but I assume this doesn't need this much complexity nor do I want to implement fetching bcachefs's UUID's and deduping all in bash.

For systemd stage 1 and stage 2, we should simply change the systemd-ask-password call to use --keyname=cryptsetup and --accept-cached

This allows for only one password, which won't be compatible with a user having multiple bcachefs filesystems with different passphrases.

[ElvishJerricco]

A user can specify their fs by /dev/disk/by-uuid/* and this will work just fine. It will also work for all the users of disko, because it does exactly that.

So does nixos-generate-config. That isn't the point.

This allows for only one password, which won't be compatible with a user having multiple bcachefs filesystems with different passphrases.

The way systemd-cryptsetup handles this is by falling back to a prompt if the cached password doesn't work.

I think there's a lot of benefit to aligning with how upstream does this for LUKS devices, so I still think we should do that.

[boomshroom]

If you're defining the mount-points in Nix, why would you not be using an identical device value for each when you can just put it in a let-binding? The autoScrub module already dedupes by device field, so that should be completely fine.

If deduping by UUID is really what you want, but the information is only available at runtime, then you could make a service that detects bcachefs filesystems at runtime and then starts a single parameterised unit passing the UUID in. Then again, this still might be overcomplicating things, since it seems kind of silly to have multiple bcachefs filesystems in the first place given it's subvolumes and multi-device capabilities.

[ElvishJerricco]

Deduping by the device is a reasonable, albeit approximate, proxy for the desirable semantics of deduping by the UUID. The further point I've been making, and continue to make, is that there are additional benefits to doing it the way that systemd-cryptsetup / LUKS do it, and I think we should do that too.

[ElvishJerricco]

Looks like bcachefs-tools is going to fix this in the mount helper next release, which will allow us to remove our custom systemd service entirely. koverstreet/bcachefs-tools#413

[beviu]

About that PR, removing Nixpkgs' bcachefs unlock service was exactly my goal 😄. I've tried it with this change and there are some issues:

• The mount unit needs to have its KeyringMode set to shared or inherit. EDIT: This is not needed. It's already the default since core: change KeyringMode= to "shared" by default for non-service unit… systemd/systemd#8172.

• The prompt from bcachefs-tools doesn't say what is being mounted unlike the current unlock service. I've tried putting the mount destination in the prompt but when we are mounting a subvolume using the X-mount.subdir option, it gets the wrong path because util-linux's mount handles the X-mount.subdir option by mounting the FS on a temporary location (/run/mount/tmptgt) first and then bind mounting the subdirectory to the real target (I think).

  I wonder if bcachefs could have its own subvol option directly like btrfs. Then we could use that instead of X-mount.subdir and the mount destination would be meaningful again.

• Mounts sometimes fail with an EBUSY error. I can still press enter to let systems retry and and it boots eventually. This is worse when the FS takes longer to mount (eg. when it does fsck).