nixos-rebuild: init build-image subcommand

[phaer]

I split preparation for this into smaller PRs, after feedback:

• make-disk-image: Allow passing of image baseName #359326
• image/images: init #359328
• virtualisation: Use system.build.image, normalize file names #359339
• image/images: Adapt remaining images to system.build.image & normalized filenames,  #359345
• nixos-rebuild: init build-image subcommand #347275 (this one)

And force-pushed this one to contain only changes to the manual & nixos-rebuild

This PR would introduce a new nixos-rebuild build-image sub-command to generate disk images for a given nixos configuration. I would nevertheless appreciate any feedback on the design and functionality!

It's part of an attempt to bring the functionality of @Lassulus https://github.com/nix-community/nixos-generators upstream, i.e. allow users to run

nixos-rebuild build-image --flake .#my-system --image-variant digital-ocean

to create a disk-image for digital-ocean. Or run it without arguments to get a list of available variants:

$ nixos-rebuild build-image --flake .#my-system
Please specify one of the following supported image variants via --image-variant:

amazon
azure
digital-ocean
google-compute
hyperv
iso
iso-installer
kexec
kubevirt
linode
lxc
lxc-metadata
oci
openstack
openstack-zfs
proxmox
proxmox-lxc
qemu
qemu-efi
raw
raw-efi
sd-card
vagrant-virtualbox
virtualbox
vmware

options

Each image/variant gains a new system.build.image attribute, which is just an alias for existing system.build-options, but provides a unified interface. Users can just import proxmox or amazon and build an image without having to wonder whether its system.build.amazon or system.build.VMA.

Also, each image imports image/file-options.nix which provides unified options for image.baseName, image.extension and image.fileName. Those are not only useful for customization purposes, but also to be able to predict the specific image name at eval time instead of relying on shell globs to find the file inside $out.

A collection of images is available in system.build.images.${variant} after importing ./image/images.nix. They can be customized per variant, via image.modules.${variant} and expose their own config for debbugging in system.build.images.${variant}.passthru.config.

name diffs

The following is a list of image file names, that would be changed upon a merge.
The goal here is to use a unified, predictable scheme for generated image filenames while still allowing consumers to change them centrally.

An exception is the proxmox image, where the filename is used to set default name and id of the imported VM, so changing it would affect semantics of the import. Judging existing file names, this should not be the case for others.

amazon: nixos-amazon-image-24.11pre-git-x86_64-linux.vhd -> nixos-image-amazon-24.11pre-git-x86_64-linux.vhd
azure: disk.vhd -> nixos-image-azure-24.11pre-git-x86_64-linux.vhd
digital-ocean: nixos.qcow2.gz -> nixos-image-digital-ocean-24.11pre-git-x86_64-linux.qcow2.gz
google-compute: nixos-image-24.11pre-git-x86_64-linux.raw.tar.gz -> nixos-image-google-compute-24.11pre-git-x86_64-linux.raw.tar.gz
hyperv: nixos-24.11pre-git-x86_64-linux.vhdx -> nixos-image-hyperv-24.11pre-git-x86_64-linux.vhdx
iso: nixos.iso -> nixos-image-24.11pre-git-x86_64-linux.iso
iso-installer: nixos-24.11pre-git-x86_64-linux.iso -> nixos-image-24.11pre-git-x86_64-linux.iso
kubevirt: nixos.qcow2 -> nixos-image-kubevirt-24.11pre-git-x86_64-linux.qcow2
linode: nixos-image-24.11pre-git-x86_64-linux.img.gz -> nixos-image-linode-24.11pre-git-x86_64-linux.img.gz
lxc: nixos-system-x86_64-linux.tar.xz -> nixos-image-lxc-24.11pre-git-x86_64-linux.tar.xz
lxc-metadata: nixos-system-x86_64-linux.tar.xz -> nixos-image-lxc-metadata-24.11pre-git-x86_64-linux.tar.xz
oci: nixos.qcow2 -> nixos-image-oci-24.11pre-git-x86_64-linux.qcow2
openstack: nixos.qcow2 -> nixos-image-openstack-24.11pre-git-x86_64-linux.qcow2
openstack-zfs: nixos-openstack-image-24.11pre-git-x86_64-linux.root.qcow2 -> nixos-image-openstack-zfs-24.11pre-git-x86_64-linux.root.qcow2
proxmox: vzdump-qemu-nixos-24.11pre-git.vma.zst -> vzdump-qemu-nixos-24.11pre-git.vma.zst
proxmox-lxc: nixos-system-x86_64-linux.tar.xz -> nixos-image-lxc-proxmox-24.11pre-git-x86_64-linux.tar.xz
sd-card: nixos-sd-image-24.11pre-git-x86_64-linux.img.zst -> nixos-image-sd-card-24.11pre-git-x86_64-linux.img.zst
vagrant-virtualbox: pi5fh4a6vwi40ali7bk0a8kldiakbsnn-nixos-image-vagrant-virtualbox-24.11pre-git-x86_64-linux.ova
virtualbox: nixos-24.11pre-git-x86_64-linux.ova -> nixos-image-virtualbox-24.11pre-git-x86_64-linux.ova
vmware: nixos-24.11pre-git-x86_64-linux.vmdk -> nixos-image-vmware-24.11pre-git-x86_64-linux.vmdk

closure diffs

The following is the output of nix-diff when ran on each of the images on a) the latest master commit before I branched off, and b) with this branch.

Most of them are just the propagated nixos.system.tags changes plus the changed nixpkgs path. There are 5 exceptions

1. iso: includes nix-rebuild, so changes to that are included as well
2. iso-installer: same as iso
3. proxmox: same as iso
4. sd-card: uses different builder, but it still looks like it's just the system path
5. vagrant-virtualbox: environment change due to image file change

see https://gist.github.com/phaer/f2fe4d863f831141b80938a1f6c702df because github says this comment is too long "body too long" :D

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[Aleksanaa]

I've been wanting to do this for a long time, thanks

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-rebuild-ng-a-nixos-rebuild-rewrite/55606/25

[arianvp]

How does this compare to the already existing nix bundle command? They seem to serve similar purposes. Though it seems nix bundle never really took off. I just find it a little bit odd we have a dedicated command for "building self-contained images" in the nix cli but nobody seems to use it :')

I could e.g. imagine:

nix bundle --bundler images#amazon  .#nixosConfigurations.myconfig.config.system.build.toplevel
nix bundle --bundler images#digitalocean  .#nixosConfigurations.myconfig.config.system.build.toplevel
nix bundle --bundler images#docker  nixpkgs#nginx

[Aleksanaa]

These two do not conflict and the implementation in nixos-rebuild is very minimal. And after all the new cli isn't enabled by default, the configuration may not be in flake and we still have nix_2_3 users.

[phaer]

I think a nix bundle interface is a good idea and could be quite easy to implement! I'd prefer to do so in a separate PR and would still like to add this proposed nixos-rebuild build-image command.

In addition to the arguments @Aleksanaa brought up, I believe current policy is not use new-style CLI commands in the manual and therefore relying on nix bundle would hurt discoverability.

[phaer]

Happy to give both, an implementation of this feature for nixos-rebuild-ng, and a nix-bundle interface a go - but I'd like to with that wait until #359345 and/or this PR are merged to avoid too many PRs & rebases in parallel in case we decide to change anything here.

So if anyone was planning on giving this a review or even test resulting images on the platforms you use, it would be great to do so soon ;)

[zimbatm]

If there is nothing else, let's merge this before it conflicts again?

[Mic92]

If there is nothing else, let's merge this before it conflicts again?

Sure go ahead, if you have tested it :)

[phaer]

Resolved the merge conflicts in release notes, didn't change anything in the code.
Good to merge in my, arguably biased, opinion ;)

[SpiderUnderUrBed]

Could support be added for qcow images directly? Like as a image variant

[SpiderUnderUrBed]

Also, there is a issue where large disks sizes are stuck here:

[spiderunderurbed@daspidercave:~/nixnuc/nixmoxer]$ nixos-rebuild build-image --image-variant raw --flake .#myvm
building the system configuration...
warning: Git tree '/home/spiderunderurbed/nixnuc/nixmoxer' is dirty
warning: Git tree '/home/spiderunderurbed/nixnuc/nixmoxer' is dirty
evaluation warning: system.stateVersion is not set, defaulting to 25.11. Read why this matters on https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion.
[1/11/12 built, 0.0 MiB DL] building nixos-disk-image: [    2.253672] random: crng init done

is there any way to pass any override to the vm or process thats building the vm? there are some qemu flags that could stop it for being stuck at:
random: crng init done for so long
its 100gbs but im sure some optimization can be done via flags, or some config option.

[phaer]

I think it'd preferable to discuss such issues either in discourse or a new issue, but not necessarily 8 months later in a long closed PR

Could support be added for qcow images directly? Like as a image variant

That's supported, as qemu and qemu-efi variants.

is there any way to pass any override to the vm or process thats building the vm?

Not really, except for overriding the specific system.build.image attribute all together, as make-disk-image.nix does not allow such overrides IMO.