Refactor module options and support rootless k3s & nix-snapshotter

[elpdt852]

Fixes #83

Highlights

• Provides NixOS & Home Manager modules for rootless Kubernetes (k3s) + nix-snapshotter
• Bumps version to 0.2.0
• Separate nix run .#vm and nix run .#vm-rootless
• Add integration tests for k3s, k3s-external, k3s-rootless
• Separate preload-container service into independent modules
• Pin k3s to v1.27.9+k3s1 with patches to enable embedded nix-snapshotter
  • k3s PR to support nix-snapshotter: Add nix-snapshotter support to the embedded containerd k3s-io/k3s#9319
• Add k8sResources flake perSystem output and plumb into specialArgs

New options

• Services preload-containerd & preload-containerd.rootless:

  config.services.preload-containerd = {
    enable = true;
    targets = [{
      archives = [ pkgs.nix-snapshotter.buildImage { /* ... */ } ];
      namespace = "k8s.io";
      address = "/run/k3s/containerd/containerd.sock";
    }];
  };

• New options for k3s & new service k3s.rootless:

  config.services.k3s = {
    enable = true;
    # Sets the snapshotter for embedded containerd.
    snapshotter = "nix";
    # Sets KUBECONFIG env var to point to k3s.
    setKubeConfig = true;
    # Sets CONTAINERD_* env vars to point to k3s embedded containerd.
    setEmbeddedContainerd = true;
  }

• New options for containerd & containerd.rootless:

  config.virtualisation.containerd = {
    enable = true;
    # Enable integration with nix-snapshotter.
    nixSnapshotterIntegration = true;
    # Set the CONTAINERD_* env vars, but also set automatically by
    # `nixSnapshotterIntegration` or by `services.k3s.setEmbeddedContainerd`.
    setAddress = "/run/containerd/containerd.sock";
    setNamespace = "default";
    setSnapshotter = "nix";
  }

• New option only for NixOS module containerd:

  config.virtualisation.containerd = {
    enable = true;
    # Enable integration with k3s. This is mutually exclusive with setting
    # `services.k3s.snapshotter` and `services.k3s.setEmbeddedContainerd`.
    k3sIntegration = true;
  };

Migration guide v0.1.x -> v0.2.0

• Removed options.services.nix-snapshotter.setContainerdSnapshotter

  # v0.1.x
  services.nix-snapshotter = {
    enable = true;
    setContainerdSnapshotter = true;
  };
  
  # v0.2.0 (same for rootless)
  virtualisation.containerd = {
    enable = true;
    nixSnapshotterIntegration = true;
  };
  services.nix-snapshotter = {
    enable = true;
  };

• Removed options.services.nix-snapshotter.preloadContainerdImages

  # v0.1.x
  services.nix-snapshotter = {
    enable = true;
    preloadContainerdImages = [ pkgs.nix-snapshotter.buildImage { /* ... */ } ];
  };
  
  # v0.2.0 (same for rootless)
  virtualisation.containerd = {
    enable = true;
    nixSnapshotterIntegration = true;
  }
  services.nix-snapshotter = {
    enable = true;
  };
  services.preload-containerd = {
    targets = [{
      archives = [ pkgs.nix-snapshotter.buildImage { /* ... */ } ];
    }];
  };

[RobbieBuxton]

I had a read through and as far as I can tell, it all looks fine. I will admit most of the Kubernetes section went over my head as I'm not very familiar with it so apologies if I've missed anything. Otherwise, awesome work and it looks like it will be a great addition! 🎉🎉🎉