lib.types: Add deferredModule

[roberth]

Description of changes

Great for submodule defaults. Example: network.defaults in NixOps.

Accepts modules, returns a list of modules, with a good default module location.

I'll quote the docs below. Also look at the example.

types.deferredModule

Whereas submodule represents an option tree, deferredModule represents a module value, such as a module file or a configuration.

It can be set multiple times.

Module authors can use its value, which is always a list of module values, in imports or in submoduleWith's modules parameter.
Note that imports must be evaluated before the module fixpoint. Because of this, deferred modules can only be imported into "other" fixpoints, such as submodules.

One use case for this type is the type of a "default" module that allow the user to affect all submodules in an attrsOf submodule at once. This is more convenient and discoverable than expecting the module user to type-merge with the attrsOf submodule option.

Refs NixOS/nixops#1508

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.05 Release Notes (or backporting 21.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[roberth]

cc @infinisil

[roberth]

ping @infinisil

[infinisil]

Shouldn't this also allow paths?

[infinisil]

And how about also allowing lists of modules, instead of just a single one

[roberth]

paths

done

lists

This should be covered by lib.mkMerge. I don't expect lists to be common enough to warrant the duplication.

[infinisil]

Hmm, it bothers me a bit that the result of this type is a list, but the input is a single element. This is in contrast to essentially all other types whose input and output types are generally the same.

Also, if the input type is also a list (which I think also makes more sense, since it's how imports, the modules argument for evalModules/submoduleWith works), then it might make sense to use the listOf type to implement it.

Which then brings me to my general dislike of lists, the main problem being that it's not possible to remove entries once added. I guess in this case it's somewhat fine though, because I believe disabledModules can be used to essentially remove modules from the list, though finding out how to actually refer to modules in the list is pretty hard.

[roberth]

This reminds me of Html in Elm, which is also an inherently monoidal type that they're constantly putting in lists (the free monoid) for no good reason. Similar to their [Html] -> Html operation (<div> or <span>), our concat operation (x: { imports = x; }) is also leaky. It leaks through in error messages with anon--something-something strings.

I've embraced the leakiness and I've work with the rules:

1. keep the conversions to a minimum (they're leaky)
2. keep the interface simple
   • we don't have a use case for defining lists of modules, and it's still covered by mkMerge
   • submodule doesn't support lists, so why should deferredModule?

It'd be a different store if lists of modules were always considered to be valid modules. This would include the submodule argument, submodule definitions, imports items, evalModules { modules } items, etc.

my general dislike of lists

I share the dislike, as they imply ordering, which is often not necessary or desirable in a declarative system. In this case though, they are only as unnecessary and undesirable as in the rest of the modules system - not something we can expect to solve here.

disabledModules

The best solution to removals is not to include the things in the first place.
disabledModules should operate just fine on the parent module and on path-imported ones. Anonymous ones will be equally problematic as anonymous submodules.

though finding out how to actually refer to modules in the list is pretty hard.

Do you mean list indexing? That's not something I'd think of, but I guess someone could try - and eventually fail horribly.

I guess this might be reason to make it slightly harder? Although x.imports !! y isn't much of an obstacle over x !! y., especially compared to writing the function that only removes one element. I don't think we have that in lib?

I suppose I'm not entirely opposed to returning a single module. Given my argumentation, do you still think I should do it? (or something else)

[infinisil]

I suppose I'm not entirely opposed to returning a single module. Given my argumentation, do you still think I should do it? (or something else)

I haven't considered that, but I really do like that idea! By returning a single module we avoid lists in the type sense while keeping the input and output types the same, that's a 👍 👍 from me! It also aligns with the type name of types.deferredModule instead of types.deferredModules

[infinisil]

Nice change :D

[roberth]

I've found an additional use case (flake.parts) where I need "static" modules with option docs support, similar to submoduleWith.

I've noticed that we can't preserve the location info of such modules yet, just like the modules passed to submoduleWith don't support this. We can solve this issue separately, perhaps even by using unsafeGetAttrPos, which would be more valuable than _file anyway.

[infinisil]

Other than two nits this looks good. I tested this with https://github.com/infinisil/dream2nix-modules, a demo for using the module system for dream2nix created with feedback from @DavHau at Zurihac. Turns out deferredModule is just what was needed there

[infinisil]

I think this check should be done earlier, ideally at the start of this function, since m is always required to be an attribute set, not just for this specific case here.

[roberth]

Turns out deferredModule is just what was needed there

Nice! I've already used it in flake.parts perSystem, NixOps 2, and I will use it again in #176557

[infinisil]

Looking good!

[roberth]

Thanks @infinisil!

[alex-ameen]

@roberth do you have any usage examples? This looks useful but the only examples I could find were two mentions in a test case that didn't reveal much.

[infinisil]

@aakropotkin You can look at the test case in this PR

[roberth]

flake-parts perSystem is one. It's for modeling anything that's a function of system, as used by the new Nix Flakes CLI.