Skip to content

lib: Add beforeApply option attribute#302905

Closed
dasJ wants to merge 1 commit intoNixOS:masterfrom
helsinki-systems:feat/options-expose-before-apply
Closed

lib: Add beforeApply option attribute#302905
dasJ wants to merge 1 commit intoNixOS:masterfrom
helsinki-systems:feat/options-expose-before-apply

Conversation

@dasJ
Copy link
Member

@dasJ dasJ commented Apr 9, 2024

See #299736 for the reasons.

Description of changes

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@dasJ dasJ requested review from infinisil and roberth as code owners April 9, 2024 19:23
@github-actions github-actions bot added 6.topic: module system About "NixOS" module system internals 6.topic: lib The Nixpkgs function library labels Apr 9, 2024
Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to hear @roberth's thoughts as well, but to me this looks looks good!

@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Apr 9, 2024
@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one person. label Apr 9, 2024
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want the extra attribute to be useful; not in less than single digit percent of modules.
It seems as if a better solution is perhaps within reach.

{ value = builtins.addErrorContext "while evaluating the option `${showOption loc}':" value;
# The merged value before applying the options `apply` function to it.
# In general though, `apply` should not be used, it's an anti-pattern.
beforeApply = res.mergedValue;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds the cost of a whole attribute to every option, but delivers almost no value in almost all cases.
Also it doesn't fully solve the problem. What if opt.beforeApply is wrong, and a non-trivial number of modules relies on opt.beforeApply? mkOption { modifyBeforeApply = f; }? opt.beforeModifyApply?

I think at some point we just have to admit that modules aren't fit for certain options or option trees anymore.
What if instead of an option tree or submodule, we could have a types.adapter, which transforms all definitions arbitrarily, transforms the returned config and options arbitrarily, and can be implemented in whichever way the use case demands?
That way we don't have to further bloat the module system with mostly unnecessary attributes, and actually do a better job at being compatible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if opt.beforeApply is wrong

Not sure what you mean by that?

and a non-trivial number of modules relies on opt.beforeApply

It only makes sense to rely on beforeApply if the option has an apply, otherwise it's the same as the options value. And apply is being discouraged already, you won't find much new uses. The main use case here really is just allowing certain reasonable configs for older options that already use apply and can't be migrated away from easily.

I agree that it's a high cost to pay for very little value though..

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also a good time to mention #299736 as an alternative implementation that doesn't add a new attribute to all options.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if opt.beforeApply is wrong

Not sure what you mean by that?

This time opt.value is wrong. Next time opt.beforeApply is wrong. Why would this sequence stop?

And apply is being discouraged already, you won't find much new uses.

I'm not convinced of that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opt.value is "wrong" this time because apply exists, so we need beforeApply. The sequence stops here because we don't have a "pre-apply" step. After the values get merged, it gets passed to apply directly.

I'm not convinced of that.

At least I'm heavily discouraging apply whenever I see it :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I might have overreacted.

Bloat argument still stands though, or are we betting on replacing evalModules in the foreseeable future?
I like the idea of having time to do that... :)

{ value = builtins.addErrorContext "while evaluating the option `${showOption loc}':" value;
# The merged value before applying the options `apply` function to it.
# In general though, `apply` should not be used, it's an anti-pattern.
beforeApply = res.mergedValue;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another angle is the current impossibility of using #257511 to simplify submodule. This can't currently be done, in large part because submodules have two values:

  • the value also known as config
  • the value with _type = "configuration", aka the evalModules result, containing config, as well as options and other "meta" or "out of band" things.

What if opt.beforeApply was already thing, submodule returned the _type = "configuration", and by convention, each submodule-typed option had apply = x: x.config or apply = x: removeAttrs ["_module"] x.config?
While that would be a little cumbersome to write, especially for more complicated hierarchies with multiple submodule/attrsOf nestings, it would be more capable, more efficient (fewer removeAttrs calls that copy almost everything), and possible to orthogonalize submodule into individually usable types, such as more or less attempted in #257511.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds interesting, though I don't think having beforeApply would be blocking that.

Once we deprecate apply entirely, beforeApply will vanish too :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once we deprecate apply entirely, beforeApply will vanish too :)

I think apply will stick around until we replace modules themselves. Do we really want to inflict its removal on users?

@infinisil
Copy link
Member

I guess the original use case for this is now resolved using #306790, so I say let's close it unless we find another one :)

@infinisil infinisil closed this May 13, 2024
@dasJ dasJ deleted the feat/options-expose-before-apply branch June 11, 2024 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: lib The Nixpkgs function library 6.topic: module system About "NixOS" module system internals 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants