all-packages.nix: introduce and use dontRecurseIntoAttrs

[oxij]

What? Why?

The first non-controversial noop commit of #45841. I find this thing very useful: it provides a single place you need to override to make nix-env -qaP evaluate all packages, which is very useful for testing.

I got tired of rebasing it and fixing conflicts all the time. Can we have it in nixpkgs too, please?

git log

• all-packages.nix: introduce and use dontRecurseIntoAttrs

nix-instantiate environment

• Host OS: Linux 4.9, SLNOS 19.03
• Nix: nix-env (Nix) 2.1.3
• Multi-user: yes
• Sandbox: yes
• NIXPKGS_CONFIG:

{
  checkMeta = true;
  doCheckByDefault = true;
}

nix-env -qaP diffs

• On x86_64-linux: noop
• On aarch64-linux: noop
• On x86_64-darwin: noop

[oxij]

also /cc @7c6f434c, as usual

[Ma27]

I like the idea behind it, so 👍

(as a side note: unless I'm missing something, you also want to apply dontRecurseIntoAttrs on weechatScripts and roundcubePlugins. I'm "responsible" for both sub-packages and intentionally skipped the recurseIntoAttrs there).

[samueldr]

Is it possible to have documentation on when one would expect to use dontRecurseIntoAttrs? What are the main use case?

Not necessarily asking the documentation in the contribution, but at least comments to the PR explaining, so it can be referenced for future documentation.

I think it's easier to grok by looking at #45841, where nodePackages has recurseIntoAttrsMaybe "nodePackages" nodePackages_8_x;, while all the version-specific attrsets have dontRecurseIntoAttrs, so they're not counted as many times as there are versioned sets, for the metrics, right? Any other uses? Does @Ma27's request makes sense here?

Thanks!

[oxij]

@Ma27 yep, either I missed it before or it was added in-between, done.

@samueldr done? Yes, with recurseIntoAttrsMaybe the idea is exactly as you say (not to count version specific things multiple times), but this PR doesn't have that.

[samueldr]

Thanks, the comment is good. (I also now fully realize what the comment meant previously.)

[7c6f434c]

Hmmm. This may even be already enough to support an overlay that does the «maybe» part outside of the main codebase.

[oxij]

ping?

[ersinakinci]

In case anyone else is coming here from Google, dontRecurseIntoAttrs and its sibling recurseIntoAttrs are functions that are intended to provide metadata/semantic context for nix-env and anyone reading the code.

---

recurseIntoAttrs

The idea behind recurseIntoAttrs is that if you're looking for a derivation using nix-env, the search will recurse into top-level attributes that are objects. It achieves this by adding the attribute recurseForDerivations = true; to whichever set is wrapped by the function, which nix-env picks up on when running queries.

For example, nixpkgs.webos contains several sub-attributes like novacom, novacomd, etc. Thanks to the fact that webos is wrapped with recurseIntoAttrs, searching for any of those sub-attributes will yield the full attribute path:

$ nix-env -qaP novacom
nixos.webos.novacom  novacom-18

Whereas if you didn't wrap webos with recurseIntoAttrs, you would see something like this, because nix-env would only search for your derivation at the top level:

$ nix-env -qaP novacom
error: selector 'novacom' matches no derivations

---

dontRecurseIntoAttrs

So then what does dontRecurseIntoAttrs do?

At the moment, nothing: it's an identity function that merely returns whatever is passed into it. Since nix-env's default behavior is to not recurse into top-level attributes, it will simply, well, not recurse into an attribute when wrapped with dontRecurseIntoAttrs.

That's not to say that it's pointless. Marking an attribute with dontRecurseIntoAttrs will:

• Lend emphasis to the fact that an attribute should not be recursed into while searching for derivations
• More importantly, allow overriding the function so that you can recurse into all attributes wrapped in dontRecurseIntoAttrs, which can be useful during development

The second point is the main reason behind dontRecurseIntoAttrs. At the moment, its applicability is limited by the fact that the vast majority of packages aren't wrapped with dontRecurseIntoAttrs, so overriding it to act like recurseIntoAttrs won't have much effect.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/wheres-the-nodejs-apps/12064/10