nixos systemPackages: rework default outputs

[vcunat]

• Now pkg.outputUnspecified = true but this attribute is missing in
  every output, so we can recognize whether the user chose or not.
  If (s)he didn't choose, we put pkg.bin or pkg.out or pkg into
  systemPackages.
• outputsToLink is replaced by extraOutputsToLink.
  We add extra outputs regardless of whether the user chose anything. (EDIT: this is why I didn't consider 7410c8f good enough.)
  It's mainly meant for outputs with docs and debug symbols.
• Note that as a result, some libraries will disappear from system path.

/cc people who touched these things: @edolstra, @lethalman, @dezgeg. Comments are welcome! I expect to merge in a few days.

[mention-bot]

By analyzing the blame information on this pull request, we identified @lethalman, @edolstra and @shlevy to be potential reviewers

[vcunat]

For reference, how we handle the outputs:

• For build inputs we take the default output (typically "dev") extended by propagating some others. That seems to work very well.
• For string references the default output is chosen. That's suboptimal, typically. The output should be always specified and we could do eval-time checks, but that would imply too much rewriting. In the meantime we might use buildEnv of some outputs, e.g. those that are propagated into build inputs by default.
• For systemPackages: minimalistic variant pkg.bin or pkg.out or pkg proposed in this PR.
• For nix-env: currently it looks at the outputs attribute and installs all those. (If it doesn't exist, it installs just what is passed.) We might better make it configurable in future, perhaps similarly to systemPackages. Note that consequently even nix-env -iA pkg.out will install all outputs. We might avoid this e.g. by removing outputs attribute from every output.

[edolstra]

What is outputUnspecified?

[vcunat]

Now pkg.outputUnspecified = true but this attribute is missing in every output

I needed a way to differentiate between [ myPackage ] and [ myPackage.dev ] so I put an attribute into the first one.

[shlevy]

@vcunat Why can't you just check outputName?

[vcunat]

@shlevy: Checking outputName isn't enough, as the default output has the same name as one of the "choosable" outputs. I originally wanted to change outputName instead of introducing outputUnspecified, but that had adverse effects on nix-env.

[shlevy]

Ah, I see. Can you give a specific use case where we need to distinguish "this is the first output by default" vs "this is the first output by selection"?

[vcunat]

Hmm, maybe outputName would be enough after carrying on with the plan (above) that without choosing you'd get a buildEnv of all outputs.

[vcunat]

@shlevy: example directly in this PR: https://github.com/NixOS/nixpkgs/pull/12653/files#diff-dcba170e079cd72958889058d01a4a02R128 I want to choose some defaults for systemPackages but only if the code didn't specify the output explicitly.

[shlevy]

Hm, I see. I guess I think it would be better to just settle on the convention that the first output is the one we want in user envs, or install all outputs.

[vcunat]

Well, the first one is what is put into build inputs, and typically that's different from what you want in user envs (e.g. headers).

[shlevy]

Right, but I think the simple path should be the one that works for users, not devs. If anyone has to specify, it should be people writing build expressions, not people just trying to get their system working.

[vcunat]

Well, string references are the largest problem here, as that's a "simple path" usage mostly used by devs. Of course, now that we can override conversion to strings, it is possible to hack around that, but I'm still leaning to make the "simple path" refer to a buildEnv of all outputs – that seems a nice backward-compatible layer suitable for both devs and users.

[edolstra]

This should probably be factored out into a separate function, e.g. getOutputsToInstall or something like that, so it can be re-used in places like the handling of systemd.services.<unit>.path.

This should probably also be overridable via a package attribute, e.g.

meta.outputsToInstall = [ "bin" "man" ];

which would also be respected by nix-env.

[vcunat]

So let's use meta.outputsToInstall directly, and stdenv shall fill it in if not specified. The needed nix-env changes are clear (and trivial, I think).

I'll think on a suitable customization mechanism via the nixpkgs config, but I feel like in the long term we would better unify this perhaps through nixUP or something similar.

[vcunat]

I'm still not certain how to best make it customizable, but the nix-env part really seems clear, so I implemented that now. People would better get that functionality early NixOS/nix#815.

[dezgeg]

IMHO this per-package meta.outputsToInstall shouldn't contain anything related to man or info but rather keep that in the systemPackages logic, since that would allow us to have a configuration option to disable man- or infopages and not install them in the first place.

[coveralls]

Changes Unknown when pulling 9a824f2 on vcunat:p/default-outputs into ** on NixOS:closure-size**.