Make callPackage observe <pkg>.override.argSelectors and by-name-overlay manage arg-selectors.nix and siblings

[ShamrockLee]

1. Introduce argSelectors as a way to specify arguments for the override interface.
   It is an attribute set of functions that takes pkgs and return the selected argument.

   {
     stdenv = { clangStdenv, ... }: clangStdenv;
   }

   The difference between the argSelectors and the pkgs: { ... } args.nix proposed as an alternative enhancement in Nix RFC 140 is that the attribute names of argSelectors is strict and accessible without calling the selector functions.

2. Teach lib.callPackageWith autoArgs f args and lib.callPackagesWith autoArgs f args to observe f.<argSelectors>, and attach argSelectors to the result as <pkg>.override.argSelectors if args == { }.

   This ensures that argSecetors will be lost after calling <pkg>.override directly or after specifying a plain attrset of custom arguments to callPackage. The presence of <pkg>.override.argSelectors therefore ensures result == pkgs.callPackage result.override { }.

   This is useful to short-cut the build tests of staging changes. If the mass-rebuild change of pkgs.foo cause its direct dependent pkgs.bar to rebuild, and pkgs.bar.override.argSelectors is present, its rebuild can be short-cut as

   pkgsMaster.callPackage pkgsStaging.bar.override {
     inherit (pkgsStaging) foo;
   }
   

3. Provide lib.makeCallPackageWithArgSelectors and pkgs.callPackageWithArgSelectors as a convenient way to add/extend <pkg>.override.argSelectors.

4. Teach the by-name-overlay.nix to understand an optional file arg-selector.nix beside package.nix, implementing the "Option 3" of Fixing .override interface stability #404946.

   Even though the hard-coded version-specific override interface proposed in pkgs/by-name/README: explicitly suggest version specific override interfaces #444420 enhances override stability, we sometimes do want override abstraction for interoperability. For example, we may want different versions/variants of a package (e.g. foo, fooMinimal and fooFull) to have the same override interface to make it possible to bar.override { foo = foo_1; }. The sibling.nix interface proposed below makes it possible to define them in pkgs/by-name.

5. Introduce a way to specify "siblings" inn pkgs/by-name. One can define a sibling with a packages.nix in the following format:

   # pkgs/by-name/fo/foo-minimal
   {
     type = "sibling";
     package = "foo";
     # packagePath = [ "fooSet" "foo" ]; # Alternative to `package`
     transform = # Optionally transform the package
       { lib, package }:
       package.overrideAttrs (finalAttrs: previousAttrs: {
         # Attributes to override
       });
   }

   A sibling can also have an optional arg-selectors.nix for <pkg>.override overriding.

   The above example provides foo-minimal with the same <pkg>.override interface as pkgs.foo the following way:

   {
     foo-minimal = transform {
       inherit lib;
       package =
         callPackageWithArgSelectors
           foo.override
           (import /path/to/by-name/fo/foo-minimal/arg-selectors.nix);
     };
   }

   The sibling design is inspired by lib.makeAlias: init with by-name support #442066

Things done

• Built on platform:
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• Tested, as applicable:
  • NixOS tests in nixos/tests.
  • Package tests at passthru.tests.
  • Tests in lib/tests or pkgs/test for functions and "core" functionality.
• Ran nixpkgs-review on this PR. See nixpkgs-review usage.
• Tested basic functionality of all binary files, usually in ./result/bin/.
• Nixpkgs Release Notes
  • Package update: when the change is major or breaking.
• NixOS Release Notes
  • Module addition: when adding a new NixOS module.
  • Module update: when the change is significant.
• Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

---

Add a 👍 reaction to pull requests you find important.

[emilazy]

(ccing myself as highly relevant to my goals for 26.05! I think the cumulative desiderata for a full migration of everything to by-name point towards a holistic solution that would be more radical than this and hopefully more ergonomic while maintaining good compatibility, but I am very happy to see this part of the design space explored.)

[ShamrockLee]

Rebased for the CI fixes. See #457845