haskellPackages.mkDerivation: New intermediates output

[Gabriella439]

This adds a new intermediates output that can be used to accelerate a Haskell build by importing build products exported from a prior similar build.

The motivation for this is explained by the following post:

https://www.haskellforall.com/2022/12/nixpkgs-support-for-incremental-haskell.html

This is a subset of the changes from #204020 that I believe are safe to merge while waiting for NixOS/nix#7362 to be reviewed. I've incorporated outstanding feedback from the last PR into this one.

I structured this to be a hash-preserving changes for Haskell packages that don't use the feature, so this won't trigger unnecessary rebuilds or breakage.

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[sternenseemann]

This is a subset of the changes from #204020 that I believe are safe to merge while waiting for NixOS/nix#7362 to be reviewed. I've incorporated outstanding feedback from the last PR into this one.

Without that Nix change, this is hardly useful, though?

[Gabriella439]

Not necessarily. There are other ways to thread the full build to the incremental build, albeit with a worse UX. See, for example: https://felixspringer.xyz/homepage/blog/incrementalHaskellBuildsWithNix

[cdepillabout]

@Gabriella439 I think the full functionality in #212107 could be pretty neat, but I'm not sure how obvious just the functionality in this PR would be to use.

Would you be willing to add a test for this functionality to pkgs/test/haskell/ (if that makes sense)? That might also work well as an example of how the new intermediates output could be used.

[9999years]

@cdepillabout I've added a test to show how to use the API. For now, how to manage the previousIntermediates is left as an exercise to the reader. But once NixOS/nix#7362 is merged (we're working on getting a review from the Nix team) we'll be able to merge in the rest of #204020 (with a smaller diff).

[cdepillabout]

LGTM. Nice test!

cc @sternenseemann about this as well, since I think he had a few concerns previously

[cdepillabout]

Would you consider adding you guys as maintainers to this test?

Or potentially adding Mercury to https://github.com/NixOS/nixpkgs/blob/b03ac42b0734da3e7be9bf8d94433a5195734b19/maintainers/team-list.nix, and then making mercury the maintainer?

[9999years]

Done. Can you check that I added the maintainers correctly?

[mweinelt]

I'm locking and closing this pull request, because through the rebase you inadvertently requested many maintainers for review, which subscribed them to update notifications, resulting in unnecessary spam inside everyone's inbox. While we can remove their review requests, we sadly cannot unsubscribe anyone.

Please create a new pull request and for the next time remember to set your PR to draft status before rebasing. In draft status, you can preview the list of maintainers that are about to be requested for review, which allows you to sidestep this issue.