stdenv: Disallow output changes in installCheck

[roberth]

Motivation for this change

Spot installation script problems and avoid tests polluting the installation.

This will indicate errors in the install script fixed up by the
installed software, or software or configuration that relies on
a writable installation directory.

NOTE: this only works for derivations which use the installCheck phase as intended. Some language integrations, like pythonPackages, need to call the new functions on their own.

Closes #120053

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• 21.11 Release Notes (or backporting 21.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[symphorien]

Maybe we can chmod -w the outputs before installCheckPhase as well. What do you think ?

[roberth]

@symphorien I think we should, but it's not as simple as that because we'll have to restore the permissions, as distPhase needs to write to outputs again. I'd like to keep it simple for now.

[roberth]

/marvin opt-in
/status needs_reviewer

[marvin-mk2]

Hi! I'm an experimental bot. My goal is to guide this PR through its stages, hopefully ending with a merge. You can read up on the usage here.

[asymmetric]

@roberth how can i test this? I tried:

❯ nix build .#tests
error: flake output attribute 'legacyPackages.x86_64-linux.tests' is not a derivation

There are some ofborg eval failures, any idea why that is?

Sorry that I can't provide any more direct help at the moment.

[roberth]

nix build .#tests

Tests contains a nested attrset of derivations. Try with nix-build -A tests, which traverses those attrs and doesn't copy all of your local nixpkgs into the store.

ofborg eval failures

I don't think it's technically an eval failure, but rather that the eval status combines the other statuses. The lib-tests job did fail because it timed out rebuilding the world.

You could "just" build a package, rebuilding whatever needs to rebuild. A lot. If the build works, the pr works, but I don't know if this will impact many subtly broken tests. Maybe this pr should run separately on hydra first?

[symphorien]

This needs doc for allowWriteInInstallCheck and I think this is too late for 22.05 given the feature freeze #167025

[symphorien]

/status awaiting_changes

[timokau]

Hi!

The marvin-mk2 bot is now discontinued. I have removed the relevant tags from this PR. If you still need someone to look at it, one option would be to ask in this discourse thread.

I am posting this notice to all open PRs with the marvin tag. Please understand that following all of these discussions would take too much time, so I will unsubscribe from this PR unless I have already been involved in it before this message.

[roberth]

I did the thing.

If you're interested, this is continued in

• Disallow output changes in installCheckPhase #303361