buildPython*: allow providing a finalAttrs function

[PerchunPak]

This adds support for finalAttrs function in python packages:

buildPythonPackage (finalAttrs: { # buildPythonApplication also supported
  # ...
})

just like

stdenv.mkDerivation (finalAttrs: {
  # ...
})

This implementation is alternative to #271387. The difference is that I don't depend on a stale PR, but #271387 should replace my code.

For now, I only updated a package that I maintain for testing. For treewide, #293452 would be a better place.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[lucasew]

Isn't there a lib function that does that?

If the process could be standardized that could be done to all builders and we could do a rec witch hunt

[PerchunPak]

That's #234651 (for python #271387), but it is stale and unlikely to be merged soon. This PR implements a temporary solution

[lucasew]

OK. I just think that it should be in lib on the final form. It'll likely be reused by other builder implementations.

[ShamrockLee]

This implementation fixes the fixed point arguments taken by mkPython* before passing down instead of leaving it to stdenv.mkDerivation. This makes the semantics completely different from that provided by stdenv.mkDerivation or overrideAttrs -- The finalAttrs here will be that of <pkg>.overridePythonAttrs (finalAttrs: previousAttrs: { }) (if we add such support) instead of <pkg>.overrideAttrs (finalAttrs: previousAttrs: { }).

One significant drawback is the lack of finalAttrs.finalPackage. While we can also re-implement the finalPackage here, that would make us depend more on custom overriders like overridePythonAttrs.

Update:

As @TomaSajt suggests, it's not even something like overridePythonAttrs (finalAttrs: previousAttrs: { }), since the finalAttrs here doesn't get updated after overriding.

[TomaSajt]

As @ShamrockLee said, this is just a shorthad for lib.fix. Just using lib.fix is, IMO, perfectly acceptable if you don't want to use rec

[PerchunPak]

I tried to avoid awful diffs, but... here we go. Rewritten everything to pass down finalAttrs to stdenv.mkDerivation

[PerchunPak]

Rewritten almost everything to use overlays for handling args. The only issue I have right now is that I cannot call clearArgs inside an overlay. Maybe setting all those values to null? Doesn't sound right...

[TomaSajt]

You could just not use lib.extends, but a custom function I have been calling "transforms". It takes in a transformation instead of an overlay.
If that transformation just does prevAttrs // at the beginning, it would act exactly like an overlay.
However, since we now explicitly // the prevAttrs you could remove some unneeded names from prevAttrs before //-ing it.

here's the definition of transforms btw:

transforms =
  transformation: rattrs: 
    final: transformation final (rattrs final)

And here's lib.extends to compare to

extends =
  overlay: rattrs:
    final: rattrs final // overlay final (rattrs final)

(Note the extra rattrs final //) (rattrs final is prev)

[PerchunPak]

BTW, @TomaSajt, you should totally add transforms to stdlib

[TomaSajt]

As @ShamrockLee mentioned previously it's very important to acknowledge is the following:

final in transformAttrs is the actual finalAttrs that's coming from inside mkDerivation
On the other hand, overridePythonAttrs is just a slap-on solution, so it only works in terms of the uppermost layer of attributes, so final in overridePythonAttrs means the fix-point of the args that will then be passed to transformAttrs.

Note:
to this day overridePythonAttrs doesn't know about whether the current package has been already overrideAttrs-ed, so it will gladly remove the effects of overrideAttrs, if done

[ShamrockLee]

Note:
to this day overridePythonAttrs doesn't know about whether the current package has been already overrideAttrs-ed, so it will gladly remove the effects of overrideAttrs, if done

So that you know, #267296 fixes the issue. Such issues are a good example of how custom overriders are hard to maintain and are not a good pattern.

[ShamrockLee]

You could just not use lib.extends, but a custom function I have been calling "transforms". It takes in a transformation instead of an overlay. If that transformation just does prevAttrs // at the beginning, it would act exactly like an overlay. However, since we now explicitly // the prevAttrs you could remove some unneeded names from prevAttrs before //-ing it.

here's the definition of transforms btw:

transforms =
  transformation: rattrs: 
    final: transformation final (rattrs final)

This function is similar to the lib.adaptMkDerivation in #234651 but without the functionArgs stuff.

[ShamrockLee]

Suggested change

	stdenv = unfixed.stdenv or python.stdenv;
	stdenv = origAttrs.stdenv or python.stdenv;

if origAttrs is { stdenv = ...; }, unfixed will be a callable equivalent to finalAttrs: { stdenv = "..."; }. When origAttrs is a function, it is impossible to specify stdenv as its arguments and get it out before fixing the arguments.

buildPythonPackage.override { stdenv = ...; } is the only possible and reasonable way to specify custom stdenv. That's what #271762 is for.

[PerchunPak]

Closing in favor of #271387