Skip to content

doc/python: document fixed-point arguments (#271387)#477149

Open
doronbehar wants to merge 4 commits intoNixOS:masterfrom
doronbehar:doc/buildPython@fixed-point
Open

doc/python: document fixed-point arguments (#271387)#477149
doronbehar wants to merge 4 commits intoNixOS:masterfrom
doronbehar:doc/buildPython@fixed-point

Conversation

@doronbehar
Copy link
Contributor

@doronbehar doronbehar commented Jan 5, 2026

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

@nixpkgs-ci nixpkgs-ci bot requested review from mweinelt and natsukium January 5, 2026 13:32
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. 6.topic: python Python is a high-level, general-purpose programming language. 8.has: documentation This PR adds or changes documentation labels Jan 5, 2026
@doronbehar doronbehar force-pushed the doc/buildPython@fixed-point branch from 5ad3d8b to 99403d3 Compare January 5, 2026 13:42
@doronbehar doronbehar force-pushed the doc/buildPython@fixed-point branch from 99403d3 to 5339e49 Compare January 5, 2026 13:47
@nixpkgs-ci nixpkgs-ci bot requested a review from GetPsyched January 5, 2026 13:52
@MattSturgeon
Copy link
Contributor

Thanks for working on the docs!

I had assumed that fixpoint-args support was deliberately undocumented until support was fully backported to all supported versions (currently just 26.05 and 25.11) and had some time to incubate. This prevents confusion when someone reads the unstable docs and assumes that they apply equally to stable Nixpkgs.

I'll defer to @ShamrockLee w.r.t. when we're ready to publicly document fixpoint-args support.

Comment on lines +235 to +236
pname = "pyspread";
inherit (finalAttrs) version;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expect it to inherit pname as well, this is one of the few times I'd think using pname would be fine (as opposed to e.g: fetchFromGitHub's repo argument).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't really. We normalize pname (e.g. apply lowercasing), but the pypi name may not reflect that, so it is often untied.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is often untied

So this means while this documentation suggest untying it, tying pname is also accepted?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it might be stable, if the library name already is normalized, but uppercase, dashes, etc. will cause flapping every once in a while.

Copy link
Contributor

@MattSturgeon MattSturgeon Jan 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if you do reuse the name for pname and src.pname, it's best to use a locally defined variable rather than finalAttrs.pname. Values in finalAttrs represent the final state, including any user defined overrides.

pname is an example of something an end-user may wish to override, e.g. pname = prevAttrs.pname + "-with-my-tweaks". Such an override shouldn't propagate to the src's project.

Copy link
Member

@Sigmanificient Sigmanificient left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets use this opportunity to document arguments unable to referenced directly in finalAttrs, such as format or optional-dependency. See:

@doronbehar
Copy link
Contributor Author

Thanks for the suggestion @Sigmanificient , I'm not sure about the full list of those, so I wrote only optional-dependencies and format. When you linked https://github.com/NixOS/nixpkgs/pull/271387/changes#diff-c263b18f23de7c1f00ed846783c3b977718aae8cdfdd89fc82adcf365d1d8bebR107-R120 I wasn't sure what is the full list of attributes.

@Sigmanificient
Copy link
Member

@ShamrockLee might be able to give you a proper list

@MattSturgeon
Copy link
Contributor

MattSturgeon commented Jan 6, 2026

Thanks for the suggestion @Sigmanificient , I'm not sure about the full list of those, so I wrote only optional-dependencies and format. When you linked #271387 (changes) I wasn't sure what is the full list of attributes.

This isn't just some args being passthru attrs instead of derivation attrs, it's also some args being entirely renamed to other attrnames. The most prominent example is renaming everything checkPhase related to installCheckPhase, mostly for historic reasons.1

That is already documented under #testing-python-packages, so we probably need a separate PR to consolidate the "args to buildPythonPackage" vs "final attrs to mkDerivation" difference and clarify that finalAttrs itself comes from the underlying mkDerivation call not the outer buildPythonPackage call.

Taking a step back, I don't think it's practical to list every single attr that is or isn't accessed via passthru2 and every single attr that gets transformed or renamed when calling the underlying mkDerivation. Instead, I think the docs should focus on highlighting that there is an intermediate step and clarify with a few demonstrative examples.

Footnotes

  1. these kinda manually maintained exhaustive docs are a nightmare to maintain and keep in-sync with reality.

@ShamrockLee
Copy link
Contributor

Thanks for the suggestion @Sigmanificient , I'm not sure about the full list of those, so I wrote only optional-dependencies and format. When you linked https://github.com/NixOS/nixpkgs/pull/271387/changes#diff-c263b18f23de7c1f00ed846783c3b977718aae8cdfdd89fc82adcf365d1d8bebR107-R120 I wasn't sure what is the full list of attributes.

You can find them by seeing both excludeDrvArgNames and passthru, and see if their intersections are referenced as finalAttrs.passthru.${name} somewhere elde inside extendDrvArgs.

As GitHub mobile is suffering from https://github.com/orgs/community/discussions/181590 , could someone help me start a review comment thread targeting lines from NOTE: to the end of the argument list? I will be able to post a suggestion then.

Comment on lines 244 to 247
NOTE: The following attributes, are not available in `finalAttrs`, but are available in `finalAttrs.passthru`:

- `format`
- `optional-dependencies`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShamrockLee please suggest something different here.

Copy link
Contributor

@ShamrockLee ShamrockLee Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
NOTE: The following attributes, are not available in `finalAttrs`, but are available in `finalAttrs.passthru`:
- `format`
- `optional-dependencies`
::: {.note}
Some `buildPythonPackage`/`buildPythonApplication` arguments are passed down to `stdenv.mkDerivation` via `passthru` instead of passing directly.
Their final states can be accessed via `finalAttrs.passthru.${name}`, and they can be overridden via [`<pkg>.overrideAttrs`](#sec-pkg-overrideAttrs) under the `passthru` attribute.
Such arguments include:
- `disabled`
- `pyproject`
- `build-system`, `dependencies` and `optional-dependencies`
:::

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, @MattSturgeon what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As always, we could polish the wording a little. But, yes it LGTM.

I still feel like a followup PR may be able to consolidate some of the "be aware: the args you pass to buildPythonPackage are not the same as what's in finalAttrs" documentation, as per my rambling in #477149 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As always, we could polish the wording a little. But, yes it LGTM.

I reworded it a little bit, I hope it is better.

I still feel like a followup PR may be able to consolidate some of the "be aware: the args you pass to buildPythonPackage are not the same as what's in finalAttrs" documentation, as per my rambling in #477149 (comment)

I agree, but I feel this topic is going under substantial changes and it might be better to document this subject once we are satisfied with the behavior :). See also my <!-- TODO --> comment there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable overall 👍🏻

If I'm nitpicking,

Don't worry about it, I'm patient and your comments are great 👍 .

it feels odd to explicitly mention overrideAttrs here:

Therefore the final states of these attributes can be accessed via finalAttrs.passthru.${name}, and they can be overridden via <pkg>.overrideAttrs under the passthru attribute.

...overrideAttrs uses the same finalAttrs fixpoint arg as mkDerivation (and by extension buildPythonPackage), so calling it out explicitly with an "and" feels like we're implying it does things differently.

That said, I get where you're coming from. It's probably helpful to highlight that this "finalAttrs is different to the initial args" quirk also applies in overrideAttrs.

TBH I'm not sure I fully understood how you understood the docs I read :). I think when @ShamrockLee chose to explicitly mention overrideAttrs they did that to imply to use .overrideAttrs and not .overridePythonAttrs, since the latter doesn't support the signature (finalAttrs: prevAttrs: {}). I tried to do that while not shaming ourselves that this feature is not yet supported due to #258246 :). The TODO comment below is the comment that should remind us to update that text once #258246 is resolved.

Maybe we can be optimistic that #258246 will be resolved soon and we can postpone this .note phrasing to afterwards? There were other voices who wanted to hold back this PR a little bit to let this new exciting feature stabilize.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

Their final states can be accessed via `finalAttrs.passthru.${name}`, or overridden via [`<pkg>.overrideAttrs`](#sec-pkg-overrideAttrs) in the same way.

The TODO comment below is the comment that should remind us to update that text once #258246 is resolved.

Personally, I'm in favour of adding finalAttrs support to overridePythonAttrs. It's actually a trivial change (I had a patch somewhere, but I can't find it to link).

I think the main argument against doing so is the desire to deprecate overridePythonAttrs entirely, in favor of exclusively using overrideAttrs:

As for overridePythonAttrs, it should be deprecated after overrideAttrs for Python packages is ready.
-- #258246 (comment)

I personally hope to push overrideAttrs, but recognize that overridePythonAttrs are still widely used and that Python packages' overrideAttrsinterface still has some unsettled parts (i.e., the check/installCheck attributes).
-- #258246 (comment)

Personally, I don't see the two ideas as mutually exclusive; we could add fixpoint-arg support now and still deprecate the function later, if we wanted to.

On the basis that its future direction is uncertain, I'm on the fence w.r.t. including the TODO comment, but I won't block on it 🙂

There were other voices who wanted to hold back this PR a little bit to let this new exciting feature stabilize.

I think that was just me justifying why we initially merged fixpoint-args support without new docs. Once @ShamrockLee is happy for this to be documented, I am too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the main argument against doing so is the desire to deprecate overridePythonAttrs entirely, in favor of exclusively using overrideAttrs:

OK I see, I wasn't aware of that since I haven't read thoroughly all of #258246. I have revised the TODO comment a bit to reflect that.

Copy link
Contributor

@ShamrockLee ShamrockLee Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally hope to push overrideAttrs, but recognize that overridePythonAttrs are still widely used and that Python packages' overrideAttrsinterface still has some unsettled parts (i.e., the check/installCheck attributes).
-- #258246 (comment)

@doronbehar Could you please help me recover the content of the quoted comment from your mailbox? GitHub Mobile tricked me into deleting it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean this?

image

It is from another thread technically.

@doronbehar doronbehar force-pushed the doc/buildPython@fixed-point branch 2 times, most recently from fe37bc4 to bd1e678 Compare January 7, 2026 00:45
@mdaniels5757 mdaniels5757 added the backport release-25.11 Backport PR automatically label Jan 7, 2026
@doronbehar doronbehar force-pushed the doc/buildPython@fixed-point branch from bd1e678 to ad54fe3 Compare January 7, 2026 12:42
@doronbehar doronbehar force-pushed the doc/buildPython@fixed-point branch from 424a126 to 5ac2996 Compare January 12, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. backport release-25.11 Backport PR automatically

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants