Skip to content

python3: propagate overrides of spliced pythons #498251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
doronbehar merged 2 commits into from
Mar 13, 2026
Merged

python3: propagate overrides of spliced pythons #498251

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
825a7f0
python3: avoid a small `rec` near passthruFun
doronbehar Mar 9, 2026
20a0e1b
python3: propagate overrides of spliced pythons
doronbehar Mar 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 41 additions & 7 deletions pkgs/development/interpreters/python/cpython/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,21 +165,55 @@ let
);
}
// __splices;
# When we override the interpreter we also need to override the spliced
# versions of the interpreter. NOTE: In lua-5/interpreter.nix, this
# filter is different - we take every attribute from @inputs, besides
# derivations. That filter causes cross compilations issues for Python
# See e.g:
#
# pkgsCross.armv7l-hf-multiplatform.buildPackages.python3Packages.bcrypt
#
# And the following Nixpkgs issues/PRs:
#
# - https://github.com/NixOS/nixpkgs/issues/48046
# - https://github.com/NixOS/nixpkgs/pull/480005 (Wrong PR)
# - https://github.com/NixOS/nixpkgs/pull/482866 (Correct fix)
# - https://github.com/NixOS/nixpkgs/pull/498251 (Re-adds this @inputs filter)
inputs' = lib.filterAttrs (
Copy link
Member

@lheckemann lheckemann Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this would still have the surprising override-dropping behaviour if inputs to the python package are overridden, e.g. python.override { sqlite = sqlite-interactive; }?

Copy link
Contributor Author

@doronbehar doronbehar Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, this would still have the surprising override-dropping behaviour if inputs to the python package are overridden, e.g. python.override { sqlite = sqlite-interactive; }?

That behavior was present before 6421482 , due to the !lib.isDerivation condition used in the filter. TBH I'm not sure why that filter function was breaking the splicing for pkgsCross.armv7l-hf-multiplatform.buildPackages.python3Packages.setuptools-rust. After opening the current PR, I noticed the Lua ecosystem also uses the same filter and pretty much the same code:

nixpkgs/pkgs/development/interpreters/lua-5/interpreter.nix

Lines 191 to 202 in f3f83f3

# copied from python
passthru =
let
# When we override the interpreter we also need to override the spliced versions of the interpreter
inputs' = lib.filterAttrs (n: v: !lib.isDerivation v && n != "passthruFun") inputs;
override =
attr:
let
lua = attr.override (inputs' // { self = lua; });
in
lua;
in

I'm involved in cross compilation of lua packages too, and I haven't noticed equivalent issues. Maybe that's due to the @inputs in lua-5/interpreter.nix v.s cpython/default.nix.

Do you think the correct thing to do now would be to investigate why this lib.filter won't work for Python? Or maybe add a NOTE comment?

Copy link
Member

@lheckemann lheckemann Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this PR doesn't introduce any new breakage, and at least partially fixes the existing breakage, I think it's good.

Maybe opening a new issue would be better than a note, but I'm also not sure if anyone really has the problem I'm describing (I do struggle to think of a real use case for overriding python inputs where an overlay replacing the package for all of nixpkgs wouldn't be the more sensible approach).

Copy link
Contributor Author

@doronbehar doronbehar Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lheckemann I just realized that packageOverrides - an important @inputs argument should also be propagated to all of these Pythons, so I added it, along with a comment striving to thoroughly explain the situation. If you could give it one more round of review it'd be great.

Copy link
Member

@lheckemann lheckemann Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems sensible :)

n: v:
(builtins.elem (builtins.typeOf v) [
"int"
"bool"
"string"
"path"
"null"
])
|| n == "packageOverrides"
) inputs;
override =
attr:
let
python = attr.override {
self = python;
};
python = attr.override (
inputs'
// {
self = python;
}
);
in
python;
pythonVersion = with sourceVersion; "${major}.${minor}";
libPrefix = "python${pythonVersion}${lib.optionalString (!enableGIL) "t"}";
in
passthruFun rec {
inherit self sourceVersion packageOverrides;
passthruFun {
inherit
self
sourceVersion
packageOverrides
libPrefix
pythonVersion
;
implementation = "cpython";
libPrefix = "python${pythonVersion}${lib.optionalString (!enableGIL) "t"}";
executable = libPrefix;
pythonVersion = with sourceVersion; "${major}.${minor}";
sitePackages = "lib/${libPrefix}/site-packages";
inherit hasDistutilsCxxPatch pythonAttr;
inherit (splices)
Expand Down
Loading