Skip to content

meson: use pyproject = true, use --replace-fail#424913

Draft
TomaSajt wants to merge 4 commits intoNixOS:stagingfrom
TomaSajt:meson
Draft

meson: use pyproject = true, use --replace-fail#424913
TomaSajt wants to merge 4 commits intoNixOS:stagingfrom
TomaSajt:meson

Conversation

@TomaSajt
Copy link
Contributor

@TomaSajt TomaSajt commented Jul 13, 2025

We'll eventually remove the legacy format = "setuptools" functionality.

Checked the files before and after: only the self-referential store-paths differed.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • Nixpkgs 25.11 Release Notes (or backporting 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other contributing documentation in corresponding paths.

Add a 👍 reaction to pull requests you find important.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Jul 13, 2025
@nix-owners nix-owners bot requested a review from alyssais July 13, 2025 20:55
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't splice properly, will it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you're right, I'll just use python3Pacakges, then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhh, eval is failing...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't think this needs to be modified.

Why not? Have you tested it's spliced correctly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does build, but it's not using the right version of the package for cross builds, so things are liable to break in future:

nix-repl> pkgsCross.aarch64-multiplatform.meson.nativeBuildInputs                                   
[
  «derivation /nix/store/larsjhyzxl4vbkz6g621ka2d0z9krqcg-python3-3.13.4.drv»
  «derivation /nix/store/n0p3hjx9hdavhzgdsb9m7p46cg0yxc45-wrap-python-hook.drv»
  «derivation /nix/store/a2dp2pis7mapql336wjf75kalv2z7nra-ensure-newer-sources-hook.drv»
  «derivation /nix/store/fwpxjqrg1gslb7cl3l84zqhl7fky9a97-python-remove-tests-dir-hook.drv»
  «derivation /nix/store/z0g5vza9c42n6qb1vpdjmwgjpxyqirfm-python-remove-bin-bytecode-hook.drv»
  «derivation /nix/store/h2a3w00qmlyhq5hn7snqfjcha82g62lq-pypa-build-hook.sh.drv»
  «derivation /nix/store/0bcg5qi6nxh6nw2n6p5mbvb9rhxwr7yp-python-runtime-deps-check-hook.sh.drv»
  «derivation /nix/store/9x9bsdk8g70s8hb1dcv3gdbhdgky6ahw-pypa-install-hook.drv»
  «derivation /nix/store/34cmg6gcd7r6ag65z1wm04b6i497q6qm-python-namespaces-hook.sh.drv»
  «derivation /nix/store/qwgacn1g3qbmlzmpv6px78szbqg6bxk4-python-output-dist-hook.drv»
  «derivation /nix/store/1gz8wg40dbp0srz0f9ljj5vymlv20jwm-install-shell-files.drv»
  «derivation /nix/store/x3v559gl0l8jxlai5a501cizg1dzfpqk-python3.13-setuptools-80.7.1-aarch64-unknown-linux-gnu.drv»
]

Note how, in this x86_64 → aarch64 cross evaluation, the aarch64 version of setuptools is used even though it's a build dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have stayed up until like 4AM to try to get to the bottom of why splicing is weird and found some interesting things:

For some reason python hooks are not spliced:

And because hooks don't have a prefix even if buildPlatform!=hostPlatform it is very hard to notice.

So if you go back to how meson was before this PR and look at the build closure of the meson drv you're gonna see /nix/store/pj9qkidq1kbfdwsslnqiaimmr6yc4wgb-setuptools-build-hook.drv which is secretly NOT taken from buildPackages, and because it's not from buildPackages, it's using the wrong setuptools!.

(I really recommend using nix-tree, it's was very helpful for debugging this).

And this happens with both python3.pkgs.buildPython* python3Packages.buildPython* and using python3Packages.callPackage and using buildPython* through that.

So I guess maybe it's intended? BUT:

What I found really weird is that this non-splicing thing DOESN'T happen if you use the top level splicing logic directly:

If you add setuptoolsBuildHook into nativeBuildInputs of a package using python3Packages.callPackage (e.g. everything in pkgs/development/python-modules)` it's not going to get spliced, as "expected".

However, if you add python3Packages.setuptoolsBuildHook into nativeBuildInputs of a derivation that's using the normal top level pkgs.callPackage, it's gonna get spliced!

Why is there a difference? I have no idea.
Why is splicing disabled in the first place for hooks? I don't know.

Note:
I believe the splicing of python hooks has been disabled right from the start:
#104201


So going back around:

Currently, before this PR, we're using the wrong setuptools. Apparently intentionally... Don't ask me why.

So which change should I do:

If we use python3.pkgs.setuptools, the splicing is broken (because python3.pkgs is weird for some reason)
If we refactor to use python3Packages and I use python3Packages.setuptools, the splicing happens.

However, even if setuptools is spliced correctly when using python3Packages.setuptools, there are still some hooks being used automatically that are unspliced, pulling in the wrong(?) type of package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably better to get splicing right where we can with this change, even though it turns out that it in some other places it's done incorrectly.

@TomaSajt TomaSajt marked this pull request as draft July 13, 2025 23:40
@nixpkgs-ci nixpkgs-ci bot added 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: stdenv Standard environment labels Jul 14, 2025
@TomaSajt
Copy link
Contributor Author

TomaSajt commented Jul 14, 2025

Looks like doing python3Packages = self in python-modules.nix breaks the splicing for python3Packages.meson.
(Note: it was broken even before this PR)

If I manually copy the contents of pkgs/by-name/me/meson/package.nix (the version where I fixed the splicing issue) to pkgs/development/python-modules/meson/default.nix, and make it into a normal python package, the splicing works.

@alyssais alyssais requested a review from mweinelt July 14, 2025 14:58
@mweinelt
Copy link
Member

Sorry, I'm no good with splicing.

@mweinelt mweinelt closed this Jul 22, 2025
@github-project-automation github-project-automation bot moved this to Done in Stdenv Jul 22, 2025
@mweinelt mweinelt reopened this Jul 22, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 4, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: stdenv Standard environment 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants