pypy: fix installPhase#442224
Merged
SuperSandro2000 merged 4 commits intoNixOS:masterfrom Sep 16, 2025
Merged
Conversation
nixpkgs-ci
bot
added
10.rebuild-linux: 1-10
nix-owners
bot
requested review from
FliegendeWurst,
andersk,
mweinelt and
natsukium
Contributor
Author
|
Member
SuperSandro2000
left a comment
SuperSandro2000
left a comment
There was a problem hiding this comment.
I try to build pypy3Packages.bcrypt which currently fails like:
bcrypt>
bcrypt> --- stderr
bcrypt> error: Could not find _sysconfigdata*.py in /nix/store/pjn9iin3qyhinadzp0amj4p7ykg8iphb-pypy3.11-7.3.19/lib/pypy3.11
bcrypt> warning: build failed, waiting for other jobs to finish...
bcrypt> error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/_bcrypt/Cargo.toml --target x86_64-unknown-linux-gnu --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code 101
bcrypt>
bcrypt> ERROR Backend subprocess exited when trying to invoke build_wheel
and if that builds, I am convinced the layout is now better than before.
Contributor
Author
|
The build fails, however, there exists _sysconfigdata.py |
Member
|
But before it did not at that location, so I consider this a win. |
Member
|
Debian's PyPy3 has a |
Member
|
I did this #240301 a long time ago... |
Contributor
Author
|
I have tried to build bcrypt from pypy311_prebuilt, also failed
|
Member
|
This got added in pypy/pypy@1588a4e upstream but disabled in pypy/pypy@7802d07 due to portable builds which is not applicable to us. I've asked here pypy/pypy#5015 (comment) to reenable it again. I think I am going to patch that into postInstall to fix that. |
Contributor
Author
|
We should gen sysconfigdata by ourself, or wait for upstream change? Debian use their own scripts generating sysconfigdata: https://sources.debian.org/src/pypy3/7.3.20%2Bdfsg-3/debian/scripts/gen-sysconfigdata.sh |
Member
That will take to long. I am cooking something locally but compiling pypy3 often takes way to long... |
Member
|
I did something quick to test: diff --git a/pkgs/development/interpreters/python/pypy/default.nix b/pkgs/development/interpreters/python/pypy/default.nix
index 723f34740cd4..c0439aec18ed 100644
--- a/pkgs/development/interpreters/python/pypy/default.nix
+++ b/pkgs/development/interpreters/python/pypy/default.nix
@@ -3,6 +3,7 @@
stdenv,
replaceVars,
fetchurl,
+ autoconf,
zlibSupport ? true,
zlib,
bzip2,
@@ -22,7 +23,13 @@
db,
xz,
python-setup-hook,
- optimizationLevel ? "jit",
+ optimizationLevel ? "1",
+ # extra args to pass to rpython when running pypy/goal/targetpypystandalone.py
+ # Using --no-compile does not work as pypy-c is then missing
+ # --no-backendopt does not work because lib_pypy is then missing
+ # To speed things up use:
+ rpythonArgs ? [ "--no-annotate" "--no-rtype" ],
+ # rpythonArgs ? [ ],
boehmgc,
# For the Python package set
hash,
@@ -150,7 +157,7 @@
# 3. ld -t (where it attaches the values in $LD_LIBRARY_PATH as -L arguments)
# The first is disabled in Nix (and wouldn't work in the build sandbox or on NixOS anyway), and
# the third was only introduced in Python 3.6 (see bugs.python.org/issue9998), so is not
- # available when buliding PyPy (which is built using Python/PyPy 2.7).
+ # available when building PyPy (which is built using Python/PyPy 2.7).
# The second requires SONAME to be set for the dynamic library for the second part not to fail.
# As libsqlite3 stopped shipping with SONAME after the switch to autosetup (>= 3.50 in Nixpkgs;
# see https://www.sqlite.org/src/forumpost/5a3b44f510df8ded). This makes the Python CFFI module
@@ -177,7 +184,10 @@
${pythonForPypy.interpreter} rpython/bin/rpython \
--make-jobs="$NIX_BUILD_CORES" \
-O${optimizationLevel} \
- --batch pypy/goal/targetpypystandalone.py
+ --batch \
+ ${lib.concatStringsSep " " rpythonArgs} \
+ pypy/goal/targetpypystandalone.py \
+ ${lib.optionalString ((toString optimizationLevel) == "1") "--withoutmod-cpyext"}
runHook postBuild
'';
@@ -202,6 +212,17 @@
if isPy38OrNewer then sitePackages else "lib/${libPrefix}/${sitePackages}"
}/sitecustomize.py
+ # Create platform specific _sysconfigdata__*.py (eg: _sysconfigdata__linux_x86_64-linux-gnu.py)
+ # Can be tested by building: pypy3Packages.bcrypt
+ # Based on the upstream build code found here:
+ # https://github.com/pypy/pypy/blob/release-pypy3.11-v7.3.20/pypy/tool/release/package.py#L176-L189
+ # Upstream is not shipping config.guess, just take one from autoconf
+ $out/bin/pypy3 -m sysconfig --generate-posix-vars HOST_GNU_TYPE "$(${autoconf}/share/autoconf/build-aux/config.guess)"
+ buildir="$(cat pybuilddir.txt)"
+ quadruplet=$(ls $buildir | sed -E 's/_sysconfigdata__(.*).py/\1/')
+ cp "$buildir/_sysconfigdata__$quadruplet.py" $out/lib_pypy/
+ ln -rs "$out/lib_pypy/_sysconfigdata__$quadruplet.py" $out/lib/pypy*/
+
runHook postInstall
'';Still need testing. When this works, I am going to commit it and push it onto this PR. |
Contributor
Author
|
can we just as the comment pypy/pypy#5015 (comment) suggested? |
SuperSandro2000
force-pushed
the
pypy
branch
from
57cfcfa to
2a7c18f
Compare
Member
|
I've successfull built pypy3Packages.bcrypt. I needed to rebase and also fixed some other erorrs. @qbisi are you happy with the other changes? If so, I would merge this |
Contributor
Author
|
|
Contributor
Author
|
Contributor
Author
|
Looks like we generate sysconfigdata before we fixup the rpath of libpypy3.11-c.dylib. |
SuperSandro2000
force-pushed
the
pypy
branch
from
60d2b3e to
7b2776c
Compare
SuperSandro2000
approved these changes
Sep 16, 2025
nixpkgs-ci
bot
added
the
12.approvals: 1
SuperSandro2000
approved these changes
Sep 16, 2025
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
part of
#442206#442540prerequset for wrapping python executable with
--inherit-argv0in python-env, otherwise issue #301498 will happen.IMO, pypy{2,3} is very different from pypy{2,3}_prebuilt in tree structure, this pr make it more consistent with prebuilt binary.
tested work on aarch64-darwin.
Things done
passthru.tests.nixpkgs-reviewon this PR. See nixpkgs-review usage../result/bin/.Add a 👍 reaction to pull requests you find important.