pypy: cleanup install layout, fix sysconfig paths by SuperSandro2000 · Pull Request #240301 · NixOS/nixpkgs

SuperSandro2000 · 2023-06-28T08:39:46Z

Description of changes

Fixes the same issue as #240187 but properly

Before:

>>>> import sys
>>>> sys.prefix
'/nix/store/dvrdpvw4j47q7snprr5xpkiqnwhsw4ck-pypy3.9-7.3.11/pypy3.9-c'
>>>> sys.exec_prefix
'/nix/store/dvrdpvw4j47q7snprr5xpkiqnwhsw4ck-pypy3.9-7.3.11/pypy3.9-c'

>>>> import sysconfig; sysconfig._main()
Platform: "linux-x86_64"
Python version: "3.9"
Current installation scheme: "posix_prefix"

Paths:
        data = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        include = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        platinclude = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        platlib = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/lib/pypy3.9/site-packages"
        platstdlib = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/lib/pypy3.9"
        purelib = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/lib/pypy3.9/site-packages"
        scripts = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/bin"
        stdlib = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/lib/pypy3.9"

Variables:
        ABIFLAGS = ""
        AR = "ar"
        ARFLAGS = "rc"
        CC = "gcc -pthread"
        CCSHARED = "-fPIC"
        CFLAGS = "-DNDEBUG -O2"
        CONFINCLUDEPY = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        CXX = "g++ -pthread"
        EXE = ""
        EXT_SUFFIX = ".pypy39-pp73-x86_64-linux-gnu.so"
        GNULD = "yes"
        INCLUDEPY = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        LDFLAGS = "-Wl,-Bsymbolic-functions"
        LDLIBRARY = "libpypy3.9-c.so"
        LDSHARED = "gcc -pthread -shared -Wl,-Bsymbolic-functions"
        LDVERSION = "3.9"
        LIBDIR = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/bin"
        LIBRARY = ""
        MULTIARCH = "x86_64-linux-gnu"
        OPT = "-DNDEBUG -O2"
        Py_DEBUG = "0"
        Py_ENABLE_SHARED = "0"
        SHLIB_SUFFIX = ".so"
        SIZEOF_VOID_P = "8"
        SO = ".pypy39-pp73-x86_64-linux-gnu.so"
        SOABI = "pypy39-pp73"
        TZPATH = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/share/zoneinfo:/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/lib/zoneinfo:/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/share/lib/zoneinfo:/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/../etc/zoneinfo:/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo"
        VERSION = "3.9"
        abiflags = ""
        base = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        exec_prefix = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        implementation = "PyPy"
        implementation_lower = "pypy"
        installed_base = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        installed_platbase = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        platbase = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        platlibdir = "lib"
        prefix = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9"
        projectbase = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/bin"
        py_version = "3.9.16"
        py_version_nodot = "39"
        py_version_short = "3.9"
        srcdir = "/nix/store/nc4h7pwpn4gnlmn6ykz1x0z3rsxbamrq-pypy3.9-7.3.11/lib/pypy3.9/bin"
        userbase = "/home/sandro/.local"
User:
        data = "/home/sandro/.local"
        include = "/home/sandro/.local/include/pypy3.9"
        platlib = "/home/sandro/.local/lib/pypy3.9/site-packages"
        platstdlib = "/home/sandro/.local/lib/pypy3.9"
        purelib = "/home/sandro/.local/lib/pypy3.9/site-packages"
        scripts = "/home/sandro/.local/bin"
        stdlib = "/home/sandro/.local/lib/pypy3.9"

What it should look like according to Debian:

Python 3.7.10 (7.3.5+dfsg-2+deb11u2, Nov 01 2022, 20:16:36)
[PyPy 7.3.5 with GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import sys; sys.prefix; sys.exec_prefix
'/usr/lib/pypy3'
'/usr/lib/pypy3'
>>>> site.getsitepackages()
['/usr/local/lib/pypy3.7/dist-packages', '/usr/lib/python3/dist-packages']

After:

$ ./result/bin/pypy3
Python 3.9.16 (feeb267ead3e6771d3f2f49b83e1894839f64fb7, Jun 28 2023, 07:04:39)
[PyPy 7.3.11 with GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import setuptools
>>>> import sys ; sys.prefix ; sys.exec_prefix
'/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env'
'/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env'
>>>> import sysconfig; sysconfig._main()
Platform: "linux-x86_64"
Python version: "3.9"
Current installation scheme: "posix_prefix"

Paths:
        data = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        include = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        platinclude = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        platlib = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        platstdlib = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        purelib = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env/lib/pypy3.9/site-packages"
        scripts = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env/bin"
        stdlib = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9"

Variables:
        ABIFLAGS = ""
        AR = "ar"
        ARFLAGS = "rc"
        CC = "cc -pthread"
        CCSHARED = "-fPIC"
        CFLAGS = "-DNDEBUG -O2"
        CONFINCLUDEPY = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        CXX = "c++ -pthread"
        EXE = ""
        EXT_SUFFIX = ".pypy39-pp73-x86_64-linux-gnu.so"
        INCLUDEPY = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/include/pypy3.9"
        LDCXXSHARED = "c++ -shared"
        LDFLAGS = "-Wl,-Bsymbolic-functions"
        LDLIBRARY = "libpypy3.9-c.so"
        LDSHARED = "cc -pthread -shared -Wl,-Bsymbolic-functions"
        LDVERSION = "3.9"
        LIBDIR = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/bin"
        LIBRARY = ""
        MULTIARCH = "x86_64-linux-gnu"
        OPT = "-DNDEBUG -O2"
        Py_DEBUG = "0"
        Py_ENABLE_SHARED = "0"
        SHLIB_SUFFIX = ".so"
        SIZEOF_VOID_P = "8"
        SO = ".pypy39-pp73-x86_64-linux-gnu.so"
        SOABI = "pypy39-pp73"
        TZPATH = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/share/zoneinfo:/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/lib/zoneinfo:/
nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/share/lib/zoneinfo:/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9/../etc/zoneinfo:/usr/share/zo
neinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo"
        VERSION = "3.9"
        abiflags = ""
        base = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        exec_prefix = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        implementation = "PyPy"
        implementation_lower = "pypy"
        installed_base = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9"
        installed_platbase = "/nix/store/rjw1k8grc7y93z7g8ns6zbvg0ss128m9-pypy3.9-7.3.11/lib/pypy3.9"
        platbase = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        platlibdir = "lib"
        prefix = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env"
        projectbase = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env/bin"
        py_version = "3.9.16"
        py_version_nodot = "39"
        py_version_short = "3.9"
        srcdir = "/nix/store/qnmhhfhllzn259jf4afll4nvqag679yc-pypy3.9-7.3.11-env/bin"
        userbase = "/home/hotpi/.local"
User:
        data = "/home/hotpi/.local"
        include = "/home/hotpi/.local/include/pypy3.9"
        platlib = "/home/hotpi/.local/lib/pypy3.9/site-packages"
        platstdlib = "/home/hotpi/.local/lib/pypy3.9"
        purelib = "/home/hotpi/.local/lib/pypy3.9/site-packages"
        scripts = "/home/hotpi/.local/bin"
        stdlib = "/home/hotpi/.local/lib/pypy3.9"

Things done

999eagle · 2023-06-28T13:57:51Z

Cherry-picking this together with #240300 on top of nixos-unstable builds fine, but building pypy3Packages.setuptools fails with error: invalid command 'bdist_wheel'.

999eagle · 2023-06-28T14:23:53Z

Cherry-picked again with the new changes, now pypy3 and pypy3Packages.setuptools build fine. A nix-shell -p 'pypy3.withPackages (ps: with ps; [setuptools])' also works and import setuptools resolves correctly.

SuperSandro2000 · 2023-07-02T02:53:56Z

I have added a patch which fixes the include directory under $out/include and in turn also a fix for pybind11 which is very tightly coupled to this.

@ofborg build pypy3 pypy3.pkgs.pybind11

SuperSandro2000 · 2023-07-02T14:36:52Z

That's the new, more standard, include dir path

 ➜ ls ./result/include/pypy3.9/
 abstract.h          cpyext_datetime.h        datetime.h      funcobject.h           marshal.h        pycapsule.h     pyport.h                   pystrtod.h       structseq.h
 boolobject.h        cpyext_descrobject.h     descrobject.h   genericaliasobject.h   memoryobject.h   pyconfig.h      pypy_decl.h                Python.h         sysmodule.h
 bytearrayobject.h   cpyext_genobject.h       dictobject.h    genobject.h            methodobject.h   pyerrors.h      pypy_macros.h              pythonrun.h      traceback.h
 bytesobject.h       cpyext_memoryobject.h    eval.h          graminit.h             missing.h        pyhash.h        pypy_marshal_decl.h        pythread.h       tupleobject.h
 ceval.h             cpyext_moduleobject.h    exports.h       import.h               modsupport.h     pylifecycle.h   pypy_structmember_decl.h   pytime.h         typeslots.h
 code.h              cpyext_object.h          fileobject.h    listobject.h           moduleobject.h   pymacro.h       pysignals.h                setobject.h      unicodeobject.h
 compile.h           cpyext_unicodeobject.h   floatobject.h   longintrepr.h          object.h         pymath.h        pystate.h                  sliceobject.h    warnings.h
 complexobject.h     cStringIO.h              frameobject.h   longobject.h           patchlevel.h     pymem.h         pystrhex.h                 structmember.h

SuperSandro2000 · 2023-07-09T21:33:39Z

@FRidh can you take a look at this, please? This is vital for have any kind of pypy env support in nixpkgs.

nixos-discourse · 2023-11-04T00:23:57Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2868

thiagokokada · 2023-11-20T14:06:17Z

@ofborg build pypy27 pypy39 pypy310

SuperSandro2000 · 2023-12-10T19:03:42Z

@ofborg build pypy27 pypy39 pypy310

Co-authored-by: Sophie Tauchert <1221984+999eagle@users.noreply.github.com>

SuperSandro2000 requested a review from FRidh as a code owner June 28, 2023 08:39

SuperSandro2000 mentioned this pull request Jun 28, 2023

pypy3: fix sitecustomize #240187

Closed

12 tasks

github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jun 28, 2023

SuperSandro2000 force-pushed the pypy-site-packages branch from 9f31bb5 to cdb4fd4 Compare June 28, 2023 09:16

ofborg bot requested a review from andersk June 28, 2023 09:18

ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jun 28, 2023

SuperSandro2000 force-pushed the pypy-site-packages branch from cdb4fd4 to f0cf662 Compare June 28, 2023 13:59

SuperSandro2000 force-pushed the pypy-site-packages branch from 068100c to 29d5ce0 Compare July 2, 2023 14:38

SuperSandro2000 force-pushed the pypy-site-packages branch 2 times, most recently from e7dce73 to 467d753 Compare November 4, 2023 00:22

ofborg bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Nov 4, 2023

SuperSandro2000 requested a review from winterqt November 20, 2023 10:52

SuperSandro2000 force-pushed the pypy-site-packages branch from 467d753 to 25ad49a Compare November 20, 2023 10:52

SuperSandro2000 requested a review from thiagokokada November 20, 2023 10:52

thiagokokada approved these changes Nov 20, 2023

View reviewed changes

SuperSandro2000 force-pushed the pypy-site-packages branch from 25ad49a to ec44187 Compare December 10, 2023 19:02

SuperSandro2000 requested a review from NickCao January 14, 2024 19:17

SuperSandro2000 force-pushed the pypy-site-packages branch 2 times, most recently from 5fb6acb to 602631c Compare January 28, 2024 22:11

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 27, 2024

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 2, 2025

SuperSandro2000 and others added 2 commits January 25, 2025 20:10

pypy: cleanup install layout, fix sysconfig paths

4500a65

Co-authored-by: Sophie Tauchert <1221984+999eagle@users.noreply.github.com>

python310Packages.pybind11: fix include directory for pypy

3ceab2d

SuperSandro2000 force-pushed the pypy-site-packages branch from 602631c to 3ceab2d Compare January 25, 2025 19:10

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 25, 2025

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 25, 2025

nix-owners bot requested review from mweinelt and natsukium January 25, 2025 19:12

github-actions bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. and removed 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Jan 25, 2025

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Apr 7, 2025

nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jun 25, 2025

nixpkgs-ci bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 25, 2025

SuperSandro2000 mentioned this pull request Sep 15, 2025

pypy: fix installPhase #442224

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

pypy: cleanup install layout, fix sysconfig paths#240301

pypy: cleanup install layout, fix sysconfig paths#240301
SuperSandro2000 wants to merge 2 commits intoNixOS:stagingfrom
SuperSandro2000:pypy-site-packages

SuperSandro2000 commented Jun 28, 2023 •

edited

Loading

Uh oh!

999eagle commented Jun 28, 2023

Uh oh!

999eagle commented Jun 28, 2023

Uh oh!

SuperSandro2000 commented Jul 2, 2023 •

edited

Loading

Uh oh!

SuperSandro2000 commented Jul 2, 2023

Uh oh!

SuperSandro2000 commented Jul 9, 2023

Uh oh!

nixos-discourse commented Nov 4, 2023

Uh oh!

thiagokokada commented Nov 20, 2023

Uh oh!

SuperSandro2000 commented Dec 10, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

SuperSandro2000 commented Jun 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

999eagle commented Jun 28, 2023

Uh oh!

999eagle commented Jun 28, 2023

Uh oh!

SuperSandro2000 commented Jul 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SuperSandro2000 commented Jul 2, 2023

Uh oh!

SuperSandro2000 commented Jul 9, 2023

Uh oh!

nixos-discourse commented Nov 4, 2023

Uh oh!

thiagokokada commented Nov 20, 2023

Uh oh!

SuperSandro2000 commented Dec 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

SuperSandro2000 commented Jun 28, 2023 •

edited

Loading

SuperSandro2000 commented Jul 2, 2023 •

edited

Loading

SuperSandro2000 commented Dec 10, 2023 •

edited

Loading