Add pypy3-exe 7.3.0 to overlay

[mtottenh]

pypy3 musl compatibilty patches

When trying to install pypy3, the default dependency calculation will opt to
use 'pypy' to build pypy3, it's the recommended way of building pypy3 by
upstream, because pypy can compile pypy3 much faster than CPython can:

• Using python2 to perform the translation. Please note that upstream
• recommends using PyPy for that. If you wish to do so, please install
• dev-python/pypy and ensure that EPYTHON variable is unset

However, dev-python/pypy pulls in a dependency on dev-python/pypy-exe-bin (as
the base 'pypy' package needs to be compiled somehow), which unfortunately is
linked against glibc. Attempting to run that binary gives the all familiar
'file not found' when it fails to find musl's dynamic loader.

#Possible approaches

Provide a musl dev-python/pypy-exe-bin

This would involve creating a dev-python/pypy-exe-bin on a musl system
(presumably by bootstrapping with CPython), and handwaving about how to resolve
the musl version of the package against the glibc version of the bin package.

Bootstrap pypy3 with CPython instead

Gentoo/musl already has a working CPython interpreter. So bootstrapping pypy3
with CPython shouldn't be difficult, it just takes longer. All that is required
to force the dev-python/pypy3 ebuild to use CPython is to add the following to
/etc/portage/package.mask:

$ echo "dev-python/pypy::gentoo" >> /etc/portage/package.mask
$ equery depgraph dev-python/pypy3

* dependency graph for dev-python/pypy3-7.3.0
 `--  dev-python/pypy3-7.3.0  amd64
   `--  dev-python/pypy3-exe-7.3.0  (dev-python/pypy3-exe) amd64  [bzip2? ncurses?]
   `--  dev-python/pypy3-exe-bin-7.3.0_p1  (dev-python/pypy3-exe-bin) amd64
   `--  dev-libs/openssl-1.1.1g  (dev-libs/openssl) amd64
   `--  dev-libs/libressl-3.0.2  (dev-libs/libressl) amd64
   `--  sys-libs/gdbm-1.13-r2  (sys-libs/gdbm) amd64
   `--  dev-db/sqlite-3.31.1  (dev-db/sqlite) amd64
   `--  dev-lang/tk-8.6.8  (dev-lang/tk) amd64
   `--  dev-tcltk/tix-8.4.3-r1  (dev-tcltk/tix) amd64
   `--  dev-lang/python-2.7.18  (>=dev-lang/python-2.7.5-r2) amd64
[ dev-python/pypy3-7.3.0 stats: packages (10), max depth (1) ]

Great it's pulling in python 2.7 for bootstrapping.

Fixing the build

Attempting to build the package leads to three compilation failures. The first
is a known issue, and has a PR open in upstream CPython with a fix:

python/cpython#18380

Failing to patch dev-lang/python2_7 with the above will cause the build of
dev-python/pypy3-exe to complain about being unable to detect the system libc.

The other two failures occur when attempting to compile dev-python/pypy3-exe
during the bootstrapping phase and are issues to do with assumptions pypy3 has
made about the system libc which only hold true for glibc based systems.

Missing definition of struct timeval

Several files in the pypy3 codebase refer to struct timeval, in fact
pypy/module/cpyext/include/pytime.h uses a pointer to timeval structs when
defining a bunch of function prototypes, however it neglects to include
<sys/time.h>. This leads to a bunch of warnings:

/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython/../pypy/module/cpyext/include/pytime.h:122:12: warning: _struct timeval_ declared inside parameter list will not be visible outside of this definitio
n or declaration
  122 |     struct timeval *tv,

whereas pypy/module/cpyext/src/pytime.c does include
<sys/time.h>, this leads to conflicting type definitions for the functions
defined in pytime.h (gcc uses int* to represent the unknown structure ), and thus a compile error:

pytime.c:506:1: error: conflicting types for __PyTime_AsTimeval_
  506 | _PyTime_AsTimeval(_PyTime_t t, struct timeval *tv, _PyTime_round_t round)
      | ^~~~~~~~~~~~~~~~~
In file included from /var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython/../pypy/module/cpyext/include/Python.h:84,
                 from pytime.c:1:
/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython/../pypy/module/cpyext/include/pytime.h:121:17: note: previous declaration of __PyTime_AsTimeval_ was here
  121 | PyAPI_FUNC(int) _PyTime_AsTimeval(_PyTime_t t,
      |                 ^~~~~~~~~~~~~~~~~

The fix is relatively simple. Patch pytime.h to include <sys/time.h>. Either by
patching pytime.h directly, or by patching Python.h (which is where many of the
c library includes are centralised in pypy's codebase). A diff for the latter
looks like as follows:

diff --git a/pypy/module/cpyext/include/Python.h b/pypy/module/cpyext/include/Python.h
index 409f1f0..e20ef40 100644
--- a/pypy/module/cpyext/include/Python.h
+++ b/pypy/module/cpyext/include/Python.h
@@ -13,6 +13,7 @@
 # include <math.h>
 # include <errno.h>
 # include <unistd.h>
+# include <sys/time.h>
 # define Py_DEPRECATED(VERSION_UNUSED) __attribute__((__deprecated__))
 # define PyAPI_FUNC(RTYPE) __attribute__((visibility("default"))) RTYPE
 # define PyAPI_DATA(RTYPE) extern PyAPI_FUNC(RTYPE)
--
2.26.2

Re-definition of stdout

At some point during the package build, pypy generates C code from rpython
scripts that re-defines stdout to extern FILE*:

$ rg 'extern FILE\* stdout'
../../temp/usession-release-pypy3.6-v7.3.0rc4-0/module_cache/module_14.c
516:#include <stdio.h>
517:#include <sys/types.h>
518:#include <unistd.h>
519:extern FILE* stdout;

This conflicts with the musl definition in stdio.h of extern FILE* const,
this leads to the following build error:

./module_cache/module_17.c -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/translator/c -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/cpyext/inc
lude -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/cpyext/parse -I.. -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/_codecs -I"/
var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/rlib/rvmprof/src -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/_multibytecodec -I"/var/tmp/portage/d
ev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/rlib/rjitlog/src -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/rlib/rvmprof/src/shared -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.
0/work/pypy3.6-v7.3.0-src/rpython"/rlib/rvmprof/src/shared/libbacktrace -I/usr/lib/libffi/include -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/_cffi_backend/src -I"/var/tmp/po
rtage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/faulthandler -I"/var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/_posixsubprocess -I"/var/tmp/portage
/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython"/../pypy/module/operator -I/usr/include/ncursesw
In file included from /var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython/../pypy/module/cpyext/include/object.h:4,
                 from /var/tmp/portage/dev-python/pypy3-exe-7.3.0/work/pypy3.6-v7.3.0-src/rpython/../pypy/module/cpyext/include/Python.h:80,
                 from ../module_cache/module_14.c:93:
../module_cache/module_14.c:519:14: error: conflicting type qualifiers for _stdout_
  519 | extern FILE* stdout;
      |              ^~~~~~
/usr/include/stdio.h:61:20: note: previous declaration of _stdout_ was here
   61 | extern FILE *const stdout;
      |                    ^~~~~~
make: *** [Makefile:895: ../module_cache/module_14.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/var/tmp/portage/dev-python/pypy3-exe-7.3.0/temp/usession-release-pypy3.6-v7.3.0rc4-0/testing_1'

full build log
Fixing this is similarly trivial, patching rpython/rlib/rfile.py to emit the
correct type does the trick.

 rpython/rlib/rfile.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/rpython/rlib/rfile.py b/rpython/rlib/rfile.py
index c135cfb..d17c9a7 100644
--- a/rpython/rlib/rfile.py
+++ b/rpython/rlib/rfile.py
@@ -106,11 +106,11 @@ c_feof = llexternal('feof', [FILEP], rffi.INT)
 c_ferror = llexternal('ferror', [FILEP], rffi.INT)
 c_clearerr = llexternal('clearerr', [FILEP], lltype.Void)

-c_stdin = rffi.CExternVariable(FILEP, 'stdin', eci, c_type='FILE*',
+c_stdin = rffi.CExternVariable(FILEP, 'stdin', eci, c_type='FILE* const',
                                getter_only=True)
-c_stdout = rffi.CExternVariable(FILEP, 'stdout', eci, c_type='FILE*',
+c_stdout = rffi.CExternVariable(FILEP, 'stdout', eci, c_type='FILE* const',
                                 getter_only=True)
-c_stderr = rffi.CExternVariable(FILEP, 'stderr', eci, c_type='FILE*',
+c_stderr = rffi.CExternVariable(FILEP, 'stderr', eci, c_type='FILE* const',
                                 getter_only=True)


--
2.26.2

With that. The package builds and installs, enough for pypy3 to print hello world.

Hopefully I've added all the required files/information to get this ebuild working, I'll give it another test tonight by attempting to install from this ebuild without patches in my /etc/portage/patches/... and update the PR if necessary.

[blueness]

pushed.

[anarchpenguin]

Please make sure you use repoman to prevent errors from creeping into the overlay.