Would it be possible to keep libpython.a?

[rdb]

The application I'm trying to build has a particular need to embed the interpreter into an executable, which requires static linking with libpython.

Would it perhaps be possible not to delete libpython.a from the image, but keep it in the _internal directory for those who are absolutely certain they need to use it?

[Moguri]

Can we get some discussion going on this? Is this something that is a complete non-starter, or are pull requests welcome? What are the issues/concerns with something like this? Larger containers? Are wheels affected? Maybe this is something we can discuss for manylinux2010 if there is hesitation to have it as part of manylinux1.

[njsmith]

I think this is out of scope for manylinux, since it's about building portable binaries of your application, not about building manylinux wheels. You're welcome to reuse our scripts and so on, but i don't want to add it to the image. It's a lot of extra download weight.

[njsmith]

Note that if you start with something like the "holy build box" then building a copy of python to embed is probably not too difficult?

[Moguri]

This is for building Panda3D wheels. We need libpython.a to provide users of the Panda3D wheel the necessary tools to deploy/freeze their Panda3D applications. At the moment, we have to keep re-integrating upstream manylinux1 changes into our Docker images for building Panda3D wheels, and it is getting tiresome. It would be much preferable to start with a manylinux1 base image.

Would you be open to discussion around making the manylinux1 scripts easier to modify? For example, if removing the libpython.a files was done in another layer of the image, we could just worry about modifying the Dockerfile (remove the line to remove libpython.a and add necessary packages for Panda3D) instead of modifying both the Dockerfile and the build scripts. This would make merging in upstream changes easier.

[njsmith]

Huh, so you're... shipping a copy of Python, inside your Python wheels?

[rdb]

Sort of; we're shipping a stub executable that embeds the interpreter and runs whatever Python bytecode is appended to the end of the executable, similar to what py2exe does. This allows users to easily compile their Python code into executables; we just need to grab the appropriate .whl of the dependency packages and inject their bytecode into the stub.

Compiling this stub requires linking in libpython.a. It would save us a significant maintenance burden if there could be some easy way to have manylinux1 contain the libpython.a file somewhere. If not, we'll just have to continue to maintain our own fork of manylinux. That said, perhaps it could be an option to enable when building the docker images, eg. via an environment variable?

[SylvainCorlay]

@Moguri we are finding the exact same usecase for our project and would love to see libpython.a included in the docker image.

Note that we are currently building our own docker image from a fork of the manylinux repository. Unfortunately, your base docker image is private making it not reproducible.

[Moguri]

@SylvainCorlay The dockerfile we are using can be found here.

[SylvainCorlay]

@Moguri thanks. I meant to say that the base docker image of the official manylinux (https://github.com/pypa/manylinux/blob/master/docker/Dockerfile-x86_64#L2) is private.

I have opened #481 about this.

[rdb]

I would kindly like to resubmit this for consideration. Until now, we have been able to build our own fork of manylinux that removes the rm libpython*.a command, but we would like to start offering aarch64 wheels, and it turns out to be prohibitive for us to build our own aarch64 manylinux image (we don't have access to the right hardware, and an attempt to build via qemu ground to a complete halt after 12 hours).

It would really help us a lot to have libpython kept in the Docker image, even if it's moved somewhere totally out of sight, so that we can continue building our executable that embeds the Python interpreter.