Is there a way to include missing shared libraries into the wheel on linux?

[olebedev]

Currently the linux wheel of v87.6 doesn't contain the following shared libraries:

• libGL.so.1
• libexpat.so.1
• libz.so.1
• libstdc++.so.6

Is there a way to include those into the wheel itself?

I've looked at the cibuildwheel module and seems to run the auditwheel repair as a part of the wheel build. However, the libraries I mentioned are not included into the artifact and I don't understand why.

[HinTak]

You probably are not looking carefully enough? Those 4 libraries are mentioned in:
https://github.com/pypa/auditwheel/blob/main/src/auditwheel/policy/manylinux-policy.json

In any case, I think this question really belongs to the auditwheel repo...

At a guess, I would think if you hack around that file and run your own hacked auditwheel, you could achieve what you want - but it is probably not a good idea: I don't know about the other three, but I am pretty sure python itself depends on libz.

[HinTak]

This explains libz and libexpat pypa/auditwheel#152 , both of which python itself depends on and therefore no need ever to bundle.

[HinTak]

Besides 'python itself depends on', apparently 'extremely stable api' (compiled against old always works with new) is another reason - that would apply to libGL and libstdc++. Anyway, this is really a discussion about what audiwheel does or does not do.

[HinTak]

What is and what is not bundled, is decided by pypa/auditwheel#192 , I think. Final answer.

[HinTak]

LibGL and libstdc++ are listed in https://peps.python.org/pep-0599/#the-manylinux2014-policy as not to bundle.

[olebedev]

Hi @HinTak, thanks for the links! This is super helpful, as always!

LibGL and libstdc++ are listed in https://peps.python.org/pep-0599/#the-manylinux2014-policy as not to bundle.

I actually think it's the other way around, it's fine depend on those libraries without having them bundled, no?

[HinTak]

Yes, the idea is basically either that the libraries are generally available with stable api and the later system ones are as good as any, or that python itself depends on, and therefore are available.

[HinTak]

@olebedev btw, it has been mentioned before, m87 is over 4 years old... Google typically supports 4 to 5 milestones (or 6 month-ish), and we are onto m132 now; and there are a lot of exciting new functionalities. We haven't had any major breakage/withdrawal since 116, and there are still two main areas of breakage between 87 and 116: codecs and filters. The latter is likely replaced by the sksl runtime effect stuff. Anyway, it is likely 2-3 months of full-time dev work to try to support most 87-like functionality. Please consider budgeting/funding/sponsoring that at the organizational level. (And get in touch privately)