Tracking issue for manylinux2014 rollout

[brainwane]

Now that PEP 599 has been accepted to adopt the manylinux2014 standard, we need a tracking issue for implementation (I'm reusing structure and prose from #179).

The dates are guesses but would (as I figure) help us get all the infrastructure in place well before Nov 2020 when CentOS 6 (supported in manylinux2010) will go EOL, and in time for the August 2020 date that @gunan asked for; please argue with me. ;-)

There are a number of further steps needed before folks can actually make use of that new baseline, and the order of operations matters (since building manylinux2014 wheels isn't very useful if installers won't install them):

Prep (ideally done by 2 September 2019):

• Gather personnel: tentatively strategize re: who's available to review & merge code, especially for repositories where maintainer time is scarce, re: who will make a build environment, and re: who will do testing or build testing infrastructure
• Curate initial list of test cases regarding build environment, target Linux distributions, specific Python packages to be built, presence/absence of specific libraries

Essential client support (ideally done by 15 September 2019):

• Update pip 19.x to install manylinux2014 wheels
  • Update packaging.tags to detect manylinux2014 compatibility (Add support for manylinux2014 packaging#186)
  • New release of packaging (19.2)
  • Add manylinux2014 support to pip._internal.pep425tags (Add manylinux2014 support pip#7102)
  • (Optional) Replace pip._internal.pep425tags w/ packaging.tags (Replace pep425tags w/ packaging.tags pip#6908)
  • New release of pip (19.3 due out in week 41, so, by 19 October)

Enable publication of manylinux2014 wheel archives (ideally done by 30 October 2019):

• Update Warehouse (PyPI) to accept manylinux2014 wheel uploads (Allow manylinux2014 wheel upload pypi/warehouse#6684)
• Update auditwheel to emit manylinux2014 wheel archives (manylinux2014 policy auditwheel#192)
  • Update auditwheel to allow new architectures introduced with manylinux2014 (Add support for new architectures in manylinux2014 auditwheel#202)
  • Pre-release of auditwheel (3.0.0.0rc1 Release 3.0.0 auditwheel#205)
  • New release of auditwheel (3.0.0 Release 3.0.0 auditwheel#205)
• Provide a manylinux2014 build environment, probably a Docker image (manylinux2014 images #348)
  • create quay.io repositories
• Confirm that twine will upload manylinux2014 wheel archives

Management of transition from manylinux1 and manylinux2010 (ideally done by 15 December 2019):

• Update https://packaging.python.org/specifications/platform-compatibility-tags/ to cover manylinux2014, with the caveat that it doesn't actually have broad tooling support yet, and to explain (per @zackw's note) pros & cons of switching to manylinux2014, e.g., future security patches, old Linux distributions new wheels won't work with, etc.
• Reword the "not yet widely supported" caveat from the platform compatibility tagging page on PyPUG

Additional projects to consider once core capability support has rolled out (ideally done by 1 March 2020):

• Resolve Why does warehouse allow linux_armv6l and linux_armv7l wheels? pypi/warehouse#3668
• Update pipenv to install manylinux2014 wheels (needed due to pipenv vendoring pip, so the support depends on the version of pipenv being used, rather than the version of pip in the host or target environments)
• Option to target manylinux2014 instead of manylinux2010 in cibuildwheel (Add support for manylinux2014 cibuildwheel#216, Add manylinux2014 x86_64/i686 support cibuildwheel#215)
• Option to target manylinux2014 instead of manylinux1 or manylinux2010 in multibuild
• Distinguish manylinux2014, manylinux2010 and manylinux1 images in dockcross
• Develop list of further relevant helper utilities and update them
• ... any other suggestions?

Tagging @di as the PEP 599 author, @pfmoore as the BDFL-Delegate who requested further planning on rollout, and @ewdurbin from the PyPI/PSF side of things.

[trishankatdatadog]

Minor comment, and probably too late now that the PEP has been accepted, but I feel it's relevant to avoid confusion: could we call it manylinux2024 (when Centos 7 actually expires) instead of manylinux2014 (sounds old and deprecated)?

[cjerdonek]

Regarding this task:

Replace pip._internal.pep425tags w/ packaging.tags

Is switching pip to use packaging.tags really a blocker formanylinux2014? @di's manylinux2014 patch was super tiny, so it seems like something similar could be done to pip's pep425tags for the short-term. I'm just concerned because it seems like switching pip to use packaging.tags is relatively a much bigger task, with things to do that aren't related to the manylinux2014 changes. (These additional tasks aren't currently reflected in the checklist above.) So I'd rather not rush the switch if there isn't really a need to. With this approach, switching to packaging.tags could be done for the pip release after the next.

[di]

Is switching pip to use packaging.tags really a blocker for manylinux2014?

No, definitely not, just figured it'd be nice to roll this in together since it seems like both are being worked on at the same time.

[di]

I just merged pypi/warehouse#6684, manylinux2014 wheels can now be uploaded to PyPI.

[xavfernandez]

I've taken the liberty to add pypa/pip#7102 to the rollout.