feat: add aarch64 manylinux wheels

[mayeut]

Summary

• OS: Linux
• Bug fix: no
• Type: wheels
• Fixes:
  fixes [Linux] Publish armv7l & aarch64 wheels #1966
  fixes [Linux][aarch64] Wheel support for aarch64 Linux #1972
  fixes ARM64 support #2206
  mitigates [AWS Graviton2] graviton gcc flags break pip install #2090
  mitigates [Ubuntu18.04] failed to install, gcc error #2175
  mitigates [Asahi Fedora] Building from source fails on Linux ARM64 #2293
  closes add aarch64 to cibuildwheel for linux #2178
  closes wheels: add arm64 builds #2224

Description

This PR continues work from #2103 (I've never removed that branch and rebased on master once in a while so that's the same branch in this PR).

cherry-picked commits from #2126 for missing test dependency in manylinux.

main differences between this PR and #2224:

• This PR does not build ppc64le & s390x.

• This PR does not build musllinux wheels.

• This PR adds tests for architectures running QEMU:
  While the tests needed some modifications in order to pass, IMHO, it's best to check than what can be tested does work as intended.
  Running with QEMU however hides some bugs (thinking about /proc/cpuinfo and cpu related stuff here) that actually do exist or comes with its own bugs (s390x and /proc endianness issue is most likely a qemu issue).
  aarch64 will most likely be able to run natively by the end of the year.

• This PR does not add Windows ARM64 wheel
  It could be added without any tests but I'd rather see Microsoft add Windows ARM64 runners to GitHub if they want to get some traction for their new platform (hopefully by the end of the year).

• This PR does not add macOS universal2 wheel
  My personal opinion is that it's not useful as long as you provide both x86_64 & arm64 wheels (which is the case).
  They do take more space for the end user, pip will always choose the most specific one so publishing 3 will change nothing for almost all users. Remain people packaging universal2 python apps, and, IMHO, this should be handled by the tools creating such apps.

• This PR feels simpler regarding the change of workflow

Regarding tests that are failing on real arm64 hardware but hidden by QEMU testing:

• Using docker on macOS M1:

======================================================================
FAIL: psutil.tests.test_system.TestCpuAPIs.test_cpu_freq
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_system.py", line 609, in test_cpu_freq
    assert ls, ls
AssertionError: []

======================================================================
FAIL: psutil.tests.test_linux.TestSystemCPUCountCores.test_method_2
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_linux.py", line 820, in test_method_2
    self.assertEqual(meth_1, meth_2)
AssertionError: 10 != None

======================================================================
FAIL: psutil.tests.test_linux.TestSystemCPUFrequency.test_emulate_use_second_file
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_linux.py", line 845, in test_emulate_use_second_file
    assert psutil.cpu_freq()
AssertionError

======================================================================
FAIL: psutil.tests.test_linux.TestSystemNetIfAddrs.test_ips
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_linux.py", line 1020, in test_ips
    self.assertEqual(addr.address, get_mac_address(name))
AssertionError: '00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00' != '00:00:00:00:00:00'
- 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
+ 00:00:00:00:00:00

• Using CirrusCI (Neoverse-N1):

======================================================================
FAIL: psutil.tests.test_system.TestCpuAPIs.test_cpu_freq
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_system.py", line 609, in test_cpu_freq
    assert ls, ls
AssertionError: []
======================================================================
FAIL: psutil.tests.test_system.TestSensorsAPIs.test_sensors_temperatures
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_system.py", line 942, in test_sensors_temperatures
    self.assertGreaterEqual(entry.high, 0)
AssertionError: -273.15 not greater than or equal to 0
======================================================================
FAIL: psutil.tests.test_linux.TestSystemCPUCountCores.test_method_2
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_linux.py", line 820, in test_method_2
    self.assertEqual(meth_1, meth_2)
AssertionError: 4 != None
======================================================================
FAIL: psutil.tests.test_linux.TestSystemCPUFrequency.test_emulate_use_second_file
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/project/psutil/tests/test_linux.py", line 845, in test_emulate_use_second_file
    assert psutil.cpu_freq()
AssertionError

[giampaolo]

Thank you Matthieu.

I added other platforms as well (ppc64le & s390x) [...] and musllinux

Please, let's just add aarch64 wheels for now (the ones with the highest demand). There's too many skipIf(...) which seems are due to emulation. Just so you know: my main concern here is keeping things as simple and maintainable as possible. Above all, we shouldn't introduce new random test failures (we already have a lot), and producing wheels / releases should be a smooth process.

[mayeut]

Please, let's just add aarch64 wheels for now (the ones with the highest demand)

I removed ppc64le & s390x (there's one demand from IBM #2210 for ppc64le).

I kept musllinux which are I think the second highest demand with 1 related issue & 3 opened PR.
If you don't want to keep musllinux, just drop the commit 445b35d.
Please note that 6f55304 is related to the addition of musllinux which comes with an up-to-date version of free/vmstat. Even if not building musllinux, this one will eventually come back as an issue once glibc distros get updated.

There's too many skipIf(...) which seems are due to emulation. Just so you know: my main concern here is keeping things as simple and maintainable as possible.

There are now less with s390x being dropped. I however understand your concern. Given GHA will most likely provide a native aarch64 before the end of the year, the QEMU test commit might be dropped completely 6aa80d3.

Above all, we shouldn't introduce new random test failures (we already have a lot), and producing wheels / releases should be a smooth process.

Agreed, that's why I disabled test_memleaks on those architectures. It was the only flaky one as far as I saw (and memleaks should mostly be independent of the processor architecture and tested otherwise).

Please feel free to force-push, drop commits, ... as you see fit. I have backups / alternatives to this work in my fork.

[giampaolo]

If you don't want to keep musllinux, just drop the commit 445b35d.

It seems I cannot modify your PR (I can push but can't see the changes reflected in here).
Can you please remove musl wheels and update the README? The rest looks good.
Also one minor thing. Is there a reason why it's called QEMU_USER instead of QEMU?

[mayeut]

It seems I cannot modify your PR (I can push but can't see the changes reflected in here).

weird...
anyway

Also one minor thing. Is there a reason why it's called QEMU_USER instead of QEMU?

QEMU provides full-system emulation or user-mode emulation. Here, we only use user-mode emulation.

[mayeut]

update the README?

I guess you meant HISTORY ?

[giampaolo]

Yes, sorry. 😄