[Linux][aarch64] Wheel support for aarch64 Linux

[franklsf95]

Summary

• OS: Linux aarch64
• Type: wheels

Description

This is the same feature request as #1782. I would really love to see wheel support for aarch64, as AWS's Graviton ARM-based VMs are getting more popular. Making a wheel for aarch64 would help so many projects that depend on psutil.

[RobinFrcd]

Hi,
I'm currently stuck with the 5.8.0 version on conda. Any chance to add the ARM build to the pypi repo ? 🙏

[johnthagen]

Linux ARM64 wheels would make it easier for ARM Mac users to test out Docker images that depend on psutil.

[johnthagen]

cibuildwheel supports building manylinux musllinux aarch64 wheels.

[altendky]

Well, my experiment over at #2178 didn't go well with various failures of which at least some are triggered by my use of QEMU to get these built on the Intel GHA runners.

As for actual test failures, here's a job where I built and tested the wheels natively on a self-hosted ARM runner.

https://github.com/Chia-Network/build-wheels/actions/runs/3568381151/jobs/5997186087

  ======================================================================
  FAIL: psutil.tests.test_linux.TestSystemCPUCountCores.test_method_2
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/psutil/tests/test_linux.py", line 749, in test_method_2
      self.assertEqual(meth_1, meth_2)
  AssertionError: 4 != None
  
  ======================================================================
  FAIL: psutil.tests.test_linux.TestSystemCPUFrequency.test_emulate_use_cpuinfo
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/psutil/tests/test_linux.py", line 791, in test_emulate_use_cpuinfo
      assert ret
  AssertionError
  
  ======================================================================
  FAIL: psutil.tests.test_linux.TestSystemCPUFrequency.test_emulate_use_second_file
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/psutil/tests/test_linux.py", line 774, in test_emulate_use_second_file
      assert psutil.cpu_freq()
  AssertionError
  
  ======================================================================
  FAIL: psutil.tests.test_linux.TestSystemCPUStats.test_ctx_switches
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/psutil/tests/test_linux.py", line 907, in test_ctx_switches
      self.assertAlmostEqual(vmstat_value, psutil_value, delta=500)
  AssertionError: 379679541 != 8969614139 within 500 delta
  
  ======================================================================
  FAIL: psutil.tests.test_linux.TestSystemCPUStats.test_interrupts
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/psutil/tests/test_linux.py", line 912, in test_interrupts
      self.assertAlmostEqual(vmstat_value, psutil_value, delta=500)
  AssertionError: 405313834 != 4700281131 within 500 delta
  
  ======================================================================
  FAIL: psutil.tests.test_system.TestCpuAPIs.test_cpu_freq
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/project/psutil/tests/test_system.py", line 534, in test_cpu_freq
      assert ls, ls
  AssertionError: []

At least a few of these are tied to ARM doing /proc/cpuinfo differently. I see the patch that was considered for one of these issues over in #1359 (comment).

@giampaolo, what do you see as a path forward here? I would be happy to try to help, though with an existing patch that isn't of interest I hesitate to create another without discussing concerns first. If it's too much to get the tests all fixed now, I would personally see even just skipping those tests on ARM as some sort of improvement. Then I could build the wheel for my uses without dealing with skipping the tests in my own fork. Though if going the QEMU/Docker route for this official repo we would have to skip a lot more tests or just not test the ARM wheel at all. :|

Anyways, I'm interested in making this situation better. Let me know what opportunities you see. Thanks for your thoughts.

Cheers,
-kyle

[Jackenmen]

One possibility could be to use another CI that has native arm64 runners (such as the free Cirrus CI) to build ARM wheels there. I'm not sure how the maintainer of this project feels about adding another CI to this project though, there are already 2 CI setups here (Appveyor and GitHub Actions), it seems like.

It would also make it a bit more complicated to publish a release to PyPI as the maintainer would have to download wheels from 2 places (or perhaps do some more advanced setup which would make it so they can download all wheels from one place once both CI runs are finished). We could also just move the whole setup to Cirrus CI instead.

[miki725]

buildjet has really good ARM integration directly in github actions. after installing their app in your github account, you simply specify a buildjet runner in runs-on clause however it seems they do not offer a free plan for open source projects https://buildjet.com/for-github-actions/pricing. Ive send them a request if they would be willing to provide some free minutes to this project (with the link to this issue).

would be amazing if Github would add that feature so you can simply run a job on ubuntu-latest-arm or something like that however at this time doesnt seem like thats a supported option outside of self-hosted runner. hopefully at some point soon this will become possible 🤞

[CAM-Gerlach]

FWIW, GitHub now offers free standard GHA runners for macOS arm64, and to my understanding there are active plans to offer Linux and Windows arm64 support within the next ≈year.

[altendky]

Good thing I'm now dealing with RISCV support for our project instead. :]

[altendky]

🎉