Wheel for Raspberry PI (ARM processor)

[FormerLurker]

Describe your feature request

Could a wheel be created for installing via pip on a raspberry pi? I tested ray on windows, and it worked amazingly well, and would LOVE to get it working on a raspberry pi too. This would make it much easier to distribute my cross platform open-source application for raspberry pi users.

Thank you for your amazing work!

[foice]

I am surprised this has not come up earlier. How can we install on raspberry? pip seems to be not an option ... or am I missing some tricks of pip for these cases?

[FormerLurker]

It will need to be manually compiled for now. I'm sure it's come up before, but it takes time and effort to build a new wheel, so it's a question of demand I think.

I haven't had the time to manually build this and compile instructions yet, but I plan to. I got slammed with issues (holiday weekends are always like this) and have been stuck on those. I will update this issue with instructions ASAP.

[foice]

Thanks for the effort!

…

On Sun, Nov 29, 2020 at 4:03 PM FormerLurker ***@***.***> wrote: It will need to be manually compiled for now. I'm sure it's come up before, but it takes time and effort to build a new wheel, so it's a question of demand I think. I haven't had the time to manually build this and compile instructions yet, but I plan to. I got slammed with issues (holiday weekends are always like this) and have been stuck on those. I will update this issue with instructions ASAP. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12128 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHRUNONPTVMB6KFFIA4OY3SSJPFTANCNFSM4T2OBIBQ> .

[PidgeyBE]

Hey
I've made a few builds last week (see #12835) for ray 1.0.1:
All python3.6:

• aarch64: https://github.com/ray-project/ray/files/5729517/ray-1.0.1-aarch64.zip
• arm7: https://github.com/ray-project/ray/files/5703429/ray_armv7l_32bit.zip

[PidgeyBE]

• ray 1.1.0 aarch64: ray-1.1.0-cp36-aarch64.zip

[foice]

Thnaks! Does it mean it is on pip, or we are supposed to test it? In the latter case how to install it on my python setup?

…

On Dec 24, 2020, at 17:39, PidgeyBE ***@***.***> wrote: Hey I've made a few builds last week (see #12835) for ray 1.0.1: aarch64: https://github.com/ray-project/ray/files/5729517/ray-1.0.1-aarch64.zip arm7: https://github.com/ray-project/ray/files/5703429/ray_armv7l_32bit.zip — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[PidgeyBE]

@foice It is not published in the Pypi repository, I don't have access to that, I've just built the wheels myself. So you should download the zip, unzip it, and install the wheels via

pip install py_spy_xxxxxxxxx.whl
pip install ray-xxxxxxxx.whl

[deetungsten]

@PidgeyBE Would you happen to have a wheel for aarch64 + Python 3.8? I'm trying to make this work natively on a mac M1 but I haven't been able to build ray correctly.

[PidgeyBE]

@deetungsten I've just compiled them for you, Merry Christmas!
ray-1.1.0-cp38-aarch64.zip
Let me know if it works! I've tested it locally in an emulator and it works...

[FormerLurker]

Hey, thank you @PidgeyBE!!! I have been buried under a mountain of end of the year work. Today's the earliest I had a chance to look at this, so THANK YOU for saving me time and effort :)

[deetungsten]

@PidgeyBE WOW thank you so much! Ahah best Christmas gift ever! I will test it later tonight after all the Christmas activities. Merry Christmas!

[Johannes-Lange]

Is it somehow possible to compile ray for arch64 + Python 3.9 for a M1 Mac? I have to decide if I go for Python 3.8 or 3.9, ray is the only missing piece for 3.9.

[deetungsten]

@Johannes-Lange I feel your pain. I spent most of last night trying to compile Pytorch for 3.8 (there's only a 3.9 beta wheel) and I couldn't get it to work because of clang issues. I thought about using tensorflow backend instead but it wouldn't work for some reason. The Apple Tensorflow version with the M1 GPU acceleration is 3.8 only. The M1 have been a viscous cycle of trying to make everything work together with the right Python version.

[Johannes-Lange]

@deetungsten pytorch runs quite nicely in my 3.9 environment, using the crosscompiled numpy from conda-forge. So I don’t want to change that...
The problem I am facing with ray is bazel for the building, all other dependencies do work.

[PidgeyBE]

@Johannes-Lange I'll try to build ray for python 3.9 and aarch64.
I can't give any guarantees though, because ray doesn't even have Python 3.9 support... Anyhow, I'll run our build scripts on this branch #12613 and see what comes out...

[PidgeyBE]

@Johannes-Lange This version (ray 1.2.0dev0!) seems to work in my qemu/docker emulation env for aarch64/python3.9:
ray-1.2.0.dev0-aarch64-cp39.zip
(the zip contains a py_spy wheel as well)

[deetungsten]

Both 3.8 and 3.9 wheels gets this issue ERROR: py_spy-0.3.3-cp38-cp38-linux_aarch64.whl is not a supported wheel on this platform. :(

[Johannes-Lange]

@deetungsten you have to change the name to "ray-1.2.0.dev0-cp39-cp39-macosx_11_0_arm64.whl" (maybe 11_1, has to be your os version), same for py-spy.

[deetungsten]

Edit: Works now! I had to run pip3 install --no-dependencies --force ray-1.2.0.dev0-cp39-cp39-macosx_11_0_arm64.whl for 3.9 but decided to use the 3.8 for tensorflow reasons (I used the same renaming trick). I think tensorflow + Apple/CoreML acceleration works with this but will need to try a couple tests to know for sure.

@Johannes-Lange @PidgeyBE IT WORKS!!! Thank you both so much! Interestingly, I never knew wheel names mattered. Is this only special to mac or is this the same for Linux too? Also, even though I have 11.1 as my OS version, 11_0 is what worked.

[Johannes-Lange]

@deetungsten afaik the name matters always. The installation worked, but I get an error when calling ray.init() because some files are an unknown format. Does it work for you? If yes, what is your configuration?

[deetungsten]

@Johannes-Lange I have issues too loading ray but with a different issue OSError: dlopen(/Users/USERNNAME/miniforge3/envs/ENVNNAME/lib/python3.8/site-packages/ray/_raylet.so, 10): no suitable image found. I'm running python 3.8 with conda, tensorflow + pytorch (nightly build).

[Johannes-Lange]

@deetungsten ok, I got the same error + some unknown file format stuff. So far it looks like a common mac error, maybe caused because the wheel is built for Linux. I will keep trying to build bazel for arm mac, then I could also build the ray wheel. If that works I’ll let you know, but I can’t promise anything.

[umbernhard]

Thanks @PidgeyBE for providing these wheels! I'm currently working on using cluster config, but I'm having issues because the docker images ray up relies on are all x86_64. Has anyone managed to munge the dockerfiles to work with aarch64 yet?

[umbernhard]

Actually, I just built my own. I couldn't quite get the ray-ml equivalent to build, but for a RPi cluster I'm not sure that all of it is needed. I also ported from Conda to Pip where possible, which required some minor versioning fixes but nothing too serious.

aarch64 compatible docker images:

• base-deps: umbernhard/base-deps-arm
• ray-deps: umbernhard/ray-deps-arm
• ray: umbernhard/ray-arm

Github repo with dockerfiles: https://github.com/umbernhard/ray-docker-arm

[PeterPirog]

@PidgeyBE Thank you for arm7 repo: https://github.com/ray-project/ray/files/5703429/ray_armv7l_32bit.zip

I would like to ask where can I find tutorial how to build whl files for raspberry. I'm beginner in this topic and I would like learn it from basics. If some one can recommand soe tutorial or book I will be gratefull.

[PidgeyBE]

• ray 1.1.0 aarch64: ray-1.1.0-cp36-aarch64.zip

Can provide aarch64 + Python 3.7 +ray 1.3.0? @PidgeyBE

ray-1.3.0-cp37-cp37m-linux_aarch64.zip
*rename .zip to .whl
Build with original instructions found in this thread, so I'm not sure what is going wrong in your case....

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[ddelange]

Any update on adding manylinux2014_aarch64/manylinux2014_armv7l wheels to Rays official CI/CD?

Manually building and passing around links to wheels doesn't seem the right way to go...

Here is an official tool that integrates well with Github Actions: https://github.com/pypa/cibuildwheel

Similar discussion ref tensorflow/tensorflow#52973

[deifactor]

FYI my team is currently getting bitten by this very hard, since some of our developers (including me) work on M1 macs but have to run some code inside a Linux image, and the performance for an x86_64 image on an M1 host is... very bad. Having to add additional steps to our process just for one package is extremely not ideal.

[ddelange]

For M1 wheel from laptop see #19925: there are wheels for arm macs https://pypi.org/project/ray/1.13.0/#files

that wont cover other arm boxes like the armv7(32)/armv8(64) raspberries though.

Probably not possible to use limited ABI for ray: otherwise one manylinux2014_aarch64 and one macosx_10_9_universal2 wheel using abi3 would be sufficient for all newer and future (3.6+) python versions ref giampaolo/psutil#2103

[autolyticus]

Glad to see that there's official progress on this issue! I've been eagerly following the updates on this issue since December 2020 when I got my M1 MacBook.

I would love to get the latest versions of ray running on my Raspberry Pi k3s cluster so I am trying to compile the manylinux2014_aarch64 wheels

I spent a few hours trying to build using @floucks's Dockerfile process on a virtualised aarch64 Linux machine on my M1 (using UTM) but ended up running into unknown build errors involving some cpp references to std::filesystem. If someone might be able to shed some light into what that's about, I can retry it and get the proper error messages. (I am not sure if I should be raising a new issue or not regarding this)

[ddelange]

Were you attempting to run the Docker image intended for arm mac on an aarch64 vm?

Mac uses the clang compiler by default. I wouldn't be surprised if this is a cpp compiler flavour setup (eg clang vs gcc). Ran into that before when compiling projects from source: https://github.com/ddelange/pyicu/blob/2804b7c34c216d7ef63cefea96f54b54757f9ec3/README.md?plain=1#L31-L36

[Zephyr69]

@PeterPirog master/ray-2.0.0.dev builds fail on armv7 at our side.. @arunbv123 I've made a python3.8 aarch64 build which should work on Nvidia Jetsons: ray-2.0.0.dev0-cp38-cp38-linux_aarch64.zip

Sir! What procedure did you use (or should we use now) to build for jetsons (with python3.8)?

And although this 2.0.0 build runs fine with cpu, it doesn't see the jetson's gpu. Is this normal?

[mattip]

I think we should close this. There are various other issues about linux aarch64 and macos arm64. This issue is about arm7, which is not supported on PyPI so even if we could create a wheel we would have to host it on a non-standard platform like piwheels.

[autolyticus]

@mattip I respectfully disagree.

• Until we have first class ARM support for Ray from the toolchain and CI/CD side, I would recommend against closing this issue. I believe that until more time and energy is invested in this, this is going to remain open for a while because it's a subtle thing that needs to be solved at the root. (How to solve it at the root is pretty much documented in the history of this issue). This issue has been open for almost 2 years for a reason.

• This issue isn't necessarily about just armv7 (Most of the discussion centers around aarch64).

• And there's also a lot of history and tips(wrt bazel, how to build arm wheels in docker etc) here that is very relevant for anyone who is working to ensure proper ports of Ray to any new platform).

• Last but not the least, this issue is very highly ranked in Google search results for Ray Python ARM, so most users who are looking for anything related to "Ray and arm" will see this issue first. If they see it as closed, they would assume that Ray is supposed to work for RPi and will be confused when it doesn't.

• The other specific sub-issues could potentially exist in isolation and as the specific platform support arrives step by step, we can close them as well, but this is the root issue.

• Lots of other issues like this one have been closed citing this as the root issue as this was the original and that was the duplicate.

[mattip]

If this issue is not about Raspberry PI then the title is misleading and should be changed. In general spreading the discussion about alternative architectures over (at least three) open issues is not going to move the needle on this: we need to decide it is a priority and move it forward. As I commented on #4309, the current blockers for providing wheels/conda packages have nothing to do with ray itself, rather on downstream dependencies (specifically gym, which depends on pygame and ale-py) providing pre-built packages.

If they see it as closed, they would assume that Ray is supposed to work for RPi

They would see it is closed as "won't fix" and realize pre-built ray on Raspberry PI is not going to happen.

[ddelange]

This issue is about arm7, which is not supported on PyPI

pre-built ray on Raspberry PI is not going to happen

fwiw, PyPI is architecture-agnostic, but only few projects choose to publish armv7 wheels (example).

moreover, RPi4 has a 64 bit chipset and mine happily runs aarch64 wheels, making this issue relevant regardless of the 32/64 bit argument.

a better title could make sense, but as rightly noted this is a landing issue for arm and should not be closed imo.

[mattip]

only few projects choose to publish armv7 wheels

That is because the armv71 designation is considered a mistake and only supports the Raspian (now called Raspberry Pi OS) distribution, I think for Raspberry Pi 4 only. As you stated, it is not very popular. Not even NumPy supports it on PyPI, so I think getting ray to distribute for that platform tag will be quite a bit of work getting all the dependent packages into PyPI.

[wenyangzz]

@PidgeyBE Hi, is it possible to provide me a ray-1.10.0. whl for python 3.9. I lack understanding of how to build the wheel. Thank a million!

[ddelange]

to create a wheel from source, specific to your machine and python version, you can try

cd ./python
pip wheel --no-deps .

• ray docs and also the section above that: "Preparing to build Ray on Linux"
• pip docs

[ddelange]

only few projects choose to publish armv7 wheels

That is because the armv71 designation is considered a mistake and only supports the Raspian (now called Raspberry Pi OS) distribution, I think for Raspberry Pi 4 only. As you stated, it is not very popular. Not even NumPy supports it on PyPI, so I think getting ray to distribute for that platform tag will be quite a bit of work getting all the dependent packages into PyPI.

@krfricke @mattip can this issue be closed as of #13780?

[pcmoritz]

I believe so, if it is not resolved, a new issue should be opened or somebody should comment on this issue so we can reopen it (I don't think anybody has tested the new wheels on Raspberry PI yet, it would be great to get feedback from somebody who owns a Raspberry PI).

[autolyticus]

I've just verified that pip install ray[default] works as expected and installs ray-2.2.0-cp310-cp310-manylinux2014_aarch64.whl as expected (on a Raspberry Pi 4).

Thanks a lot to everyone for the effort in providing support for this platform!