Build wheels for additional architectures

[jkterry1]

CUDA supports ARM, POWER8, POWER9, and x86_64. Ray only builds wheels for x86_64. Tons of supercomputers with GPUs use POWER, and Jetson boards (popularly used in robots) use ARM. Both of those are things that need to be supported.

Last I checked, building ray from source on POWER8 worked just fine.

This was first mentioned in issue #2358, but that was recently closed without explanation.

[simon-mo]

We currently don't have POWER architecture in our CI to compile our binary and send it out for distribution. It also adds additional overhead for release testing. I can recommend the following:

• If possible, can you help us adding the wheels building steps to produce the POWER9 wheels? The relevant code is at

  ray/.travis.yml

  Lines 130 to 170 in e29f2ef

  	# Build Linux wheels.
  	- os: linux
  	env: LINUX_WHEELS=1 PYTHONWARNINGS=ignore RAY_INSTALL_JAVA=1
  	install:
  	- eval `python $TRAVIS_BUILD_DIR/ci/travis/determine_tests_to_run.py`
  	- if [ $RAY_CI_LINUX_WHEELS_AFFECTED != "1" ]; then exit; fi
  	- ./ci/suppress_output ./ci/travis/install-dependencies.sh
  	# Mount bazel cache dir to the docker container.
  	# For the linux wheel build, we use a shared cache between all
  	# wheels, but not between different travis runs, because that
  	# caused timeouts in the past. See the "cache: false" line below.
  	- export MOUNT_BAZEL_CACHE="-v $HOME/ray-bazel-cache:/root/ray-bazel-cache -e TRAVIS=true -e TRAVIS_PULL_REQUEST=$TRAVIS_PULL_REQUEST -e encrypted_1c30b31fe1ee_key=$encrypted_1c30b31fe1ee_key -e encrypted_1c30b31fe1ee_iv=$encrypted_1c30b31fe1ee_iv"
  	# This command should be kept in sync with ray/python/README-building-wheels.md,
  	# except the `$MOUNT_BAZEL_CACHE` part.
  	- ./ci/suppress_output docker run -e TRAVIS_COMMIT=$TRAVIS_COMMIT --rm -w /ray -v `pwd`:/ray $MOUNT_BAZEL_CACHE -ti rayproject/arrow_linux_x86_64_base:ARROW-5631 /ray/python/build-wheel-manylinux1.sh
  	script:
  	- if [ $RAY_CI_LINUX_WHEELS_AFFECTED != "1" ]; then exit; fi
  	- ./ci/travis/test-wheels.sh
  	cache: false
  	# Build MacOS wheels.
  	- os: osx
  	osx_image: xcode7
  	env: MAC_WHEELS=1 PYTHONWARNINGS=ignore RAY_INSTALL_JAVA=1
  	install:
  	- eval `python $TRAVIS_BUILD_DIR/ci/travis/determine_tests_to_run.py`
  	- if [ $RAY_CI_MACOS_WHEELS_AFFECTED != "1" ]; then exit; fi
  	- ./ci/suppress_output ./ci/travis/install-dependencies.sh
  	# This command should be kept in sync with ray/python/README-building-wheels.md.
  	- ./ci/suppress_output ./python/build-wheel-macos.sh
  	script:
  	- if [ $RAY_CI_MACOS_WHEELS_AFFECTED != "1" ]; then exit; fi
  	- ./ci/travis/test-wheels.sh

  We might have to use clang cross-compile or running power on qemu.
• the wheels will have limited support, for example, no fully covered testing. similar to tier 2-3 in rust support guideline https://forge.rust-lang.org/release/platform-support.html#tier-3

[ddelange]

Adding aarch64 to the mix (ARM64), would make #12128 a duplicate

Here a tool from the python packaging authority to build multi arch wheels: https://github.com/pypa/cibuildwheel

[mattip]

Also see #4309 (for ppc64le) and #13780 (for aarch64/macos arm64).

[mattip]

We should leave only one of the ARM issue #7476 (this one) #13780 and #12128 open.

[richardliaw]

I'm going to close this as duplicate of #13780