Skip to content

Commit

Permalink
Merge pull request #592 from Farama-Foundation/macos-workflows-update
Browse files Browse the repository at this point in the history
Update workflows to support building macOS Apple Silicon (arm64) wheels
  • Loading branch information
mwydmuch authored Aug 20, 2024
2 parents 04a0d6a + 74b2e53 commit dc444f4
Show file tree
Hide file tree
Showing 10 changed files with 234 additions and 136 deletions.
183 changes: 183 additions & 0 deletions .github/workflows/build-and-test-wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
name: Build and test Python wheels and make PyPI release

on:
workflow_dispatch:
push:
paths:
- '.github/workflows/build-and-test-wheels.yml'
- 'include/**'
- 'scripts/**'
- 'src/**'
- 'CMakeLists.txt'
- 'setup.py'
- 'pyproject.toml'
branches: [master]
release:
types: [published]

jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-22.04, macos-12, macos-14]
fail-fast: false

steps:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Report OS
run: |
echo ${{ matrix.os }}
echo ${{ runner.os }}
uname -p
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v2
with:
platforms: all

- name: Build manylinux wheels
if: matrix.os == 'ubuntu-22.04'
uses: pypa/[email protected]
env:
# Configure cibuildwheel to build native archs, and some emulated ones
CIBW_ARCHS_LINUX: x86_64 aarch64
CIBW_BUILD_VERBOSITY: 1

- name: Build macOS Intel wheels
if: matrix.os == 'macos-12'
uses: pypa/[email protected]
env:
CIBW_ARCHS_MACOS: x86_64
CIBW_ENVIRONMENT_MACOS: VIZDOOM_MACOS_ARCH=x86_64 HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 MACOSX_DEPLOYMENT_TARGET=12.0
CIBW_BUILD_VERBOSITY: 1

- name: Build macOS Apple Silicon wheels
if: matrix.os == 'macos-14'
uses: pypa/[email protected]
env:
CIBW_ARCHS_MACOS: arm64
CIBW_ENVIRONMENT_MACOS: VIZDOOM_MACOS_ARCH=arm64 HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 MACOSX_DEPLOYMENT_TARGET=14.0
CIBW_BUILD_VERBOSITY: 1

- name: Report built wheels
run: |
ls -l ./wheelhouse/*.whl
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
path: ./wheelhouse/*.whl

test_wheels:
name: Test wheels on ${{ matrix.os }}
needs: [build_wheels]
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-22.04, ubuntu-24.04, macos-12, macos-13, macos-14]
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
fail-fast: false

steps:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Set up Python ${{ matrix.python-version }} environment
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Download all dists
uses: actions/download-artifact@v3
with:
# Unpacks default artifact into dist/
# If `name: artifact` is omitted, the action will create extra parent dir
name: artifact
path: dist

- name: Report dist directory
run: ls -l dist

- name: Report environment
run: |
echo ${{ matrix.os }}
echo ${{ runner.os }}
uname -p
python -c "import sys; print(sys.version)"
- name: Install macOS Intel wheel on ${{ matrix.os }}
if: matrix.os == 'macos-12' || matrix.os == 'macos-13'
run: |
export PYTHON_VERSION=$(python -c "import sys; print(f'{sys.version_info.major}{sys.version_info.minor}')")
export WHEEL=$(ls dist/vizdoom*cp${PYTHON_VERSION}*macosx*x86_64.whl)
python -m pip install ${WHEEL}[test]
- name: Install macOS Apple Silicon wheel on ${{ matrix.os }}
if: matrix.os == 'macos-14'
run: |
export PYTHON_VERSION=$(python -c "import sys; print(f'{sys.version_info.major}{sys.version_info.minor}')")
export WHEEL=$(ls dist/vizdoom*cp${PYTHON_VERSION}*macosx*arm64.whl)
python -m pip install ${WHEEL}[test]
- name: Install manylinux wheel on ${{ matrix.os }}
if: matrix.os == 'ubuntu-22.04' || matrix.os == 'ubuntu-24.04'
run: |
export PYTHON_VERSION=$(python -c "import sys; print(f'{sys.version_info.major}{sys.version_info.minor}')")
export WHEEL=$(ls dist/vizdoom*cp${PYTHON_VERSION}*manylinux*x86_64.whl)
python -m pip install ${WHEEL}[test]
- name: Import check
run: python -c "import vizdoom"

- name: Run tests
# Skip tests on macOS with Apple Silicon, because they are slow (TODO: investigate)
if: matrix.os != 'macos-14'
run: pytest tests

build_sdist:
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Build sdist
run: pipx run build --sdist

- name: Upload sdist
uses: actions/upload-artifact@v3
with:
path: dist/*.tar.gz

upload_pypi:
name: Upload to PyPI
needs: [build_wheels, build_sdist, test_wheels]
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write
if: github.event_name == 'release' && github.event.action == 'published'
steps:
- name: Download all dists
uses: actions/download-artifact@v3
with:
# Unpacks default artifact into dist/
# If `name: artifact` is omitted, the action will create extra parent dir
name: artifact
path: dist

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}
# To test:
# with:
# repository_url: https://test.pypi.org/legacy/
2 changes: 1 addition & 1 deletion .github/workflows/build-and-test-windows-wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
branches: [master]
pull_request:
paths:
- '.github/workflows/**'
- '.github/workflows/build-and-test-windows-wheels.yml'
- 'include/**'
- 'scripts/**'
- 'src/**'
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
branches: [master]
pull_request:
paths:
- '.github/workflows/**'
- '.github/workflows/build-and-test.yml'
- 'include/**'
- 'scripts/**'
- 'src/**'
Expand Down Expand Up @@ -76,4 +76,6 @@ jobs:
run: python -c "import vizdoom"

- name: Run tests
# Skip tests on macOS with Apple Silicon, because they are slow (TODO: investigate)
if: matrix.os != 'macos-14'
run: pytest tests
115 changes: 0 additions & 115 deletions .github/workflows/build-wheels.yml

This file was deleted.

15 changes: 9 additions & 6 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,15 @@ if(APPLE)
set(CMAKE_MACOSX_RPATH ON)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden -fvisibility-inlines-hidden")

if(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64")
set(CMAKE_OSX_ARCHITECTURES "arm64" CACHE INTERNAL "" FORCE)
message(STATUS "Apple Silicon detected, building for arm64")
else()
set(CMAKE_OSX_ARCHITECTURES "x86_64" CACHE INTERNAL "" FORCE)
message(STATUS "Intel CPU detected, building for x86_64")
if(CMAKE_APPLE_SILICON_PROCESSOR MATCHES "")
message(STATUS "CMAKE_APPLE_SILICON_PROCESSOR not set")
if(CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64")
set(CMAKE_OSX_ARCHITECTURES "arm64" CACHE INTERNAL "" FORCE)
message(STATUS "Apple Silicon detected, building for arm64")
else()
set(CMAKE_OSX_ARCHITECTURES "x86_64" CACHE INTERNAL "" FORCE)
message(STATUS "Intel CPU detected, building for x86_64")
endif()
endif()
endif(APPLE)

Expand Down
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ or

## Python quick start

#### At the moment ViZDoom does not work with numpy 2.0. Using ViZDoom with numpy 2.0 results in a silent bug - all the state buffers containing all data. At the moment, please use ViZDoom with numpy 1.26.
#### Versions 1.2.3 and below does not work correctly with numpy 2.0+. Please upgrade ViZDoom to the newest version or downgrade numpy to the last 1.21+ version.

### Linux
To install the latest release of ViZDoom, just run:
Expand All @@ -85,12 +85,18 @@ ViZDoom requires a C++11 compiler, CMake 3.12+, Boost 1.54+ SDL2, OpenAL (option


### macOS
To install the latest release of ViZDoom, just run (it may take a few minutes as it will build ViZDoom from source on M1/M2 chips):
To install the latest release of ViZDoom, just run:
```sh
brew install cmake boost sdl2 openal-soft
pip install vizdoom
```
Both Intel and Apple Silicon CPUs are supported.
Pre-build wheels are available for Intel macOS 12.0+ and Apple Silicon macOS 14.0+.

If Python wheel is not available for your platform (Python version <3.8, older macOS version), pip will try to install (build) ViZDoom from the source.
In this case, install the required dependencies using Homebrew:
```sh
brew install cmake boost sdl2
```
We recommend using at least macOS High Sierra 10.13+ with Python 3.8+.
On Apple Silicon (M1, M2, and M3), make sure you are using Python/Pip for Apple Silicon.

Expand All @@ -101,7 +107,7 @@ To install the latest release of ViZDoom, just run:
pip install vizdoom
```
At the moment, only x86-64 architecture is supported on Windows.
Wheels are available for Python 3.8+ on Windows.
Wheels are available for Python 3.9+ on Windows.

Please note that the Windows version is not as well-tested as Linux and macOS versions.
It can be used for development and testing but if you want to conduct serious (time and resource-extensive) experiments on Windows,
Expand All @@ -124,7 +130,7 @@ See [documentation](https://github.com/Farama-Foundation/ViZDoom/blob/master/doc
- [Python](https://github.com/Farama-Foundation/ViZDoom/blob/master/examples/python) (contain learning examples implemented in PyTorch, TensorFlow, and Theano)
- [C++](https://github.com/Farama-Foundation/ViZDoom/blob/master/examples/c%2B%2B)

Python examples are currently the richest, so we recommend looking at them, even if you plan to use C++.
Python examples are currently the richest, so we recommend looking at them, even if you plan to use C++.
The API is almost identical between the languages, with the only difference being that Python uses snake_case and C++ camelCase for methods and functions.


Expand Down
Loading

0 comments on commit dc444f4

Please sign in to comment.