Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manylinux wheels #552

Merged
merged 12 commits into from
Sep 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Build
name: Build and test

on:
push:
Expand All @@ -11,8 +11,7 @@ on:
- 'CMakeLists.txt'
- 'setup.py'
- 'pyproject.toml'
branches:
- master
branches: [master]
pull_request:
paths:
- '.github/workflows/**'
Expand All @@ -23,8 +22,7 @@ on:
- 'CMakeLists.txt'
- 'setup.py'
- 'pyproject.toml'
branches:
- master
branches: [master]

env:
HOMEBREW_NO_AUTO_UPDATE: 1
Expand All @@ -36,7 +34,6 @@ jobs:
matrix:
os: [ubuntu-20.04, ubuntu-22.04, macos-12]
python-version: ['3.8', '3.9', '3.10', '3.11']

runs-on: ${{ matrix.os }}

steps:
Expand All @@ -46,7 +43,7 @@ jobs:
if: runner.os == 'Linux'
run: |
sudo apt update
sudo apt install -y cmake libboost-all-dev libsdl2-dev libfreetype6-dev libgl1-mesa-dev libglu1-mesa-dev libpng-dev libjpeg-dev libbz2-dev libfluidsynth-dev libgme-dev libopenal-dev zlib1g-dev timidity tar nasm
sudo apt install -y cmake git libboost-all-dev libsdl2-dev libopenal-dev

- name: Apt report
if: runner.os == 'Linux'
Expand Down
91 changes: 91 additions & 0 deletions .github/workflows/build-wheels,yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Build Python wheels and make release

on:
workflow_dispatch:
pull_request:
push:
paths:
- '.github/workflows/**'
- 'include/**'
- 'scripts/**'
- 'src/**'
- 'CMakeLists.txt'
- 'setup.py'
- 'pyproject.toml'
branches: [master]
release:
types: [published]

jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-22.04, macos-11]

steps:
- uses: actions/checkout@v3

- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v2
with:
platforms: all

- name: Build wheels
uses: pypa/[email protected]
env:
# Configure cibuildwheel to build native archs, and some emulated ones
CIBW_ARCHS_LINUX: x86_64 aarch64
CIBW_ARCHS_MACOS: x86_64
CIBW_BUILD_VERBOSITY: 3
CIBW_REPAIR_WHEEL_COMMAND_LINUX: >
auditwheel show {wheel} && auditwheel repair -w {dest_dir} {wheel}

- name: Report built wheels
run: |
ls -l ./wheelhouse/*.whl

- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
path: ./wheelhouse/*.whl

build_sdist:
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Build sdist
run: pipx run build --sdist

- name: Upload sdist
uses: actions/upload-artifact@v3
with:
path: dist/*.tar.gz

upload_pypi:
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write
if: github.event_name == 'release' && github.event.action == 'published'
steps:
- name: Download all dists
uses: actions/download-artifact@v3
with:
# Unpacks default artifact into dist/
# If `name: artifact` is omitted, the action will create extra parent dir
name: artifact
path: dist

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pseudo-rnd-thoughts I need this secret to be setup to make this step work on release. Could you add it?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

# To test:
# with:
# repository_url: https://test.pypi.org/legacy/
3 changes: 2 additions & 1 deletion .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# https://pre-commit.com
# This GitHub Action assumes that the repo contains a valid .pre-commit-config.yaml file.
name: Pre-commit
name: Pre-commit checks
on:
pull_request:
push:
Expand All @@ -11,6 +11,7 @@ permissions:

jobs:
pre-commit:
name: Pre-commit checks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand Down
24 changes: 4 additions & 20 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
ViZDoomVersion.h
viz_version.h
Makefile
!docs/Makefile
ImportExecutables.cmake
**/link-make
cmake-build-*
src/boost_*

*~
*.o
Expand Down Expand Up @@ -47,20 +47,13 @@ cmake-build-*
examples/python/*.cfg
examples/python/*.ini
dist
build
venv
.eggs
vizdoom.egg-info
wheelhouse

# Lua
examples/lua/*.cfg
examples/lua/*.ini
examples/lua/vizdoom
*.rock

# Java
*.jar
*.class
# Tests
test_dockerfiles

# Copied from the original ZDoom repository (and modified)
*.cbp
Expand Down Expand Up @@ -119,14 +112,5 @@ src/vizdoom/zlib/x64/
# CLion & PyCharm
**/.idea

# Visual Studio Code
.vscode

# MacOS
.DS_Store

# Dockerfiles
tmp_dockerfiles

# Documentation
docs_html
13 changes: 9 additions & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.4)
cmake_minimum_required(VERSION 3.12)

# Project + versions
#-----------------------------------------------------------------------------------------------------------------------
Expand All @@ -12,7 +12,7 @@ set(ViZDoom_VERSION_ID ${ViZDoom_VERSION_MAJOR}${ViZDoom_VERSION_MINOR}${ViZDoom
#-----------------------------------------------------------------------------------------------------------------------

option(BUILD_PYTHON "Build ViZDoom Python (3) binding/module" ON)
option(BUILD_ENGINE "Build ViZDoom Engine" ON)
option(BUILD_ENGINE "Build ViZDoom Engine (required to build Python package)" ON)


# CMake options
Expand Down Expand Up @@ -76,6 +76,9 @@ endif (APPLE)

if (CMAKE_COMPILER_IS_GNUCXX)
add_definitions("-fPIC")
# Explicitly silence some warnings
add_definitions("-Wno-implicit-fallthrough -Wno-shift-negative-value -Wno-stringop-truncation")
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-class-memaccess -Wno-deprecated-copy") # C++ only
endif ()

if (MSVC)
Expand All @@ -98,7 +101,7 @@ set(VIZDOOM_LIB_SRC_DIR ${VIZDOOM_SRC_DIR}/lib)
set(VIZDOOM_LIB_INCLUDE_DIR ${VIZDOOM_INCLUDE_DIR} ${VIZDOOM_LIB_SRC_DIR})
set(VIZDOOM_OUTPUT_NAME vizdoom)

find_package(Boost 1.65.0 COMPONENTS filesystem thread system date_time chrono regex iostreams REQUIRED)
find_package(Boost 1.53 COMPONENTS filesystem thread system date_time chrono regex iostreams REQUIRED)
find_package(Threads REQUIRED)

include_directories(${VIZDOOM_LIB_INCLUDE_DIR} ${Boost_INCLUDE_DIR})
Expand Down Expand Up @@ -159,7 +162,9 @@ set_target_properties(libvizdoom_shared
# ViZDoom Engine
#-----------------------------------------------------------------------------------------------------------------------

add_subdirectory(${VIZDOOM_SRC_DIR}/vizdoom)
if (BUILD_ENGINE)
add_subdirectory(${VIZDOOM_SRC_DIR}/vizdoom)
endif ()


# Python bindings
Expand Down
45 changes: 22 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,57 +71,56 @@ or
## Python quick start

### Linux
Both x86-64 and ARM64 architectures are supported.
ViZDoom requires C++11 compiler, CMake 3.4+, Boost 1.65+ SDL2, OpenAL (optional) and Python 3.8+. Below you will find instructrion how to install these dependencies.
To install the latest release of ViZDoom, just run:
```
pip install vizdoom
```
Both x86-64 and AArch64 (ARM64) architectures are supported.

If Python wheel is not available for your platform (Python version <3.8, distros below manylinux_2_28 standard), pip will try to install (build) ViZDoom from source.
ViZDoom requires C++11 compiler, CMake 3.12+, Boost 1.54+ SDL2, OpenAL (optional) and Python 3.7+. Below you will find instructrion how to install these dependencies.

#### apt-based distros (Ubuntu, Debian, Linux Mint, etc.)

To install ViZDoom run (may take few minutes):
To build ViZDoom run (it may take few minutes):
```
apt install cmake git libboost-all-dev libsdl2-dev libopenal-dev
pip install vizdoom
```
We recommend using at least Ubuntu 18.04+ or Debian 10+ with Python 3.8+.
We recommend using at least Ubuntu 18.04+ or Debian 10+ with Python 3.7+.

#### dnf/yum-based distros (Fedora, RHEL, CentOS, Alma/Rocky Linux, etc.)

To install ViZDoom run (may take few minutes):
To install ViZDoom run (it may take few minutes):
```
dnf install cmake git boost-devel SDL2-devel openal-soft-devel
pip install vizdoom
```
We recommend using at least Fedora 35+ or RHEL/CentOS/Alma/Rocky Linux 9+ with Python 3.8+. To install openal-soft-devel on RHEL/CentOS/Alma/Rocky Linux 9, one needs to use `dnf --enablerepo=crb install`.

#### Conda-based installation
To install ViZDoom on a conda environment (no system-wide installations required):
```
conda install -c conda-forge boost cmake sdl2
git clone https://github.com/mwydmuch/ViZDoom.git --recurse-submodules
cd ViZDoom
python setup.py build && python setup.py install
```
Note that `pip install vizdoom` won't work with conda install and you have to follow these steps.
We recommend using at least Fedora 35+ or RHEL/CentOS/Alma/Rocky Linux 9+ with Python 3.7+.
To install openal-soft-devel on RHEL/CentOS/Alma/Rocky Linux 9, one needs to use `dnf --enablerepo=crb install`.


### macOS
Both Intel and Apple Silicon CPUs are supported.

To install ViZDoom on run (may take few minutes):
To install the latest release of ViZDoom just run (it may take few minutes as it will build ViZDoom from source on M1/M2 chips):
```
brew install cmake git boost openal-soft sdl2
pip install vizdoom
```
We recommend using at least macOS High Sierra 10.13+ with Python 3.8+.
On Apple Silicon (M1 and M2), make sure you are using Python for Apple Silicon.
Both Intel and Apple Silicon CPUs are supported.
We recommend using at least macOS High Sierra 10.13+ with Python 3.7+.
On Apple Silicon (M1 and M2), make sure you are using Python/Pip for Apple Silicon.


### Windows
To install pre-build release for Windows 10 or 11 64-bit and Python 3.8+ just run (should take few seconds):
To install the latest release of ViZDoom, just run:
```
pip install vizdoom
```
At the moment only x86-64 architecture is supported on Windows.

Please note that the Windows version is not as well-tested as Linux and macOS versions. It can be used for development and testing if you want to conduct experiments on Windows, please consider using [Docker](https://docs.docker.com/docker-for-windows/install/) or [WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
Please note that the Windows version is not as well-tested as Linux and macOS versions.
It can be used for development and testing but if you want to conduct serious (time and resource-extensive) experiments on Windows,
please consider using [Docker](https://docs.docker.com/docker-for-windows/install/) or [WSL](https://docs.microsoft.com/en-us/windows/wsl/install-win10) with Linux version.


### Gymnasium/Gym wrappers
Expand Down
Loading
Loading