Use maturin container (or even action) for manylinux builds

[pechersky]

Thanks for proposing a new feature!

Motivation

The current builds (https://github.com/wasmerio/wasmer-python/actions/runs/4852776764/job/13149022773#step:18:11) are at a high ABI: manylinux_2_34. This is much higher than ABIs that are still supported, like manylinux_2_17. Switching to using an existing maturin container will provide a build context that already supplies Python and Rust.

Proposed solution

Here is the workflow for building the api and the cranelift (as an example):

  build:
    name: Build and Release using a container

    strategy:
      matrix:
        python: ["3.7", "3.8", "3.9", "3.10", "3.11"]
        target:
          - id: 'linux-amd64'
            os: 'ubuntu-latest'
            target-name: 'x86_64-unknown-linux-gnu'
            rust-toolchain: 'stable'
      fail-fast: true

    runs-on: ${{ matrix.target.os }}
    container:
        image: ghcr.io/pyo3/maturin

    steps:
      - name: Check out code
        uses: actions/checkout@v2

      - name: Set up Rust
        run: rustup default ${{ matrix.target.rust-toolchain }}

      - name: Build api
        run: |
            pip${{ matrix.python }} install virtualenv
            virtualenv .env
            source .env/bin/activate
            maturin build --bindings pyo3 --release --target ${{ matrix.target.target-name }} --strip --interpreter ${{ matrix.python }} -m packages/api/Cargo.toml 

      - name: Build cranelist
        run: |
            pip${{ matrix.python }} install virtualenv
            virtualenv .env
            source .env/bin/activate
            maturin build --bindings pyo3 --release --target ${{ matrix.target.target-name }} --strip --interpreter ${{ matrix.python }} -m packages/compiler-cranelift/Cargo.toml 

      - name: Upload artifacts
        uses: actions/upload-artifact@v2
        with:
          name: wheels-${{ matrix.python }}-${{ matrix.target.id }}
          path: target/wheels
          if-no-files-found: error
          retention-days: 1

Some additional work in necessary to make LLVM builds installable.

Alternatives

This allows the maturin team to maintain the containers/toolkit combinations that allow a broad manylinux build possible. Alternatives are switching to a different context without using the container. This means, in either case, rewriting the justfile.

Additional context

Some of the cloud containers out there are on ABIs that are too low for manylinux_2_34. Having broader ABIs allows installing these packages in those context too.

[rbtcollins]

The python3.10 build of wasmer_compiler_llvm links to libtinfo-91270aa7.so.5:

readelf -d .../site-packages/wasmer_compiler_llvm/wasmer_compiler_llvm.cpython-310-x86_64-linux-gnu.so | grep .so.
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libtinfo-91270aa7.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]

I suspect that thats a build artifact as well, and maturin would correct this.