Skip to content

Move checkout step to CMake and Conda custom actions and relax version requirements #3461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ramilbakhshyiev wants to merge 3 commits into from
Closed

Move checkout step to CMake and Conda custom actions and relax version requirements #3461

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d420306
Workaround for CUDA 11.4.4 build in Conda on Ubuntu 22 / v6 kernel (#…
ramilbakhshyiev May 20, 2024
28599df
Enable linux-x86_64-GPU-packages-CUDA-11-4-4 build via GitHub Actions…
ramilbakhshyiev May 20, 2024
c10cc23
Move checkout step to CMake and Conda custom actions and relax versio…
ramilbakhshyiev May 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/actions/build_cmake/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ runs:
using: composite
steps:
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@v3.0.3
uses: conda-incubator/setup-miniconda@v3
with:
python-version: '3.11'
miniconda-version: latest
Expand Down Expand Up @@ -97,7 +97,7 @@ runs:
FAISS_DISABLE_CPU_FEATURES=AVX2 LD_DEBUG=libs $CONDA/bin/python -c "import faiss" 2>&1 | grep faiss.so
LD_DEBUG=libs $CONDA/bin/python -c "import faiss" 2>&1 | grep faiss_avx2.so
- name: Upload test results
uses: actions/upload-artifact@v4.3.1
uses: actions/upload-artifact@v4
with:
name: test-results-${{ inputs.opt_level }}-${{ inputs.gpu }}-${{ inputs.raft }}
path: test-results
2 changes: 1 addition & 1 deletion .github/actions/build_conda/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ runs:
echo "shell=pwsh" >> "$GITHUB_OUTPUT"
fi
- name: Setup miniconda
uses: conda-incubator/setup-miniconda@v3.0.3
uses: conda-incubator/setup-miniconda@v3
with:
python-version: '3.11'
miniconda-version: latest
Expand Down
48 changes: 32 additions & 16 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
- name: Install clang-format
run: |
sudo apt-get update -y
Expand All @@ -40,13 +40,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
- uses: ./.github/actions/build_cmake
linux-x86_64-AVX2-cmake:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
- uses: ./.github/actions/build_cmake
with:
opt_level: avx2
Expand All @@ -55,7 +55,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
- uses: ./.github/actions/build_cmake
with:
opt_level: avx512
Expand All @@ -64,15 +64,15 @@ jobs:
runs-on: 4-core-ubuntu-gpu-t4
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
- uses: ./.github/actions/build_cmake
with:
gpu: ON
linux-x86_64-GPU-w-RAFT-cmake:
runs-on: 4-core-ubuntu-gpu-t4
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
- uses: ./.github/actions/build_cmake
with:
gpu: ON
Expand All @@ -81,7 +81,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -90,7 +90,7 @@ jobs:
runs-on: windows-2019
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -99,7 +99,7 @@ jobs:
runs-on: 2-core-ubuntu-arm
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -109,21 +109,37 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
with:
label: main
linux-x86_64-GPU-packages-CUDA-11-4-4:
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "60-real;61-real;62-real;70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
- uses: ./.github/actions/build_conda
with:
label: main
cuda: "11.4.4"
compiler_version: "11.2"
linux-x86_64-GPU-RAFT-packages-CUDA11-8-0:
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
runs-on: 4-core-ubuntu-gpu-t4
env:
CUDA_ARCHS: "70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -140,7 +156,7 @@ jobs:
CUDA_ARCHS: "70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -156,7 +172,7 @@ jobs:
CUDA_ARCHS: "70-real;72-real;75-real;80;86-real"
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -171,7 +187,7 @@ jobs:
runs-on: windows-2019
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -183,7 +199,7 @@ jobs:
runs-on: macos-14
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand All @@ -195,7 +211,7 @@ jobs:
runs-on: 2-core-ubuntu-arm
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
uses: actions/checkout@v4
with:
fetch-depth: 0
fetch-tags: true
Expand Down
6 changes: 6 additions & 0 deletions conda/faiss-gpu/build-lib.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@

set -e

# Workaround for CUDA 11.4.4 builds. Moves all necessary headers to include root.
if [[ -n "$FAISS_FLATTEN_CONDA_INCLUDES" && "$FAISS_FLATTEN_CONDA_INCLUDES" == "1" ]]; then
cp -r -n $CONDA_PREFIX/x86_64-conda-linux-gnu/sysroot/usr/include/* $CONDA_PREFIX/include/
cp -r -n $CONDA_PREFIX/x86_64-conda-linux-gnu/include/c++/11.2.0/* $CONDA_PREFIX/include/
cp -r -n $CONDA_PREFIX/x86_64-conda-linux-gnu/include/c++/11.2.0/x86_64-conda-linux-gnu/* $CONDA_PREFIX/include/
fi

# Build libfaiss.so/libfaiss_avx2.so/libfaiss_avx512.so
cmake -B _build \
Expand Down
3 changes: 3 additions & 0 deletions conda/faiss-gpu/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ outputs:
- {{ pin_compatible('libfaiss', exact=True) }}
script_env:
- CUDA_ARCHS
{% if cudatoolkit == '11.4.4' %}
- FAISS_FLATTEN_CONDA_INCLUDES=1
{% endif %}
requirements:
build:
- {{ compiler('cxx') }}
Expand Down