[CI] Introduce driver and nightly containers #4777
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| name: SYCL Nightly Builds | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| schedule: | ||
| - cron: '0 3 * * *' | ||
|
|
||
| jobs: | ||
| ubuntu2004_build_test: | ||
| if: github.repository == 'intel/llvm' | ||
| uses: intel/llvm/.github/workflows/sycl_linux_build_and_test.yml@sycl | ||
| with: | ||
| build_github_cache: true | ||
| build_cache_root: ${{ github.workspace }} | ||
| build_artifact_suffix: default | ||
| ubuntu2004_docker_build_push: | ||
| if: github.repository == 'intel/llvm' | ||
| runs-on: ubuntu-latest | ||
| needs: ubuntu2004_build_test | ||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - uses: actions/download-artifact@v2 | ||
| with: | ||
| name: sycl_nightly_ubuntu2004 | ||
| path: devops/ | ||
| - name: Login to GitHub Container Registry | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.repository_owner }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
| - name: Build and Push Container | ||
| uses: docker/build-push-action@v2 | ||
| with: | ||
| push: true | ||
| build-args: | | ||
| base_image=ghcr.io/intel/llvm/ubuntu2004_intel_drivers | ||
| base_tag=stable | ||
| tags: | | ||
| ghcr.io/${{ github.repository }}/sycl_ubuntu2004_nightly:${{ github.sha }} | ||
| ghcr.io/${{ github.repository }}/sycl_ubuntu2004_nightly:latest | ||
| context: ${{ github.workspace }}/devops | ||
| file: ${{ github.workspace }}/devops/containers/ubuntu2004_preinstalled.Dockerfile | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,138 @@ | ||
| name: Linux Post Commit Checks | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - sycl | ||
| jobs: | ||
| check: | ||
| runs-on: ubuntu-20.04 | ||
|
Contributor
There was a problem hiding this comment. Can't we re-use of the containers to avoid configuring the system from scratch? |
||
| if: github.repository == 'intel/llvm' | ||
| strategy: | ||
|
Contributor
There was a problem hiding this comment. Hm... I thought we can re-use the workflow added by #4773. I see a lot of duplicated commands from .github/workflows/sycl_linux_build_and_test.yml.
Contributor
Author
There was a problem hiding this comment. Yes, that's the intention. I'll fix that before moving PR to ready for review. |
||
| fail-fast: false | ||
| matrix: | ||
| config: ["Default", "SharedLibs", "NoAssertions"] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| with: | ||
| path: src | ||
| - name: Install Ubuntu deps | ||
| run: sudo apt install -y ninja-build ccache | ||
| - name: Setup Cache | ||
| uses: actions/cache@v2 | ||
| id: cache | ||
| with: | ||
| path: ${{ github.workspace }}/cache | ||
| key: build-${{ runner.os }}-${{ matrix.config }}-${{ github.sha }} | ||
| restore-keys: | | ||
| build-${{ runner.os }}-${{ matrix.config }}- | ||
| - name: Configure | ||
| run: | | ||
| CONFIG=${{ matrix.config }} | ||
| case $CONFIG in | ||
| Default) | ||
| export ARGS="" | ||
| ;; | ||
| SharedLibs) | ||
| wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add - | ||
| sudo add-apt-repository "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main" | ||
| sudo apt-get update | ||
| sudo apt-get install -y clang-13 | ||
| export ARGS="--shared-libs" | ||
| export CC="clang-13" | ||
| export CXX="clang++-13" | ||
| ;; | ||
| NoAssertions) | ||
| export ARGS="--no-assertions" | ||
| ;; | ||
| esac | ||
| mkdir -p $GITHUB_WORKSPACE/build | ||
| cd $GITHUB_WORKSPACE/build | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/configure.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build -t Release \ | ||
| --ci-defaults $ARGS --cmake-opt="-DLLVM_CCACHE_BUILD=ON" \ | ||
| --cmake-opt="-DLLVM_CCACHE_DIR=$GITHUB_WORKSPACE/cache" \ | ||
| --cmake-opt="-DLLVM_CCACHE_MAXSIZE=2G" | ||
| - name: Compile | ||
| run: | | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/compile.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build | ||
| - name: check-llvm | ||
| if: always() | ||
| run: | | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/check.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build -t check-llvm | ||
| - name: check-clang | ||
| if: always() | ||
| run: | | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/check.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build -t check-clang | ||
| - name: check-sycl | ||
| if: always() | ||
| run: | | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/check.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build -t check-sycl | ||
| - name: check-llvm-spirv | ||
| if: always() | ||
| run: | | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/check.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build -t check-llvm-spirv | ||
| - name: check-xptifw | ||
| if: always() | ||
| run: | | ||
| python3 $GITHUB_WORKSPACE/src/buildbot/check.py -w $GITHUB_WORKSPACE \ | ||
| -s $GITHUB_WORKSPACE/src -o $GITHUB_WORKSPACE/build -t check-xptifw | ||
| - name: Pack | ||
| run: tar -czvf llvm_sycl.tar.gz -C $GITHUB_WORKSPACE/build/install . | ||
| - name: Upload artifacts | ||
| uses: actions/upload-artifact@v1 | ||
| with: | ||
| name: sycl_linux_${{ matrix.config }} | ||
| path: llvm_sycl.tar.gz | ||
| drivers_image_ubuntu2004: | ||
| name: Intel Drivers Ubuntu 20.04 Docker image (stable versions) | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v2 | ||
| with: | ||
| fetch-depth: 2 | ||
| - name: Login to GitHub Container Registry | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.repository_owner }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
| - name: Check if dependencies have changed | ||
| id: deps_changed | ||
| shell: bash {0} | ||
| run: | | ||
| git diff --exit-code HEAD~1 -- devops/dependencies.json | ||
| echo "::set-output name=flag::$?" | ||
| - name: Get dependencies configuration | ||
| id: get_deps | ||
| if: ${{ steps.deps_changed.outputs.flag }} | ||
| run: | | ||
| DEPS=`cat devops/dependencies.json` | ||
| DEPS="${DEPS//'%'/'%25'}" | ||
| DEPS="${DEPS//$'\n'/'%0A'}" | ||
| DEPS="${DEPS//$'\r'/'%0D'}" | ||
| echo $DEPS | ||
| echo "::set-output name=dependenciesJson::$DEPS" | ||
| - name: Build and Push Container | ||
| uses: docker/build-push-action@v2 | ||
| if: ${{ steps.deps_changed.outputs.flag }} | ||
| with: | ||
| push: true | ||
| build-args: | | ||
| compute_runtime_tag=${{fromJson(steps.get_deps.outputs.dependenciesJson).linux.compute_runtime_tag}} | ||
| igc_tag=${{fromJson(steps.get_deps.outputs.dependenciesJson).linux.igc_tag}} | ||
| tbb_tag=${{fromJson(steps.get_deps.outputs.dependenciesJson).linux.tbb_tag}} | ||
| fpgaemu_tag=${{fromJson(steps.get_deps.outputs.dependenciesJson).linux.fpgaemu_tag}} | ||
| cpu_tag=${{fromJson(steps.get_deps.outputs.dependenciesJson).linux.cpu_tag}} | ||
| tags: | | ||
| ghcr.io/${{ github.repository }}/ubuntu2004_intel_drivers:stable-${{ github.sha }} | ||
| ghcr.io/${{ github.repository }}/ubuntu2004_intel_drivers:stable | ||
| context: ${{ github.workspace }}/devops | ||
| file: ${{ github.workspace }}/devops/containers/ubuntu2004_intel_drivers.Dockerfile | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally, we should save the time on checking out sources by using an image where some old SHA is checked out already and we need just update it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure saving 1 minute on a 1,5 hour build is a significant improvement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe not significant, but still an improvement!
I'm okay to get back to this after we configure everything else.