Skip to content
This repository was archived by the owner on May 9, 2024. It is now read-only.

Fix docker cache #369

Merged
merged 1 commit into from
Apr 11, 2023
Merged

Fix docker cache #369

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 0 additions & 89 deletions .github/workflows/build-docker-hdk.yml
View file
Open in desktop

This file was deleted.

42 changes: 21 additions & 21 deletions .github/workflows/build-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ on:
options:
type: string
default: -DENABLE_CUDA=off
secrets:
DOCKERHUB_PASSWORD:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that secrets are global vars for all github jobs, but after discussion with Alexei I understood that it's special github behavior https://github.blog/changelog/2022-05-03-github-actions-simplify-using-secrets-with-reusable-workflows/ . Maybe there are some WA, because secrets added as https://docs.github.com/en/actions/learn-github-actions/variables#creating-configuration-variables-for-an-environment .

This is only to mark and summarize our discussion.

required: true

jobs:
build:
Expand All @@ -25,47 +28,44 @@ jobs:

- name: Set env context
run: |
set -vx
echo RUN_STAMP=docker-${{ runner.os }}-${{ inputs.name }} >>$GITHUB_ENV
cat omniscidb/docker/dev/Dockerfile omniscidb/docker/dev/Dockerfile.${{ inputs.name }} >Dockerfile
cat docker/Dockerfile docker/Dockerfile.${{ inputs.name }} >Dockerfile
echo DOCKER_NAME=docker.io/dataved/build.${{ inputs.name }}:${{ hashFiles('docker/Dockerfile', 'docker/Dockerfile.${{ inputs.name }}') }} >>$GITHUB_ENV

- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@master

- name: Cache Docker layers
uses: actions/cache@v3
- name: Login to image repository
uses: docker/login-action@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ hashFiles('**/Dockerfile') }}
restore-keys: |
${{ runner.os }}-buildx-

username: dataved
password: ${{ secrets.DOCKERHUB_PASSWORD }}

- name: Test the image
id: test-docker
run: |
docker run -u ghrunner ${{ env.DOCKER_NAME }} dpkg -l
continue-on-error: true

- name: Build image
if: steps.test-docker.outcome != 'success'
uses: docker/build-push-action@v3
with:
context: .
builder: ${{ steps.buildx.outputs.name }}
file: Dockerfile
push: false
load: true
tags: build.${{ inputs.name }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new

# https://github.com/docker/build-push-action/issues/252
- name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
push: true
tags: ${{ env.DOCKER_NAME }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor note. It's possible to reduce amount of pulls from docker.io with caching docker local storage with github cache. As it reduces build time ~ 2 mins we decided to keep current (more simple and direct) approach. Also github cache is limited with 10 GB and current image size is ~5.6 Gb, conda env/ maven should also be cached so caching for docker storage will be checked later.


- name: Start the container
run: |
mkdir -p build
docker run -id --name build.${{ inputs.name }} --network host -v $(pwd):/_work build.${{ inputs.name }}:latest
docker run -id --name build.${{ inputs.name }} --network host -v $(pwd):/_work ${{ env.DOCKER_NAME }}

- name: Configure and build the project
run: |
docker exec -u ghrunner build.${{ inputs.name }} dpkg -l
docker exec -u ghrunner build.${{ inputs.name }} sh /_work/omniscidb/scripts/conda/build.sh ${{ inputs.options }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think maven cache also can used from cache.
docker cp ~/.m2/. verify:/home/ghrunner/.m2 or with -v ${PWD}/.m2:/home/ghrunner/.m2.

docker exec -u ghrunner build.${{ inputs.name }} tar -zcf /tmp/build.tgz -C /_work .
docker cp build.${{ inputs.name }}:/tmp/build.tgz .
Expand Down
6 changes: 0 additions & 6 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,12 +89,6 @@ jobs:
name: ${{ env.RUN_STAMP }}-build
path: /tmp/build.tgz

- name: Upload release
uses: actions/upload-artifact@v3
with:
name: ${{ env.RUN_STAMP }}-release
path: /tmp/release.tgz

- name: Upload logs
if: always()
uses: actions/upload-artifact@v3
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ jobs:
with:
name: cuda
options: -DENABLE_PYTHON=off
secrets:
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}

test-cuda-docker:
needs: build-cuda-docker
Expand All @@ -67,12 +69,16 @@ jobs:
name: cuda
# input for a push event is an empty string, convert it to boolean
reset-cache: ${{ !!inputs.reset-cache }}
secrets:
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}

build-l0-docker:
uses: ./.github/workflows/build-docker.yml
with:
name: l0
options: -DENABLE_L0=on -DENABLE_CUDA=off -DENABLE_PYTHON=off
secrets:
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}

test-l0-docker:
needs: build-l0-docker
Expand All @@ -81,9 +87,6 @@ jobs:
name: l0
reset-cache: ${{ !!inputs.reset-cache }}

#
# sanity-test:
# uses: ./.github/workflows/build-docker-hdk.yml

pytest:
uses: ./.github/workflows/pytest.yml

17 changes: 12 additions & 5 deletions .github/workflows/test-docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ on:
reset-cache:
type: boolean
default: false
secrets:
DOCKERHUB_PASSWORD:
required: true

jobs:
build:
Expand All @@ -37,14 +40,18 @@ jobs:
run: |
tar -zxf build.tgz

- name: Build docker image
run: |
cat omniscidb/docker/dev/Dockerfile omniscidb/docker/dev/Dockerfile.${{ inputs.name }} >Dockerfile
docker build . ${{ inputs.reset-cache && '--no-cache' || '' }} --tag hdk-build.${{ inputs.name }}
- name: Login to image repository
uses: docker/login-action@v2
with:
username: dataved
password: ${{ secrets.DOCKERHUB_PASSWORD }}

- name: Start the container
run: |
docker run -id --name hdk-build.${{ inputs.name }} --network host --device /dev/nvidia-modeset:/dev/nvidia-modeset --device /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools --device /dev/nvidia-uvm:/dev/nvidia-uvm --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl hdk-build.${{ inputs.name }}:latest
cat docker/Dockerfile docker/Dockerfile.${{ inputs.name }} >Dockerfile
DOCKER_NAME=docker.io/dataved/build.${{ inputs.name }}:${{ hashFiles('docker/Dockerfile', 'docker/Dockerfile.${{ inputs.name }}') }}

docker run -id --name hdk-build.${{ inputs.name }} --network host --device /dev/nvidia-modeset:/dev/nvidia-modeset --device /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools --device /dev/nvidia-uvm:/dev/nvidia-uvm --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl $DOCKER_NAME
docker exec hdk-build.${{ inputs.name }} mkdir -p /_work/
docker cp build.tgz hdk-build.${{ inputs.name }}:/_work/
docker exec hdk-build.${{ inputs.name }} chown -R ghrunner:ghrunner /_work/
Expand Down
Loading